2024-08-19T01:21:28.769 INFO:root:teuthology version: 1.2.0 2024-08-19T01:21:28.793 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T01:21:28.881 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7860121' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1664 sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 sleep_before_teardown: 0 subset: 36/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHN93diYdhnQC/9XgUFx7GCAJ7CcKQcUJkhDeXgcvIkHC/t26svdBvwbuLcwjfOF+EeRUKmG//Zs8AAFCv/DdO0= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM//RKXb9+0d5Acti2yrK4cIgnMqWLhC131Kf0ailWAMgS5QeFQIAdqpwAr3jZZnTxiIHpDeKmQTjpjpcNQGuOg= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJmwbrYyaypfzuWwodCOh4snjZRKXa7lCwEUhnQakAxcp4+/h3ivLcYBaTbzuyItWL3Mb2wzkHMEgIJitBzJEtI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-18_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-19T01:21:28.882 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa; will attempt to use it 2024-08-19T01:21:28.882 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks 2024-08-19T01:21:28.883 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-19T01:21:28.884 INFO:teuthology.task.internal:Checking packages... 2024-08-19T01:21:28.904 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'de09dd018de4baddfa6ca4d43bc8fb784cf75833' 2024-08-19T01:21:28.904 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-19T01:21:28.904 INFO:teuthology.packaging:ref: None 2024-08-19T01:21:28.904 INFO:teuthology.packaging:tag: None 2024-08-19T01:21:28.904 INFO:teuthology.packaging:branch: squid 2024-08-19T01:21:28.904 INFO:teuthology.packaging:sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:21:28.904 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-08-19T01:21:29.100 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1386.g78215f5a 2024-08-19T01:21:29.101 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-19T01:21:29.108 INFO:teuthology.task.internal:no buildpackages task found 2024-08-19T01:21:29.108 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-19T01:21:29.190 INFO:teuthology.task.internal:Saving configuration 2024-08-19T01:21:29.208 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-19T01:21:29.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-19T01:21:29.274 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 01:16:00.323616', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHN93diYdhnQC/9XgUFx7GCAJ7CcKQcUJkhDeXgcvIkHC/t26svdBvwbuLcwjfOF+EeRUKmG//Zs8AAFCv/DdO0='} 2024-08-19T01:21:29.296 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 01:16:00.324853', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM//RKXb9+0d5Acti2yrK4cIgnMqWLhC131Kf0ailWAMgS5QeFQIAdqpwAr3jZZnTxiIHpDeKmQTjpjpcNQGuOg='} 2024-08-19T01:21:29.322 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-19 01:16:00.322178', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJmwbrYyaypfzuWwodCOh4snjZRKXa7lCwEUhnQakAxcp4+/h3ivLcYBaTbzuyItWL3Mb2wzkHMEgIJitBzJEtI='} 2024-08-19T01:21:29.322 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-19T01:21:29.328 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-19T01:21:29.328 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-19T01:21:29.328 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-19T01:21:29.328 INFO:teuthology.run_tasks:Running task console_log... 2024-08-19T01:21:29.416 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffa9a1d8b80>, signals=[15]) 2024-08-19T01:21:29.416 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-19T01:21:29.425 INFO:teuthology.task.internal:Opening connections... 2024-08-19T01:21:29.425 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:21:29.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:21:29.503 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-08-19T01:21:29.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:21:29.579 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T01:21:29.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:21:29.653 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-19T01:21:29.660 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-08-19T01:21:29.678 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-08-19T01:21:29.679 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-08-19T01:21:29.735 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-08-19T01:21:29.735 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T01:21:29.736 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T01:21:29.737 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-08-19T01:21:29.760 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-08-19T01:21:29.778 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-08-19T01:21:29.779 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-08-19T01:21:29.837 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-08-19T01:21:29.838 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T01:21:29.838 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-08-19T01:21:29.838 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T01:21:29.838 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T01:21:29.838 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T01:21:29.838 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-08-19T01:21:29.862 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-08-19T01:21:29.880 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-08-19T01:21:29.880 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-08-19T01:21:29.938 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-08-19T01:21:29.939 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-19T01:21:29.939 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-08-19T01:21:29.939 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-19T01:21:29.939 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-19T01:21:29.939 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-19T01:21:29.939 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-08-19T01:21:29.960 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-19T01:21:29.972 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-19T01:21:29.985 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-19T01:21:29.985 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-08-19T01:21:29.988 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-08-19T01:21:29.990 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-08-19T01:21:30.006 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-19T01:21:30.014 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-19T01:21:30.014 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-08-19T01:21:30.045 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-08-19T01:21:30.048 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-08-19T01:21:30.152 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-19T01:21:30.220 INFO:teuthology.run_tasks:Running task kernel... 2024-08-19T01:21:30.238 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-19T01:21:30.238 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-19T01:21:30.239 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T01:21:30.239 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T01:21:30.240 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-19T01:21:30.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:21:30.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:21:30.258 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-08-19T01:21:30.258 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-08-19T01:21:30.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:21:30.259 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-08-19T01:21:30.314 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-480.el9.x86_64 2024-08-19T01:21:30.315 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-480.el9.x86_64 2024-08-19T01:21:30.315 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-08-19T01:21:30.316 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-480.el9.x86_64 2024-08-19T01:21:30.316 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.14.0-480.el9.x86_64 2024-08-19T01:21:30.316 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum install -y kernel 2024-08-19T01:21:30.317 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-480.el9.x86_64 2024-08-19T01:21:30.317 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-480.el9.x86_64 2024-08-19T01:21:30.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-08-19T01:21:32.820 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-08-19T01:21:32.873 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.2 MB 00:00 2024-08-19T01:21:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.2 MB 00:01 2024-08-19T01:21:34.604 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-08-19T01:21:34.607 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-08-19T01:21:36.238 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-08-19T01:21:39.014 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.6 MB 00:00 2024-08-19T01:21:40.531 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 18 kB 00:00 2024-08-19T01:21:40.699 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.6 MB 00:02 2024-08-19T01:21:41.393 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 30 MB/s | 22 MB 00:00 2024-08-19T01:21:42.250 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 98 kB/s | 18 kB 00:00 2024-08-19T01:21:42.497 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.6 MB 00:02 2024-08-19T01:21:43.162 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-08-19T01:21:44.139 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 18 kB 00:00 2024-08-19T01:21:45.268 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-19T01:21:46.841 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-08-19T01:21:48.509 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-08-19T01:21:48.797 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-19T01:21:48.797 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-480.el9.x86_64 is already installed. 2024-08-19T01:21:48.874 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T01:21:48.875 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-08-19T01:21:48.875 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-08-19T01:21:48.976 DEBUG:teuthology.orchestra.run.smithi103:> echo no | sudo yum reinstall kernel || true 2024-08-19T01:21:49.536 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:03 ago on Mon 19 Aug 2024 01:21:46 AM UTC. 2024-08-19T01:21:49.669 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalling: 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 1.5 M 2024-08-19T01:21:49.682 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 0 2024-08-19T01:21:49.683 INFO:teuthology.orchestra.run.smithi103.stderr:Operation aborted. 2024-08-19T01:21:49.740 INFO:teuthology.orchestra.run.smithi103.stdout:Is this ok [y/N]: 2024-08-19T01:21:49.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum reinstall -y kernel || true 2024-08-19T01:21:50.258 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:04 ago on Mon 19 Aug 2024 01:21:46 AM UTC. 2024-08-19T01:21:50.382 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T01:21:50.382 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T01:21:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalling: 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 1.5 M 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 0 2024-08-19T01:21:50.383 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-08-19T01:21:50.466 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-19T01:21:50.466 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-480.el9.x86_64 is already installed. 2024-08-19T01:21:50.532 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-08-19T01:21:50.533 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-08-19T01:21:50.533 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-08-19T01:21:50.635 DEBUG:teuthology.orchestra.run.smithi146:> echo no | sudo yum reinstall kernel || true 2024-08-19T01:21:50.649 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-08-19T01:21:51.060 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-480.el9.x86_64.rpm 2.6 MB/s | 1.5 MB 00:00 2024-08-19T01:21:51.060 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:21:51.060 INFO:teuthology.orchestra.run.smithi103.stdout:Total 2.2 MB/s | 1.5 MB 00:00 2024-08-19T01:21:51.099 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-08-19T01:21:51.156 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:03 ago on Mon 19 Aug 2024 01:21:48 AM UTC. 2024-08-19T01:21:51.314 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-08-19T01:21:51.325 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-08-19T01:21:51.325 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-08-19T01:21:51.325 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-08-19T01:21:51.325 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalling: 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 1.5 M 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 0 2024-08-19T01:21:51.326 INFO:teuthology.orchestra.run.smithi146.stderr:Operation aborted. 2024-08-19T01:21:51.346 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-08-19T01:21:51.346 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-08-19T01:21:51.370 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-08-19T01:21:51.370 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-08-19T01:21:51.384 INFO:teuthology.orchestra.run.smithi146.stdout:Is this ok [y/N]: 2024-08-19T01:21:51.384 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum reinstall -y kernel || true 2024-08-19T01:21:51.522 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-08-19T01:21:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: Reinstalling : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T01:21:51.620 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:51.910 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:03 ago on Mon 19 Aug 2024 01:21:48 AM UTC. 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalling: 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:21:52.034 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-08-19T01:21:52.035 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-08-19T01:21:52.035 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:21:52.035 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 1.5 M 2024-08-19T01:21:52.035 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 0 2024-08-19T01:21:52.035 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-08-19T01:21:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:52.355 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T01:21:52.605 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-19T01:21:52.605 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-480.el9.x86_64 is already installed. 2024-08-19T01:21:52.680 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-08-19T01:21:52.681 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2024-08-19T01:21:52.681 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-08-19T01:21:52.711 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-480.el9.x86_64.rpm 2.6 MB/s | 1.5 MB 00:00 2024-08-19T01:21:52.711 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:21:52.711 INFO:teuthology.orchestra.run.smithi146.stdout:Total 2.2 MB/s | 1.5 MB 00:00 2024-08-19T01:21:52.742 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-08-19T01:21:52.799 DEBUG:teuthology.orchestra.run.smithi190:> echo no | sudo yum reinstall kernel || true 2024-08-19T01:21:52.989 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-08-19T01:21:52.989 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-08-19T01:21:53.013 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-08-19T01:21:53.014 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-08-19T01:21:53.126 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:53.126 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:21:53.126 INFO:teuthology.orchestra.run.smithi103.stdout:Reinstalled: 2024-08-19T01:21:53.126 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-19T01:21:53.126 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:21:53.126 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-08-19T01:21:53.178 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-08-19T01:21:53.271 INFO:teuthology.orchestra.run.smithi146.stdout: Reinstalling : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T01:21:53.272 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:53.291 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-08-19T01:21:53.347 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-19T01:21:53.347 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-19T01:21:53.347 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-19T01:21:53.348 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-19T01:21:53.348 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-19T01:21:53.363 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:03 ago on Mon 19 Aug 2024 01:21:50 AM UTC. 2024-08-19T01:21:53.374 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-08-19T01:21:53.395 DEBUG:teuthology.parallel:result is None 2024-08-19T01:21:53.500 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-08-19T01:21:53.504 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout:Reinstalling: 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 1.5 M 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 0 2024-08-19T01:21:53.505 INFO:teuthology.orchestra.run.smithi190.stderr:Operation aborted. 2024-08-19T01:21:53.562 INFO:teuthology.orchestra.run.smithi190.stdout:Is this ok [y/N]: 2024-08-19T01:21:53.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum reinstall -y kernel || true 2024-08-19T01:21:53.957 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:53.958 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T01:21:54.085 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:00:04 ago on Mon 19 Aug 2024 01:21:50 AM UTC. 2024-08-19T01:21:54.208 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-08-19T01:21:54.209 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-08-19T01:21:54.209 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-08-19T01:21:54.209 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-08-19T01:21:54.209 INFO:teuthology.orchestra.run.smithi190.stdout:Reinstalling: 2024-08-19T01:21:54.209 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-19T01:21:54.210 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:21:54.210 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-08-19T01:21:54.210 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-08-19T01:21:54.210 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:21:54.210 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 1.5 M 2024-08-19T01:21:54.210 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 0 2024-08-19T01:21:54.210 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-08-19T01:21:54.567 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-480.el9.x86_64.rpm 5.8 MB/s | 1.5 MB 00:00 2024-08-19T01:21:54.567 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:21:54.567 INFO:teuthology.orchestra.run.smithi190.stdout:Total 4.2 MB/s | 1.5 MB 00:00 2024-08-19T01:21:54.599 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-08-19T01:21:54.723 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:54.724 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:21:54.724 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalled: 2024-08-19T01:21:54.724 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-19T01:21:54.724 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:21:54.724 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-08-19T01:21:54.832 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-08-19T01:21:54.832 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-08-19T01:21:54.856 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-08-19T01:21:54.857 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-08-19T01:21:54.911 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-08-19T01:21:54.967 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-19T01:21:54.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-19T01:21:54.968 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-19T01:21:54.968 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-19T01:21:54.968 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-19T01:21:55.004 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2024-08-19T01:21:55.008 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-08-19T01:21:55.033 DEBUG:teuthology.parallel:result is None 2024-08-19T01:21:55.119 INFO:teuthology.orchestra.run.smithi190.stdout: Reinstalling : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T01:21:55.119 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-19T01:21:56.603 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-19T01:21:56.603 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:21:56.603 INFO:teuthology.orchestra.run.smithi190.stdout:Reinstalled: 2024-08-19T01:21:56.603 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-19T01:21:56.603 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:21:56.603 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-08-19T01:21:56.772 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-08-19T01:21:56.829 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-19T01:21:56.829 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-19T01:21:56.829 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-19T01:21:56.829 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-08-19T01:21:56.830 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-19T01:21:56.858 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-08-19T01:21:56.885 DEBUG:teuthology.parallel:result is None 2024-08-19T01:21:56.885 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-19T01:21:56.892 INFO:teuthology.task.internal:Creating test directory... 2024-08-19T01:21:56.892 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T01:21:56.895 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T01:21:56.897 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-19T01:21:56.940 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-19T01:21:56.949 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-19T01:21:56.970 INFO:teuthology.task.internal:Creating archive directory... 2024-08-19T01:21:56.970 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T01:21:56.973 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T01:21:56.975 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-19T01:21:57.007 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-19T01:21:57.015 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-19T01:21:57.016 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T01:21:57.044 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T01:21:57.047 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-19T01:21:57.075 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:21:57.076 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:21:57.085 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:21:57.087 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:21:57.088 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:21:57.100 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-19T01:21:57.102 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-19T01:21:57.109 INFO:teuthology.task.internal:Configuring sudo... 2024-08-19T01:21:57.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T01:21:57.112 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T01:21:57.132 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-19T01:21:57.157 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-19T01:21:57.172 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-19T01:21:57.172 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T01:21:57.180 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T01:21:57.200 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-19T01:21:57.215 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:21:57.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:21:57.341 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:21:57.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:21:57.425 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:21:57.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T01:21:57.490 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:21:57.534 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:21:57.580 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:21:57.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:21:57.705 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:21:57.705 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T01:21:57.771 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:21:57.816 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-19T01:21:57.875 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:21:57.955 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-19T01:21:58.002 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:21:58.002 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-19T01:21:58.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-08-19T01:21:58.071 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-08-19T01:21:58.073 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-08-19T01:21:58.112 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:21:58.116 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:21:58.150 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:21:58.669 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-19T01:21:58.678 INFO:teuthology.task.internal:Starting timer... 2024-08-19T01:21:58.678 INFO:teuthology.run_tasks:Running task pcp... 2024-08-19T01:21:58.700 INFO:teuthology.run_tasks:Running task selinux... 2024-08-19T01:21:58.749 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-08-19T01:21:58.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-08-19T01:21:58.808 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-08-19T01:21:58.810 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-08-19T01:21:58.864 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-08-19T01:21:58.865 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-08-19T01:21:58.923 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-08-19T01:21:58.926 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-19T01:21:58.926 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-08-19T01:21:58.959 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-08-19T01:21:58.959 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-08-19T01:21:58.985 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-08-19T01:21:58.985 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-08-19T01:21:59.012 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-08-19T01:21:59.012 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-08-19T01:21:59.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:21:59.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:21:59.040 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:21:59.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:21:59.065 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:21:59.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:21:59.093 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-19T01:21:59.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-08-19T01:21:59.119 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-08-19T01:21:59.154 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-08-19T01:21:59.182 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-19T01:21:59.193 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-19T01:21:59.196 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-19T01:21:59.302 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-19T01:21:59.312 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-19T01:21:59.339 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi146.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-19T01:23:43.181 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-08-19T01:23:43.183 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-08-19T01:23:43.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:23:43.262 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-08-19T01:23:43.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-08-19T01:23:43.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-19T01:23:43.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:23:43.427 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-08-19T01:23:43.506 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-19T01:23:43.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T01:23:43.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-19T01:23:43.582 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-08-19T01:23:43.662 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-08-19T01:23:43.662 INFO:teuthology.run_tasks:Running task clock... 2024-08-19T01:23:43.672 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-19T01:23:43.673 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T01:23:43.673 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:23:43.676 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T01:23:43.676 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:23:43.678 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-19T01:23:43.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:23:43.704 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T01:23:43.706 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T01:23:43.718 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T01:23:43.722 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T01:23:43.733 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-19T01:23:43.743 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-08-19T01:23:43.746 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-08-19T01:23:43.747 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-19T01:23:43.754 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-08-19T01:23:43.757 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-08-19T01:23:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T01:23:43.772 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-08-19T01:23:43.773 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T01:23:43.782 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T01:23:43.783 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-08-19T01:23:43.786 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T01:23:43.797 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-19T01:23:43.811 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-19T01:23:43.840 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:23:43.843 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:23:43.843 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-08-19T01:23:43.843 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.843 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.844 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.844 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.845 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:23:43.848 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:23:43.848 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-08-19T01:23:43.848 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.848 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.848 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.848 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.876 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:23:43.881 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:23:43.881 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-08-19T01:23:43.881 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.881 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.881 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.881 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:23:43.881 INFO:teuthology.run_tasks:Running task install... 2024-08-19T01:23:43.889 DEBUG:teuthology.task.install:project ceph 2024-08-19T01:23:43.889 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'}} 2024-08-19T01:23:43.889 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'} 2024-08-19T01:23:43.889 INFO:teuthology.task.install:Using flavor: default 2024-08-19T01:23:43.895 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-19T01:23:43.895 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-19T01:23:43.895 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T01:23:43.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:23:43.898 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T01:23:43.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:23:43.899 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-19T01:23:43.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:23:44.052 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T01:23:44.052 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T01:23:44.057 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T01:23:44.058 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T01:23:44.060 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-19T01:23:44.061 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-19T01:23:44.161 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T01:23:44.161 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:23:44.161 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T01:23:44.172 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T01:23:44.172 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:23:44.172 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T01:23:44.187 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-19T01:23:44.187 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:23:44.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-19T01:23:44.189 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-19T01:23:44.189 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T01:23:44.198 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-19T01:23:44.198 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T01:23:44.215 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-19T01:23:44.215 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-19T01:23:44.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T01:23:44.264 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T01:23:44.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-19T01:23:44.330 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T01:23:44.339 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T01:23:44.358 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-19T01:23:44.394 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-08-19T01:23:44.395 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-08-19T01:23:44.403 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-08-19T01:23:44.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-08-19T01:23:44.422 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-08-19T01:23:44.424 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-08-19T01:23:44.619 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-08-19T01:23:44.630 INFO:teuthology.orchestra.run.smithi146.stdout:45 files removed 2024-08-19T01:23:44.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-19T01:23:44.648 INFO:teuthology.orchestra.run.smithi190.stdout:45 files removed 2024-08-19T01:23:44.658 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-19T01:23:44.675 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-19T01:23:45.200 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-08-19T01:23:45.201 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-08-19T01:23:45.235 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 290 kB/s | 84 kB 00:00 2024-08-19T01:23:45.494 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-08-19T01:23:45.519 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-08-19T01:23:45.535 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-08-19T01:23:45.769 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-08-19T01:23:45.787 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-08-19T01:23:45.793 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 8.6 kB/s | 1.9 kB 00:00 2024-08-19T01:23:46.345 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-08-19T01:23:47.261 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.2 MB 00:01 2024-08-19T01:23:47.360 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.2 MB 00:01 2024-08-19T01:23:49.196 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-08-19T01:23:50.455 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 20 MB 00:03 2024-08-19T01:23:51.428 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:02 2024-08-19T01:23:54.563 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.6 MB 00:01 2024-08-19T01:23:55.214 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.6 MB 00:01 2024-08-19T01:23:55.870 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.6 MB 00:00 2024-08-19T01:23:56.155 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 18 kB 00:00 2024-08-19T01:23:56.723 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 18 kB 00:00 2024-08-19T01:23:57.276 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-19T01:23:57.387 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 18 kB 00:00 2024-08-19T01:23:57.860 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-19T01:23:58.235 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-08-19T01:24:02.706 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-08-19T01:24:03.282 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-08-19T01:24:03.624 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-08-19T01:24:04.900 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:24:04.901 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:24:04.912 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-19T01:24:04.913 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-19T01:24:04.914 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-19T01:24:04.915 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-19T01:24:04.916 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-19T01:24:04.919 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-19T01:24:04.920 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-19T01:24:04.980 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-08-19T01:24:04.988 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-08-19T01:24:04.988 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-08-19T01:24:04.988 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-08-19T01:24:04.988 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-08-19T01:24:04.988 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-19T01:24:04.988 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T01:24:04.989 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-19T01:24:04.990 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-19T01:24:04.991 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-19T01:24:04.991 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-19T01:24:04.991 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-19T01:24:04.991 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-19T01:24:04.991 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T01:24:04.991 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T01:24:04.992 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T01:24:04.992 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T01:24:04.992 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T01:24:04.992 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T01:24:04.992 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T01:24:04.992 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-19T01:24:04.992 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-08-19T01:24:04.993 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T01:24:04.993 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T01:24:04.993 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-08-19T01:24:04.993 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T01:24:04.993 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-19T01:24:04.993 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T01:24:04.993 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T01:24:04.994 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T01:24:04.995 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T01:24:04.995 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T01:24:04.995 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T01:24:04.995 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T01:24:04.995 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T01:24:04.995 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-19T01:24:04.996 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T01:24:04.997 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T01:24:04.998 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T01:24:04.999 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T01:24:05.000 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T01:24:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T01:24:05.002 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout:Install 148 Packages 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 2 Packages 2024-08-19T01:24:05.003 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:05.004 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 218 M 2024-08-19T01:24:05.004 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-08-19T01:24:05.433 INFO:teuthology.orchestra.run.smithi146.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 55 kB/s | 6.6 kB 00:00 2024-08-19T01:24:05.476 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:24:05.476 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:24:05.488 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-19T01:24:05.489 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-19T01:24:05.489 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-19T01:24:05.490 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-19T01:24:05.492 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-19T01:24:05.494 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-19T01:24:05.495 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-19T01:24:05.556 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-08-19T01:24:05.564 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T01:24:05.565 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-19T01:24:05.566 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-08-19T01:24:05.567 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T01:24:05.568 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T01:24:05.569 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T01:24:05.569 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T01:24:05.569 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T01:24:05.569 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T01:24:05.569 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T01:24:05.569 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T01:24:05.570 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T01:24:05.570 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T01:24:05.570 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T01:24:05.570 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T01:24:05.570 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T01:24:05.571 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T01:24:05.571 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T01:24:05.571 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T01:24:05.571 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T01:24:05.571 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T01:24:05.571 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-19T01:24:05.572 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T01:24:05.573 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T01:24:05.574 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T01:24:05.575 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T01:24:05.576 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T01:24:05.577 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-08-19T01:24:05.578 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T01:24:05.579 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:05.579 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-08-19T01:24:05.579 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-08-19T01:24:05.579 INFO:teuthology.orchestra.run.smithi190.stdout:Install 148 Packages 2024-08-19T01:24:05.579 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 2 Packages 2024-08-19T01:24:05.579 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:05.579 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 218 M 2024-08-19T01:24:05.580 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-08-19T01:24:05.666 INFO:teuthology.orchestra.run.smithi146.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 3.8 MB/s | 896 kB 00:00 2024-08-19T01:24:05.716 INFO:teuthology.orchestra.run.smithi146.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 2.8 MB/s | 145 kB 00:00 2024-08-19T01:24:05.871 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:24:05.872 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-19T01:24:05.886 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-19T01:24:05.886 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-19T01:24:05.888 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-19T01:24:05.889 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-19T01:24:05.891 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-19T01:24:05.894 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-19T01:24:05.895 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-19T01:24:05.958 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-08-19T01:24:05.966 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-08-19T01:24:05.966 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-08-19T01:24:05.966 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-08-19T01:24:05.966 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-19T01:24:05.967 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-19T01:24:05.968 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-19T01:24:05.969 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-19T01:24:05.970 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-19T01:24:05.971 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-19T01:24:05.972 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-19T01:24:05.973 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-19T01:24:05.974 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-19T01:24:05.975 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-19T01:24:05.975 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-19T01:24:05.975 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-19T01:24:05.976 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-19T01:24:05.977 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-19T01:24:05.978 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-08-19T01:24:05.979 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-08-19T01:24:05.980 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-08-19T01:24:05.980 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 2 Packages 2024-08-19T01:24:05.980 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:05.980 INFO:teuthology.orchestra.run.smithi146.stdout:(4/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 9.2 MB/s | 2.4 MB 00:00 2024-08-19T01:24:05.980 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 218 M 2024-08-19T01:24:05.981 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-08-19T01:24:06.051 INFO:teuthology.orchestra.run.smithi146.stdout:(5/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 7.3 MB/s | 5.4 MB 00:00 2024-08-19T01:24:06.092 INFO:teuthology.orchestra.run.smithi190.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 46 kB/s | 6.6 kB 00:00 2024-08-19T01:24:06.170 INFO:teuthology.orchestra.run.smithi146.stdout:(6/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 7.9 MB/s | 1.5 MB 00:00 2024-08-19T01:24:06.400 INFO:teuthology.orchestra.run.smithi190.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 2.8 MB/s | 896 kB 00:00 2024-08-19T01:24:06.409 INFO:teuthology.orchestra.run.smithi146.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 13 MB/s | 4.7 MB 00:00 2024-08-19T01:24:06.450 INFO:teuthology.orchestra.run.smithi190.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 2.8 MB/s | 145 kB 00:00 2024-08-19T01:24:06.851 INFO:teuthology.orchestra.run.smithi103.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-08-19T01:24:07.017 INFO:teuthology.orchestra.run.smithi190.stdout:(4/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 4.2 MB/s | 2.4 MB 00:00 2024-08-19T01:24:07.101 INFO:teuthology.orchestra.run.smithi190.stdout:(5/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 4.7 MB/s | 5.4 MB 00:01 2024-08-19T01:24:07.109 INFO:teuthology.orchestra.run.smithi146.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 12 MB/s | 21 MB 00:01 2024-08-19T01:24:07.160 INFO:teuthology.orchestra.run.smithi103.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 2.8 MB/s | 896 kB 00:00 2024-08-19T01:24:07.176 INFO:teuthology.orchestra.run.smithi146.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 378 kB/s | 25 kB 00:00 2024-08-19T01:24:07.260 INFO:teuthology.orchestra.run.smithi103.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 1.4 MB/s | 145 kB 00:00 2024-08-19T01:24:07.276 INFO:teuthology.orchestra.run.smithi190.stdout:(6/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 5.9 MB/s | 1.5 MB 00:00 2024-08-19T01:24:07.335 INFO:teuthology.orchestra.run.smithi146.stdout:(10/150): ceph-radosgw-19.1.0-1382.gde09dd01.el 11 MB/s | 11 MB 00:00 2024-08-19T01:24:07.443 INFO:teuthology.orchestra.run.smithi146.stdout:(11/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 2.0 MB/s | 225 kB 00:00 2024-08-19T01:24:07.669 INFO:teuthology.orchestra.run.smithi146.stdout:(12/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 11 MB/s | 17 MB 00:01 2024-08-19T01:24:07.694 INFO:teuthology.orchestra.run.smithi146.stdout:(13/150): cephfs-shell-19.1.0-1382.gde09dd01.el 108 kB/s | 27 kB 00:00 2024-08-19T01:24:07.752 INFO:teuthology.orchestra.run.smithi146.stdout:(14/150): libcephfs-devel-19.1.0-1382.gde09dd01 381 kB/s | 32 kB 00:00 2024-08-19T01:24:07.760 INFO:teuthology.orchestra.run.smithi190.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 7.1 MB/s | 4.7 MB 00:00 2024-08-19T01:24:07.803 INFO:teuthology.orchestra.run.smithi146.stdout:(15/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 6.8 MB/s | 753 kB 00:00 2024-08-19T01:24:07.836 INFO:teuthology.orchestra.run.smithi146.stdout:(16/150): libcephsqlite-19.1.0-1382.gde09dd01.e 1.9 MB/s | 163 kB 00:00 2024-08-19T01:24:07.937 INFO:teuthology.orchestra.run.smithi146.stdout:(17/150): librados-devel-19.1.0-1382.gde09dd01. 947 kB/s | 127 kB 00:00 2024-08-19T01:24:08.185 INFO:teuthology.orchestra.run.smithi103.stdout:(4/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-08-19T01:24:08.319 INFO:teuthology.orchestra.run.smithi103.stdout:(5/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-08-19T01:24:08.518 INFO:teuthology.orchestra.run.smithi190.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 8.3 MB/s | 21 MB 00:02 2024-08-19T01:24:08.612 INFO:teuthology.orchestra.run.smithi190.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 268 kB/s | 25 kB 00:00 2024-08-19T01:24:08.712 INFO:teuthology.orchestra.run.smithi146.stdout:(18/150): libradosstriper1-19.1.0-1382.gde09dd0 579 kB/s | 507 kB 00:00 2024-08-19T01:24:08.771 INFO:teuthology.orchestra.run.smithi190.stdout:(10/150): ceph-radosgw-19.1.0-1382.gde09dd01.el 10 MB/s | 11 MB 00:01 2024-08-19T01:24:08.846 INFO:teuthology.orchestra.run.smithi190.stdout:(11/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 2.9 MB/s | 225 kB 00:00 2024-08-19T01:24:08.854 INFO:teuthology.orchestra.run.smithi146.stdout:(19/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-08-19T01:24:08.879 INFO:teuthology.orchestra.run.smithi146.stdout:(20/150): python3-ceph-argparse-19.1.0-1382.gde 270 kB/s | 45 kB 00:00 2024-08-19T01:24:08.885 INFO:teuthology.orchestra.run.smithi103.stdout:(6/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-08-19T01:24:08.930 INFO:teuthology.orchestra.run.smithi190.stdout:(12/150): cephfs-shell-19.1.0-1382.gde09dd01.el 325 kB/s | 27 kB 00:00 2024-08-19T01:24:08.946 INFO:teuthology.orchestra.run.smithi146.stdout:(21/150): python3-ceph-common-19.1.0-1382.gde09 1.4 MB/s | 136 kB 00:00 2024-08-19T01:24:09.022 INFO:teuthology.orchestra.run.smithi190.stdout:(13/150): libcephfs-devel-19.1.0-1382.gde09dd01 346 kB/s | 32 kB 00:00 2024-08-19T01:24:09.046 INFO:teuthology.orchestra.run.smithi146.stdout:(22/150): python3-rados-19.1.0-1382.gde09dd01.e 3.6 MB/s | 369 kB 00:00 2024-08-19T01:24:09.130 INFO:teuthology.orchestra.run.smithi190.stdout:(14/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 6.8 MB/s | 753 kB 00:00 2024-08-19T01:24:09.180 INFO:teuthology.orchestra.run.smithi146.stdout:(23/150): python3-cephfs-19.1.0-1382.gde09dd01. 626 kB/s | 188 kB 00:00 2024-08-19T01:24:09.214 INFO:teuthology.orchestra.run.smithi190.stdout:(15/150): libcephsqlite-19.1.0-1382.gde09dd01.e 1.9 MB/s | 163 kB 00:00 2024-08-19T01:24:09.281 INFO:teuthology.orchestra.run.smithi190.stdout:(16/150): librados-devel-19.1.0-1382.gde09dd01. 1.9 MB/s | 127 kB 00:00 2024-08-19T01:24:09.305 INFO:teuthology.orchestra.run.smithi146.stdout:(24/150): python3-rgw-19.1.0-1382.gde09dd01.el9 933 kB/s | 117 kB 00:00 2024-08-19T01:24:09.356 INFO:teuthology.orchestra.run.smithi190.stdout:(17/150): libradosstriper1-19.1.0-1382.gde09dd0 6.6 MB/s | 507 kB 00:00 2024-08-19T01:24:09.389 INFO:teuthology.orchestra.run.smithi146.stdout:(25/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.0 MB/s | 85 kB 00:00 2024-08-19T01:24:09.402 INFO:teuthology.orchestra.run.smithi103.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 4.3 MB/s | 4.7 MB 00:01 2024-08-19T01:24:09.564 INFO:teuthology.orchestra.run.smithi146.stdout:(26/150): python3-rbd-19.1.0-1382.gde09dd01.el9 676 kB/s | 350 kB 00:00 2024-08-19T01:24:09.581 INFO:teuthology.orchestra.run.smithi190.stdout:(18/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 7.3 MB/s | 17 MB 00:02 2024-08-19T01:24:09.639 INFO:teuthology.orchestra.run.smithi146.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 2.2 MB/s | 170 kB 00:00 2024-08-19T01:24:09.656 INFO:teuthology.orchestra.run.smithi190.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 604 kB/s | 45 kB 00:00 2024-08-19T01:24:09.698 INFO:teuthology.orchestra.run.smithi146.stdout:(28/150): ceph-grafana-dashboards-19.1.0-1382.g 545 kB/s | 32 kB 00:00 2024-08-19T01:24:09.732 INFO:teuthology.orchestra.run.smithi190.stdout:(20/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 14 MB/s | 5.4 MB 00:00 2024-08-19T01:24:09.748 INFO:teuthology.orchestra.run.smithi190.stdout:(21/150): python3-ceph-common-19.1.0-1382.gde09 1.4 MB/s | 136 kB 00:00 2024-08-19T01:24:09.756 INFO:teuthology.orchestra.run.smithi146.stdout:(29/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 2.4 MB/s | 146 kB 00:00 2024-08-19T01:24:09.799 INFO:teuthology.orchestra.run.smithi190.stdout:(22/150): python3-cephfs-19.1.0-1382.gde09dd01. 2.8 MB/s | 188 kB 00:00 2024-08-19T01:24:09.824 INFO:teuthology.orchestra.run.smithi190.stdout:(23/150): python3-rados-19.1.0-1382.gde09dd01.e 4.8 MB/s | 369 kB 00:00 2024-08-19T01:24:09.883 INFO:teuthology.orchestra.run.smithi190.stdout:(24/150): python3-rbd-19.1.0-1382.gde09dd01.el9 4.1 MB/s | 350 kB 00:00 2024-08-19T01:24:09.908 INFO:teuthology.orchestra.run.smithi190.stdout:(25/150): python3-rgw-19.1.0-1382.gde09dd01.el9 1.4 MB/s | 117 kB 00:00 2024-08-19T01:24:09.958 INFO:teuthology.orchestra.run.smithi190.stdout:(26/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.1 MB/s | 85 kB 00:00 2024-08-19T01:24:10.059 INFO:teuthology.orchestra.run.smithi190.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 1.7 MB/s | 170 kB 00:00 2024-08-19T01:24:10.150 INFO:teuthology.orchestra.run.smithi190.stdout:(28/150): ceph-grafana-dashboards-19.1.0-1382.g 348 kB/s | 32 kB 00:00 2024-08-19T01:24:10.242 INFO:teuthology.orchestra.run.smithi190.stdout:(29/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 1.6 MB/s | 146 kB 00:00 2024-08-19T01:24:10.490 INFO:teuthology.orchestra.run.smithi146.stdout:(30/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 2.8 MB/s | 3.1 MB 00:01 2024-08-19T01:24:10.549 INFO:teuthology.orchestra.run.smithi146.stdout:(31/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 4.5 MB/s | 3.6 MB 00:00 2024-08-19T01:24:10.593 INFO:teuthology.orchestra.run.smithi190.stdout:(30/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 10 MB/s | 3.6 MB 00:00 2024-08-19T01:24:10.624 INFO:teuthology.orchestra.run.smithi146.stdout:(32/150): ceph-mgr-modules-core-19.1.0-1382.gde 3.3 MB/s | 249 kB 00:00 2024-08-19T01:24:10.733 INFO:teuthology.orchestra.run.smithi146.stdout:(33/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 454 kB/s | 49 kB 00:00 2024-08-19T01:24:10.768 INFO:teuthology.orchestra.run.smithi190.stdout:(31/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 3.6 MB/s | 3.1 MB 00:00 2024-08-19T01:24:10.834 INFO:teuthology.orchestra.run.smithi146.stdout:(34/150): ceph-prometheus-alerts-19.1.0-1382.gd 164 kB/s | 16 kB 00:00 2024-08-19T01:24:11.343 INFO:teuthology.orchestra.run.smithi190.stdout:(32/150): ceph-mgr-diskprediction-local-19.1.0- 9.9 MB/s | 7.4 MB 00:00 2024-08-19T01:24:11.435 INFO:teuthology.orchestra.run.smithi190.stdout:(33/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 538 kB/s | 49 kB 00:00 2024-08-19T01:24:11.477 INFO:teuthology.orchestra.run.smithi190.stdout:(34/150): ceph-mgr-modules-core-19.1.0-1382.gde 352 kB/s | 249 kB 00:00 2024-08-19T01:24:11.510 INFO:teuthology.orchestra.run.smithi190.stdout:(35/150): ceph-prometheus-alerts-19.1.0-1382.gd 219 kB/s | 16 kB 00:00 2024-08-19T01:24:11.517 INFO:teuthology.orchestra.run.smithi146.stdout:(35/150): ceph-volume-19.1.0-1382.gde09dd01.el9 417 kB/s | 285 kB 00:00 2024-08-19T01:24:11.677 INFO:teuthology.orchestra.run.smithi190.stdout:(36/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 4.5 MB/s | 766 kB 00:00 2024-08-19T01:24:11.778 INFO:teuthology.orchestra.run.smithi190.stdout:(37/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 182 kB/s | 18 kB 00:00 2024-08-19T01:24:11.809 INFO:teuthology.orchestra.run.smithi146.stdout:(36/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 2.6 MB/s | 766 kB 00:00 2024-08-19T01:24:11.909 INFO:teuthology.orchestra.run.smithi146.stdout:(37/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 183 kB/s | 18 kB 00:00 2024-08-19T01:24:11.962 INFO:teuthology.orchestra.run.smithi190.stdout:(38/150): ceph-volume-19.1.0-1382.gde09dd01.el9 589 kB/s | 285 kB 00:00 2024-08-19T01:24:12.129 INFO:teuthology.orchestra.run.smithi190.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 228 kB/s | 80 kB 00:00 2024-08-19T01:24:12.244 INFO:teuthology.orchestra.run.smithi103.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 3.7 MB/s | 21 MB 00:05 2024-08-19T01:24:12.320 INFO:teuthology.orchestra.run.smithi190.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-08-19T01:24:12.386 INFO:teuthology.orchestra.run.smithi103.stdout:(9/150): ceph-radosgw-19.1.0-1382.gde09dd01.el9 3.5 MB/s | 11 MB 00:02 2024-08-19T01:24:12.411 INFO:teuthology.orchestra.run.smithi103.stdout:(10/150): ceph-selinux-19.1.0-1382.gde09dd01.el 151 kB/s | 25 kB 00:00 2024-08-19T01:24:12.434 INFO:teuthology.orchestra.run.smithi146.stdout:(38/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 9.3 MB/s | 49 MB 00:05 2024-08-19T01:24:12.526 INFO:teuthology.orchestra.run.smithi146.stdout:(39/150): ceph-mgr-diskprediction-local-19.1.0- 3.6 MB/s | 7.4 MB 00:02 2024-08-19T01:24:12.545 INFO:teuthology.orchestra.run.smithi103.stdout:(11/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 1.6 MB/s | 225 kB 00:00 2024-08-19T01:24:12.628 INFO:teuthology.orchestra.run.smithi103.stdout:(12/150): cephfs-shell-19.1.0-1382.gde09dd01.el 324 kB/s | 27 kB 00:00 2024-08-19T01:24:12.720 INFO:teuthology.orchestra.run.smithi103.stdout:(13/150): libcephfs-devel-19.1.0-1382.gde09dd01 346 kB/s | 32 kB 00:00 2024-08-19T01:24:12.738 INFO:teuthology.orchestra.run.smithi190.stdout:(41/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 12 MB/s | 49 MB 00:04 2024-08-19T01:24:12.763 INFO:teuthology.orchestra.run.smithi190.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 573 kB/s | 253 kB 00:00 2024-08-19T01:24:12.863 INFO:teuthology.orchestra.run.smithi190.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-08-19T01:24:12.922 INFO:teuthology.orchestra.run.smithi190.stdout:(44/150): libgfortran-11.4.1-3.el9.x86_64.rpm 837 kB/s | 803 kB 00:00 2024-08-19T01:24:12.968 INFO:teuthology.orchestra.run.smithi146.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 75 kB/s | 80 kB 00:01 2024-08-19T01:24:13.022 INFO:teuthology.orchestra.run.smithi190.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-08-19T01:24:13.047 INFO:teuthology.orchestra.run.smithi190.stdout:(46/150): python3-pycparser-2.20-6.el9.noarch.r 735 kB/s | 135 kB 00:00 2024-08-19T01:24:13.071 INFO:teuthology.orchestra.run.smithi103.stdout:(14/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 2.1 MB/s | 753 kB 00:00 2024-08-19T01:24:13.131 INFO:teuthology.orchestra.run.smithi190.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-19T01:24:13.163 INFO:teuthology.orchestra.run.smithi103.stdout:(15/150): libcephsqlite-19.1.0-1382.gde09dd01.e 1.7 MB/s | 163 kB 00:00 2024-08-19T01:24:13.263 INFO:teuthology.orchestra.run.smithi103.stdout:(16/150): librados-devel-19.1.0-1382.gde09dd01. 1.2 MB/s | 127 kB 00:00 2024-08-19T01:24:13.327 INFO:teuthology.orchestra.run.smithi146.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 246 kB/s | 195 kB 00:00 2024-08-19T01:24:13.373 INFO:teuthology.orchestra.run.smithi190.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-08-19T01:24:13.423 INFO:teuthology.orchestra.run.smithi190.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-08-19T01:24:13.430 INFO:teuthology.orchestra.run.smithi103.stdout:(17/150): libradosstriper1-19.1.0-1382.gde09dd0 3.0 MB/s | 507 kB 00:00 2024-08-19T01:24:13.473 INFO:teuthology.orchestra.run.smithi190.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-08-19T01:24:13.540 INFO:teuthology.orchestra.run.smithi190.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 943 kB 00:00 2024-08-19T01:24:13.590 INFO:teuthology.orchestra.run.smithi190.stdout:(52/150): python3-cryptography-36.0.1-4.el9.x86 1.4 MB/s | 1.2 MB 00:00 2024-08-19T01:24:13.616 INFO:teuthology.orchestra.run.smithi190.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-08-19T01:24:13.641 INFO:teuthology.orchestra.run.smithi190.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-08-19T01:24:13.666 INFO:teuthology.orchestra.run.smithi190.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-08-19T01:24:13.691 INFO:teuthology.orchestra.run.smithi190.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-08-19T01:24:13.742 INFO:teuthology.orchestra.run.smithi190.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-08-19T01:24:13.758 INFO:teuthology.orchestra.run.smithi190.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-08-19T01:24:13.836 INFO:teuthology.orchestra.run.smithi146.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 292 kB/s | 253 kB 00:00 2024-08-19T01:24:13.909 INFO:teuthology.orchestra.run.smithi190.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-08-19T01:24:13.961 INFO:teuthology.orchestra.run.smithi146.stdout:(43/150): libgfortran-11.4.1-3.el9.x86_64.rpm 526 kB/s | 803 kB 00:01 2024-08-19T01:24:13.967 INFO:teuthology.orchestra.run.smithi190.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-08-19T01:24:13.993 INFO:teuthology.orchestra.run.smithi190.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-08-19T01:24:14.018 INFO:teuthology.orchestra.run.smithi190.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-08-19T01:24:14.093 INFO:teuthology.orchestra.run.smithi190.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-08-19T01:24:14.118 INFO:teuthology.orchestra.run.smithi190.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-08-19T01:24:14.130 INFO:teuthology.orchestra.run.smithi103.stdout:(18/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-08-19T01:24:14.144 INFO:teuthology.orchestra.run.smithi190.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-08-19T01:24:14.205 INFO:teuthology.orchestra.run.smithi103.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 604 kB/s | 45 kB 00:00 2024-08-19T01:24:14.264 INFO:teuthology.orchestra.run.smithi103.stdout:(20/150): python3-ceph-common-19.1.0-1382.gde09 2.3 MB/s | 136 kB 00:00 2024-08-19T01:24:14.339 INFO:teuthology.orchestra.run.smithi103.stdout:(21/150): python3-cephfs-19.1.0-1382.gde09dd01. 2.5 MB/s | 188 kB 00:00 2024-08-19T01:24:14.415 INFO:teuthology.orchestra.run.smithi103.stdout:(22/150): python3-rados-19.1.0-1382.gde09dd01.e 4.8 MB/s | 369 kB 00:00 2024-08-19T01:24:14.507 INFO:teuthology.orchestra.run.smithi103.stdout:(23/150): python3-rbd-19.1.0-1382.gde09dd01.el9 3.7 MB/s | 350 kB 00:00 2024-08-19T01:24:14.519 INFO:teuthology.orchestra.run.smithi146.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-08-19T01:24:14.527 INFO:teuthology.orchestra.run.smithi190.stdout:(66/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-08-19T01:24:14.553 INFO:teuthology.orchestra.run.smithi146.stdout:(45/150): python3-cryptography-36.0.1-4.el9.x86 1.0 MB/s | 1.2 MB 00:01 2024-08-19T01:24:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:(24/150): python3-rgw-19.1.0-1382.gde09dd01.el9 2.0 MB/s | 117 kB 00:00 2024-08-19T01:24:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:(25/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.1 MB/s | 85 kB 00:00 2024-08-19T01:24:14.670 INFO:teuthology.orchestra.run.smithi146.stdout:(46/150): python3-pycparser-2.20-6.el9.noarch.r 191 kB/s | 135 kB 00:00 2024-08-19T01:24:14.808 INFO:teuthology.orchestra.run.smithi103.stdout:(26/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 2.8 MB/s | 17 MB 00:05 2024-08-19T01:24:14.837 INFO:teuthology.orchestra.run.smithi146.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 4.1 MB/s | 689 kB 00:00 2024-08-19T01:24:14.904 INFO:teuthology.orchestra.run.smithi146.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-08-19T01:24:14.946 INFO:teuthology.orchestra.run.smithi146.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-08-19T01:24:14.958 INFO:teuthology.orchestra.run.smithi103.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 1.1 MB/s | 170 kB 00:00 2024-08-19T01:24:14.980 INFO:teuthology.orchestra.run.smithi146.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 650 kB/s | 22 kB 00:00 2024-08-19T01:24:14.986 INFO:teuthology.orchestra.run.smithi190.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-08-19T01:24:15.021 INFO:teuthology.orchestra.run.smithi146.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-08-19T01:24:15.034 INFO:teuthology.orchestra.run.smithi103.stdout:(28/150): ceph-grafana-dashboards-19.1.0-1382.g 424 kB/s | 32 kB 00:00 2024-08-19T01:24:15.053 INFO:teuthology.orchestra.run.smithi190.stdout:(68/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-08-19T01:24:15.055 INFO:teuthology.orchestra.run.smithi146.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 981 kB/s | 33 kB 00:00 2024-08-19T01:24:15.103 INFO:teuthology.orchestra.run.smithi190.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-08-19T01:24:15.109 INFO:teuthology.orchestra.run.smithi103.stdout:(29/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 1.9 MB/s | 146 kB 00:00 2024-08-19T01:24:15.155 INFO:teuthology.orchestra.run.smithi146.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-08-19T01:24:15.189 INFO:teuthology.orchestra.run.smithi146.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 548 kB/s | 18 kB 00:00 2024-08-19T01:24:15.222 INFO:teuthology.orchestra.run.smithi146.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-08-19T01:24:15.247 INFO:teuthology.orchestra.run.smithi146.stdout:(56/150): python3-requests-2.25.1-8.el9.noarch. 172 kB/s | 125 kB 00:00 2024-08-19T01:24:15.262 INFO:teuthology.orchestra.run.smithi190.stdout:(70/150): openblas-openmp-0.3.21-2.el9.x86_64.r 4.1 MB/s | 4.7 MB 00:01 2024-08-19T01:24:15.273 INFO:teuthology.orchestra.run.smithi146.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-08-19T01:24:15.323 INFO:teuthology.orchestra.run.smithi146.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-08-19T01:24:15.329 INFO:teuthology.orchestra.run.smithi190.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 2.6 MB/s | 911 kB 00:00 2024-08-19T01:24:15.349 INFO:teuthology.orchestra.run.smithi146.stdout:(59/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-08-19T01:24:15.350 INFO:teuthology.orchestra.run.smithi103.stdout:(30/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 4.4 MB/s | 3.1 MB 00:00 2024-08-19T01:24:15.374 INFO:teuthology.orchestra.run.smithi146.stdout:(60/150): python3-urllib3-1.26.5-6.el9.noarch.r 262 kB/s | 215 kB 00:00 2024-08-19T01:24:15.379 INFO:teuthology.orchestra.run.smithi190.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-08-19T01:24:15.399 INFO:teuthology.orchestra.run.smithi146.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 505 kB/s | 25 kB 00:00 2024-08-19T01:24:15.424 INFO:teuthology.orchestra.run.smithi146.stdout:(62/150): libtool-2.4.6-46.el9.x86_64.rpm 5.6 MB/s | 581 kB 00:00 2024-08-19T01:24:15.466 INFO:teuthology.orchestra.run.smithi146.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-08-19T01:24:15.492 INFO:teuthology.orchestra.run.smithi146.stdout:(64/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 404 kB/s | 47 kB 00:00 2024-08-19T01:24:15.517 INFO:teuthology.orchestra.run.smithi146.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 698 kB/s | 35 kB 00:00 2024-08-19T01:24:15.542 INFO:teuthology.orchestra.run.smithi146.stdout:(66/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-08-19T01:24:15.626 INFO:teuthology.orchestra.run.smithi146.stdout:(67/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-08-19T01:24:15.654 INFO:teuthology.orchestra.run.smithi190.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-08-19T01:24:15.676 INFO:teuthology.orchestra.run.smithi146.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-08-19T01:24:15.704 INFO:teuthology.orchestra.run.smithi190.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-19T01:24:15.718 INFO:teuthology.orchestra.run.smithi146.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-08-19T01:24:15.743 INFO:teuthology.orchestra.run.smithi146.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-08-19T01:24:15.780 INFO:teuthology.orchestra.run.smithi190.stdout:(75/150): python3-devel-3.9.19-4.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-08-19T01:24:15.810 INFO:teuthology.orchestra.run.smithi146.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-08-19T01:24:15.844 INFO:teuthology.orchestra.run.smithi146.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-08-19T01:24:15.863 INFO:teuthology.orchestra.run.smithi190.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-08-19T01:24:15.869 INFO:teuthology.orchestra.run.smithi146.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-08-19T01:24:15.914 INFO:teuthology.orchestra.run.smithi190.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-08-19T01:24:15.953 INFO:teuthology.orchestra.run.smithi146.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-08-19T01:24:15.964 INFO:teuthology.orchestra.run.smithi190.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-08-19T01:24:15.978 INFO:teuthology.orchestra.run.smithi146.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 563 kB/s | 61 kB 00:00 2024-08-19T01:24:16.012 INFO:teuthology.orchestra.run.smithi146.stdout:(76/150): python3-devel-3.9.19-4.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-08-19T01:24:16.037 INFO:teuthology.orchestra.run.smithi146.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-08-19T01:24:16.039 INFO:teuthology.orchestra.run.smithi190.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-08-19T01:24:16.062 INFO:teuthology.orchestra.run.smithi146.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-08-19T01:24:16.087 INFO:teuthology.orchestra.run.smithi146.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-08-19T01:24:16.112 INFO:teuthology.orchestra.run.smithi146.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-08-19T01:24:16.138 INFO:teuthology.orchestra.run.smithi146.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-08-19T01:24:16.139 INFO:teuthology.orchestra.run.smithi190.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-08-19T01:24:16.222 INFO:teuthology.orchestra.run.smithi146.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-08-19T01:24:16.247 INFO:teuthology.orchestra.run.smithi146.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-08-19T01:24:16.272 INFO:teuthology.orchestra.run.smithi146.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-19T01:24:16.297 INFO:teuthology.orchestra.run.smithi146.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-19T01:24:16.314 INFO:teuthology.orchestra.run.smithi146.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-08-19T01:24:16.348 INFO:teuthology.orchestra.run.smithi146.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-19T01:24:16.373 INFO:teuthology.orchestra.run.smithi146.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-08-19T01:24:16.384 INFO:teuthology.orchestra.run.smithi103.stdout:(31/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 2.8 MB/s | 3.6 MB 00:01 2024-08-19T01:24:16.398 INFO:teuthology.orchestra.run.smithi190.stdout:(81/150): postgresql-server-13.14-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-08-19T01:24:16.407 INFO:teuthology.orchestra.run.smithi146.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-08-19T01:24:16.424 INFO:teuthology.orchestra.run.smithi146.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-19T01:24:16.441 INFO:teuthology.orchestra.run.smithi146.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-08-19T01:24:16.498 INFO:teuthology.orchestra.run.smithi190.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-08-19T01:24:16.509 INFO:teuthology.orchestra.run.smithi103.stdout:(32/150): ceph-mgr-modules-core-19.1.0-1382.gde 1.9 MB/s | 249 kB 00:00 2024-08-19T01:24:16.516 INFO:teuthology.orchestra.run.smithi146.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-08-19T01:24:16.533 INFO:teuthology.orchestra.run.smithi146.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 452 kB/s | 42 kB 00:00 2024-08-19T01:24:16.550 INFO:teuthology.orchestra.run.smithi146.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-08-19T01:24:16.575 INFO:teuthology.orchestra.run.smithi146.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-08-19T01:24:16.576 INFO:teuthology.orchestra.run.smithi103.stdout:(33/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 741 kB/s | 49 kB 00:00 2024-08-19T01:24:16.582 INFO:teuthology.orchestra.run.smithi190.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-08-19T01:24:16.592 INFO:teuthology.orchestra.run.smithi146.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 858 kB/s | 36 kB 00:00 2024-08-19T01:24:16.609 INFO:teuthology.orchestra.run.smithi146.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-08-19T01:24:16.643 INFO:teuthology.orchestra.run.smithi146.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-19T01:24:16.643 INFO:teuthology.orchestra.run.smithi103.stdout:(34/150): ceph-prometheus-alerts-19.1.0-1382.gd 246 kB/s | 16 kB 00:00 2024-08-19T01:24:16.657 INFO:teuthology.orchestra.run.smithi190.stdout:(84/150): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-08-19T01:24:16.668 INFO:teuthology.orchestra.run.smithi146.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 984 kB/s | 58 kB 00:00 2024-08-19T01:24:16.682 INFO:teuthology.orchestra.run.smithi190.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-08-19T01:24:16.693 INFO:teuthology.orchestra.run.smithi146.stdout:(100/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-19T01:24:16.708 INFO:teuthology.orchestra.run.smithi190.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-08-19T01:24:16.718 INFO:teuthology.orchestra.run.smithi146.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-19T01:24:16.727 INFO:teuthology.orchestra.run.smithi103.stdout:(35/150): ceph-volume-19.1.0-1382.gde09dd01.el9 3.3 MB/s | 285 kB 00:00 2024-08-19T01:24:16.744 INFO:teuthology.orchestra.run.smithi146.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-08-19T01:24:16.766 INFO:teuthology.orchestra.run.smithi190.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-08-19T01:24:16.791 INFO:teuthology.orchestra.run.smithi190.stdout:(88/150): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-08-19T01:24:16.833 INFO:teuthology.orchestra.run.smithi190.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-08-19T01:24:16.859 INFO:teuthology.orchestra.run.smithi190.stdout:(90/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-08-19T01:24:16.875 INFO:teuthology.orchestra.run.smithi190.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-08-19T01:24:16.877 INFO:teuthology.orchestra.run.smithi103.stdout:(36/150): ceph-mgr-diskprediction-local-19.1.0- 4.8 MB/s | 7.4 MB 00:01 2024-08-19T01:24:16.917 INFO:teuthology.orchestra.run.smithi190.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-08-19T01:24:16.944 INFO:teuthology.orchestra.run.smithi103.stdout:(37/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 275 kB/s | 18 kB 00:00 2024-08-19T01:24:16.959 INFO:teuthology.orchestra.run.smithi190.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-08-19T01:24:16.969 INFO:teuthology.orchestra.run.smithi146.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-08-19T01:24:16.969 INFO:teuthology.orchestra.run.smithi103.stdout:(38/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 3.1 MB/s | 766 kB 00:00 2024-08-19T01:24:17.019 INFO:teuthology.orchestra.run.smithi146.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-19T01:24:17.035 INFO:teuthology.orchestra.run.smithi190.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-08-19T01:24:17.045 INFO:teuthology.orchestra.run.smithi146.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-08-19T01:24:17.070 INFO:teuthology.orchestra.run.smithi103.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 638 kB/s | 80 kB 00:00 2024-08-19T01:24:17.078 INFO:teuthology.orchestra.run.smithi190.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-08-19T01:24:17.079 INFO:teuthology.orchestra.run.smithi146.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 737 kB/s | 19 kB 00:00 2024-08-19T01:24:17.119 INFO:teuthology.orchestra.run.smithi190.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-08-19T01:24:17.128 INFO:teuthology.orchestra.run.smithi103.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-08-19T01:24:17.153 INFO:teuthology.orchestra.run.smithi103.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-08-19T01:24:17.203 INFO:teuthology.orchestra.run.smithi146.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-08-19T01:24:17.236 INFO:teuthology.orchestra.run.smithi190.stdout:(97/150): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-08-19T01:24:17.237 INFO:teuthology.orchestra.run.smithi146.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 983 kB/s | 26 kB 00:00 2024-08-19T01:24:17.254 INFO:teuthology.orchestra.run.smithi103.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-08-19T01:24:17.270 INFO:teuthology.orchestra.run.smithi146.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-08-19T01:24:17.296 INFO:teuthology.orchestra.run.smithi146.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-19T01:24:17.303 INFO:teuthology.orchestra.run.smithi190.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 868 kB/s | 58 kB 00:00 2024-08-19T01:24:17.312 INFO:teuthology.orchestra.run.smithi103.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-08-19T01:24:17.321 INFO:teuthology.orchestra.run.smithi146.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 6.1 MB/s | 151 kB 00:00 2024-08-19T01:24:17.355 INFO:teuthology.orchestra.run.smithi146.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 24 MB/s | 839 kB 00:00 2024-08-19T01:24:17.362 INFO:teuthology.orchestra.run.smithi190.stdout:(99/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-08-19T01:24:17.363 INFO:teuthology.orchestra.run.smithi103.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-08-19T01:24:17.380 INFO:teuthology.orchestra.run.smithi146.stdout:(113/150): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2024-08-19T01:24:17.388 INFO:teuthology.orchestra.run.smithi103.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-08-19T01:24:17.414 INFO:teuthology.orchestra.run.smithi146.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 9.3 MB/s | 548 kB 00:00 2024-08-19T01:24:17.420 INFO:teuthology.orchestra.run.smithi190.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-08-19T01:24:17.430 INFO:teuthology.orchestra.run.smithi103.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-08-19T01:24:17.439 INFO:teuthology.orchestra.run.smithi146.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-08-19T01:24:17.464 INFO:teuthology.orchestra.run.smithi146.stdout:(116/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 518 kB/s | 43 kB 00:00 2024-08-19T01:24:17.470 INFO:teuthology.orchestra.run.smithi190.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-08-19T01:24:17.472 INFO:teuthology.orchestra.run.smithi103.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-08-19T01:24:17.489 INFO:teuthology.orchestra.run.smithi146.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-19T01:24:17.515 INFO:teuthology.orchestra.run.smithi146.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-19T01:24:17.540 INFO:teuthology.orchestra.run.smithi146.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-19T01:24:17.574 INFO:teuthology.orchestra.run.smithi146.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-08-19T01:24:17.596 INFO:teuthology.orchestra.run.smithi190.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-08-19T01:24:17.599 INFO:teuthology.orchestra.run.smithi146.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-08-19T01:24:17.625 INFO:teuthology.orchestra.run.smithi146.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-08-19T01:24:17.650 INFO:teuthology.orchestra.run.smithi146.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-08-19T01:24:17.675 INFO:teuthology.orchestra.run.smithi146.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-08-19T01:24:17.689 INFO:teuthology.orchestra.run.smithi103.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-08-19T01:24:17.720 INFO:teuthology.orchestra.run.smithi146.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 517 kB/s | 23 kB 00:00 2024-08-19T01:24:17.742 INFO:teuthology.orchestra.run.smithi146.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 884 kB/s | 19 kB 00:00 2024-08-19T01:24:17.748 INFO:teuthology.orchestra.run.smithi103.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-08-19T01:24:17.767 INFO:teuthology.orchestra.run.smithi146.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-08-19T01:24:17.773 INFO:teuthology.orchestra.run.smithi103.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-08-19T01:24:17.779 INFO:teuthology.orchestra.run.smithi190.stdout:(103/150): python3-numpy-1.20.1-5.el9.x86_64.rp 3.1 MB/s | 5.1 MB 00:01 2024-08-19T01:24:17.807 INFO:teuthology.orchestra.run.smithi103.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-08-19T01:24:17.818 INFO:teuthology.orchestra.run.smithi146.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-08-19T01:24:17.830 INFO:teuthology.orchestra.run.smithi190.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-19T01:24:17.832 INFO:teuthology.orchestra.run.smithi103.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-08-19T01:24:17.843 INFO:teuthology.orchestra.run.smithi146.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-19T01:24:17.863 INFO:teuthology.orchestra.run.smithi190.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-19T01:24:17.866 INFO:teuthology.orchestra.run.smithi103.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-08-19T01:24:17.868 INFO:teuthology.orchestra.run.smithi146.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-19T01:24:17.889 INFO:teuthology.orchestra.run.smithi190.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-08-19T01:24:17.893 INFO:teuthology.orchestra.run.smithi146.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-19T01:24:17.922 INFO:teuthology.orchestra.run.smithi190.stdout:(107/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-08-19T01:24:17.924 INFO:teuthology.orchestra.run.smithi103.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-08-19T01:24:17.927 INFO:teuthology.orchestra.run.smithi146.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-08-19T01:24:17.953 INFO:teuthology.orchestra.run.smithi146.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-08-19T01:24:17.956 INFO:teuthology.orchestra.run.smithi190.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 785 kB/s | 26 kB 00:00 2024-08-19T01:24:17.973 INFO:teuthology.orchestra.run.smithi190.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-08-19T01:24:17.983 INFO:teuthology.orchestra.run.smithi103.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-08-19T01:24:17.986 INFO:teuthology.orchestra.run.smithi146.stdout:(134/150): protobuf-compiler-3.14.0-13.el9.x86_ 695 kB/s | 863 kB 00:01 2024-08-19T01:24:17.998 INFO:teuthology.orchestra.run.smithi190.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-19T01:24:18.020 INFO:teuthology.orchestra.run.smithi146.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-08-19T01:24:18.032 INFO:teuthology.orchestra.run.smithi190.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-08-19T01:24:18.033 INFO:teuthology.orchestra.run.smithi103.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-08-19T01:24:18.053 INFO:teuthology.orchestra.run.smithi146.stdout:(136/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-08-19T01:24:18.079 INFO:teuthology.orchestra.run.smithi146.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 592 kB/s | 31 kB 00:00 2024-08-19T01:24:18.084 INFO:teuthology.orchestra.run.smithi103.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-08-19T01:24:18.099 INFO:teuthology.orchestra.run.smithi190.stdout:(112/150): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-08-19T01:24:18.104 INFO:teuthology.orchestra.run.smithi146.stdout:(138/150): python3-pyperclip-1.8.2-2.el9.noarch 240 kB/s | 28 kB 00:00 2024-08-19T01:24:18.109 INFO:teuthology.orchestra.run.smithi103.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-08-19T01:24:18.129 INFO:teuthology.orchestra.run.smithi146.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-08-19T01:24:18.132 INFO:teuthology.orchestra.run.smithi190.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-08-19T01:24:18.151 INFO:teuthology.orchestra.run.smithi103.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-08-19T01:24:18.154 INFO:teuthology.orchestra.run.smithi146.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-08-19T01:24:18.158 INFO:teuthology.orchestra.run.smithi190.stdout:(114/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-08-19T01:24:18.175 INFO:teuthology.orchestra.run.smithi190.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-08-19T01:24:18.180 INFO:teuthology.orchestra.run.smithi146.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-08-19T01:24:18.191 INFO:teuthology.orchestra.run.smithi190.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 961 kB/s | 32 kB 00:00 2024-08-19T01:24:18.193 INFO:teuthology.orchestra.run.smithi103.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-08-19T01:24:18.205 INFO:teuthology.orchestra.run.smithi146.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-08-19T01:24:18.209 INFO:teuthology.orchestra.run.smithi190.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 419 kB/s | 14 kB 00:00 2024-08-19T01:24:18.218 INFO:teuthology.orchestra.run.smithi103.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-08-19T01:24:18.230 INFO:teuthology.orchestra.run.smithi146.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-19T01:24:18.234 INFO:teuthology.orchestra.run.smithi190.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-08-19T01:24:18.244 INFO:teuthology.orchestra.run.smithi103.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-08-19T01:24:18.255 INFO:teuthology.orchestra.run.smithi146.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-19T01:24:18.267 INFO:teuthology.orchestra.run.smithi190.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-08-19T01:24:18.281 INFO:teuthology.orchestra.run.smithi146.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-08-19T01:24:18.285 INFO:teuthology.orchestra.run.smithi103.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-08-19T01:24:18.301 INFO:teuthology.orchestra.run.smithi190.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-08-19T01:24:18.306 INFO:teuthology.orchestra.run.smithi146.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-19T01:24:18.311 INFO:teuthology.orchestra.run.smithi103.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-08-19T01:24:18.326 INFO:teuthology.orchestra.run.smithi190.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-08-19T01:24:18.331 INFO:teuthology.orchestra.run.smithi146.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-19T01:24:18.336 INFO:teuthology.orchestra.run.smithi103.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-08-19T01:24:18.352 INFO:teuthology.orchestra.run.smithi190.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-08-19T01:24:18.377 INFO:teuthology.orchestra.run.smithi190.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-08-19T01:24:18.381 INFO:teuthology.orchestra.run.smithi146.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-19T01:24:18.402 INFO:teuthology.orchestra.run.smithi190.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-08-19T01:24:18.427 INFO:teuthology.orchestra.run.smithi190.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 925 kB/s | 23 kB 00:00 2024-08-19T01:24:18.453 INFO:teuthology.orchestra.run.smithi190.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-08-19T01:24:18.478 INFO:teuthology.orchestra.run.smithi190.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 777 kB/s | 20 kB 00:00 2024-08-19T01:24:18.537 INFO:teuthology.orchestra.run.smithi190.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-08-19T01:24:18.562 INFO:teuthology.orchestra.run.smithi190.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-19T01:24:18.587 INFO:teuthology.orchestra.run.smithi190.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-19T01:24:18.611 INFO:teuthology.orchestra.run.smithi103.stdout:(66/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-08-19T01:24:18.629 INFO:teuthology.orchestra.run.smithi190.stdout:(131/150): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-08-19T01:24:18.655 INFO:teuthology.orchestra.run.smithi190.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-08-19T01:24:18.678 INFO:teuthology.orchestra.run.smithi103.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-08-19T01:24:18.680 INFO:teuthology.orchestra.run.smithi190.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-08-19T01:24:18.682 INFO:teuthology.orchestra.run.smithi146.stdout:(149/150): librados2-19.1.0-1382.gde09dd01.el9. 9.1 MB/s | 3.4 MB 00:00 2024-08-19T01:24:18.705 INFO:teuthology.orchestra.run.smithi190.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-08-19T01:24:18.731 INFO:teuthology.orchestra.run.smithi190.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-08-19T01:24:18.754 INFO:teuthology.orchestra.run.smithi103.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-08-19T01:24:18.764 INFO:teuthology.orchestra.run.smithi190.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-08-19T01:24:18.782 INFO:teuthology.orchestra.run.smithi146.stdout:(150/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 7.0 MB/s | 3.2 MB 00:00 2024-08-19T01:24:18.787 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:24:18.788 INFO:teuthology.orchestra.run.smithi146.stdout:Total 16 MB/s | 218 MB 00:13 2024-08-19T01:24:18.789 INFO:teuthology.orchestra.run.smithi190.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-08-19T01:24:18.815 INFO:teuthology.orchestra.run.smithi190.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-19T01:24:18.820 INFO:teuthology.orchestra.run.smithi103.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-08-19T01:24:18.840 INFO:teuthology.orchestra.run.smithi190.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-19T01:24:18.846 INFO:teuthology.orchestra.run.smithi103.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-08-19T01:24:18.866 INFO:teuthology.orchestra.run.smithi190.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-08-19T01:24:18.891 INFO:teuthology.orchestra.run.smithi190.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-08-19T01:24:18.916 INFO:teuthology.orchestra.run.smithi190.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-19T01:24:18.938 INFO:teuthology.orchestra.run.smithi103.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-08-19T01:24:18.941 INFO:teuthology.orchestra.run.smithi190.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-19T01:24:18.966 INFO:teuthology.orchestra.run.smithi190.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 797 kB/s | 20 kB 00:00 2024-08-19T01:24:18.992 INFO:teuthology.orchestra.run.smithi190.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-08-19T01:24:19.018 INFO:teuthology.orchestra.run.smithi190.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-19T01:24:19.176 INFO:teuthology.orchestra.run.smithi190.stdout:(147/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-08-19T01:24:19.226 INFO:teuthology.orchestra.run.smithi190.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-08-19T01:24:19.430 INFO:teuthology.orchestra.run.smithi103.stdout:(72/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 6.9 MB/s | 49 MB 00:07 2024-08-19T01:24:19.455 INFO:teuthology.orchestra.run.smithi103.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 169 kB/s | 87 kB 00:00 2024-08-19T01:24:19.467 INFO:teuthology.orchestra.run.smithi190.stdout:(149/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 11 MB/s | 3.2 MB 00:00 2024-08-19T01:24:19.556 INFO:teuthology.orchestra.run.smithi103.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 7.9 MB/s | 5.8 MB 00:00 2024-08-19T01:24:19.581 INFO:teuthology.orchestra.run.smithi103.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 488 kB/s | 61 kB 00:00 2024-08-19T01:24:19.615 INFO:teuthology.orchestra.run.smithi103.stdout:(76/150): python3-devel-3.9.19-4.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-08-19T01:24:19.640 INFO:teuthology.orchestra.run.smithi103.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-08-19T01:24:19.657 INFO:teuthology.orchestra.run.smithi103.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 26 kB 00:00 2024-08-19T01:24:19.682 INFO:teuthology.orchestra.run.smithi103.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-08-19T01:24:19.707 INFO:teuthology.orchestra.run.smithi103.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-19T01:24:19.724 INFO:teuthology.orchestra.run.smithi103.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 826 kB/s | 35 kB 00:00 2024-08-19T01:24:19.783 INFO:teuthology.orchestra.run.smithi103.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-08-19T01:24:19.867 INFO:teuthology.orchestra.run.smithi103.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-08-19T01:24:19.903 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-08-19T01:24:19.925 INFO:teuthology.orchestra.run.smithi103.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-08-19T01:24:19.934 INFO:teuthology.orchestra.run.smithi190.stdout:(150/150): librados2-19.1.0-1382.gde09dd01.el9. 3.7 MB/s | 3.4 MB 00:00 2024-08-19T01:24:19.939 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:24:19.939 INFO:teuthology.orchestra.run.smithi190.stdout:Total 15 MB/s | 218 MB 00:14 2024-08-19T01:24:19.984 INFO:teuthology.orchestra.run.smithi103.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-08-19T01:24:20.030 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-08-19T01:24:20.031 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-08-19T01:24:20.051 INFO:teuthology.orchestra.run.smithi103.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-08-19T01:24:20.126 INFO:teuthology.orchestra.run.smithi103.stdout:(87/150): python3-babel-2.9.1-2.el9.noarch.rpm 8.6 MB/s | 6.0 MB 00:00 2024-08-19T01:24:20.151 INFO:teuthology.orchestra.run.smithi103.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-08-19T01:24:20.219 INFO:teuthology.orchestra.run.smithi103.stdout:(89/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-08-19T01:24:20.244 INFO:teuthology.orchestra.run.smithi103.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 554 kB/s | 51 kB 00:00 2024-08-19T01:24:20.269 INFO:teuthology.orchestra.run.smithi103.stdout:(91/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-08-19T01:24:20.294 INFO:teuthology.orchestra.run.smithi103.stdout:(92/150): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-08-19T01:24:20.328 INFO:teuthology.orchestra.run.smithi103.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-08-19T01:24:20.353 INFO:teuthology.orchestra.run.smithi103.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-08-19T01:24:20.387 INFO:teuthology.orchestra.run.smithi103.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-08-19T01:24:20.412 INFO:teuthology.orchestra.run.smithi103.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-08-19T01:24:20.471 INFO:teuthology.orchestra.run.smithi103.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-08-19T01:24:20.496 INFO:teuthology.orchestra.run.smithi103.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-08-19T01:24:20.538 INFO:teuthology.orchestra.run.smithi103.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-08-19T01:24:20.563 INFO:teuthology.orchestra.run.smithi103.stdout:(100/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-08-19T01:24:20.597 INFO:teuthology.orchestra.run.smithi103.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-08-19T01:24:20.631 INFO:teuthology.orchestra.run.smithi103.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 667 kB/s | 45 kB 00:00 2024-08-19T01:24:20.731 INFO:teuthology.orchestra.run.smithi103.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 166 kB/s | 22 kB 00:00 2024-08-19T01:24:20.781 INFO:teuthology.orchestra.run.smithi103.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-19T01:24:20.815 INFO:teuthology.orchestra.run.smithi103.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-19T01:24:20.840 INFO:teuthology.orchestra.run.smithi103.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-08-19T01:24:20.941 INFO:teuthology.orchestra.run.smithi103.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-08-19T01:24:20.966 INFO:teuthology.orchestra.run.smithi103.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-19T01:24:20.999 INFO:teuthology.orchestra.run.smithi103.stdout:(109/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-08-19T01:24:21.025 INFO:teuthology.orchestra.run.smithi103.stdout:(110/150): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-08-19T01:24:21.047 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-08-19T01:24:21.058 INFO:teuthology.orchestra.run.smithi103.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-08-19T01:24:21.101 INFO:teuthology.orchestra.run.smithi103.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 24 MB/s | 839 kB 00:00 2024-08-19T01:24:21.172 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-08-19T01:24:21.173 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-08-19T01:24:21.268 INFO:teuthology.orchestra.run.smithi103.stdout:(113/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-08-19T01:24:21.293 INFO:teuthology.orchestra.run.smithi103.stdout:(114/150): luarocks-3.9.2-5.el9.noarch.rpm 565 kB/s | 151 kB 00:00 2024-08-19T01:24:21.318 INFO:teuthology.orchestra.run.smithi103.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.7 MB/s | 32 kB 00:00 2024-08-19T01:24:21.343 INFO:teuthology.orchestra.run.smithi103.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 608 kB/s | 14 kB 00:00 2024-08-19T01:24:21.368 INFO:teuthology.orchestra.run.smithi103.stdout:(117/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 432 kB/s | 43 kB 00:00 2024-08-19T01:24:21.394 INFO:teuthology.orchestra.run.smithi103.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-19T01:24:21.419 INFO:teuthology.orchestra.run.smithi103.stdout:(119/150): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-08-19T01:24:21.453 INFO:teuthology.orchestra.run.smithi103.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-08-19T01:24:21.478 INFO:teuthology.orchestra.run.smithi103.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-08-19T01:24:21.503 INFO:teuthology.orchestra.run.smithi103.stdout:(122/150): python3-google-auth-2.33.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-08-19T01:24:21.529 INFO:teuthology.orchestra.run.smithi103.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-08-19T01:24:21.554 INFO:teuthology.orchestra.run.smithi103.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-08-19T01:24:21.579 INFO:teuthology.orchestra.run.smithi103.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-08-19T01:24:21.604 INFO:teuthology.orchestra.run.smithi103.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-08-19T01:24:21.629 INFO:teuthology.orchestra.run.smithi103.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-08-19T01:24:21.655 INFO:teuthology.orchestra.run.smithi103.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-08-19T01:24:21.680 INFO:teuthology.orchestra.run.smithi103.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-19T01:24:21.714 INFO:teuthology.orchestra.run.smithi103.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-08-19T01:24:21.739 INFO:teuthology.orchestra.run.smithi103.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-19T01:24:21.770 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-08-19T01:24:21.773 INFO:teuthology.orchestra.run.smithi103.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-08-19T01:24:21.778 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-08-19T01:24:21.798 INFO:teuthology.orchestra.run.smithi103.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-08-19T01:24:21.824 INFO:teuthology.orchestra.run.smithi103.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-19T01:24:21.849 INFO:teuthology.orchestra.run.smithi103.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-08-19T01:24:21.874 INFO:teuthology.orchestra.run.smithi103.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-08-19T01:24:21.900 INFO:teuthology.orchestra.run.smithi103.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-08-19T01:24:21.925 INFO:teuthology.orchestra.run.smithi103.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-19T01:24:21.950 INFO:teuthology.orchestra.run.smithi103.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-19T01:24:21.975 INFO:teuthology.orchestra.run.smithi103.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-08-19T01:24:22.034 INFO:teuthology.orchestra.run.smithi103.stdout:(141/150): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-08-19T01:24:22.059 INFO:teuthology.orchestra.run.smithi103.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 891 kB/s | 74 kB 00:00 2024-08-19T01:24:22.085 INFO:teuthology.orchestra.run.smithi103.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-19T01:24:22.110 INFO:teuthology.orchestra.run.smithi103.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-08-19T01:24:22.135 INFO:teuthology.orchestra.run.smithi103.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-08-19T01:24:22.160 INFO:teuthology.orchestra.run.smithi103.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-19T01:24:22.194 INFO:teuthology.orchestra.run.smithi103.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-08-19T01:24:22.253 INFO:teuthology.orchestra.run.smithi103.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-08-19T01:24:22.512 INFO:teuthology.orchestra.run.smithi103.stdout:(149/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 10 MB/s | 3.2 MB 00:00 2024-08-19T01:24:22.761 INFO:teuthology.orchestra.run.smithi103.stdout:(150/150): librados2-19.1.0-1382.gde09dd01.el9. 5.7 MB/s | 3.4 MB 00:00 2024-08-19T01:24:22.767 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:24:22.767 INFO:teuthology.orchestra.run.smithi103.stdout:Total 13 MB/s | 218 MB 00:16 2024-08-19T01:24:22.887 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-08-19T01:24:22.887 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-08-19T01:24:23.885 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-08-19T01:24:24.010 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-08-19T01:24:24.010 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-08-19T01:24:25.722 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-08-19T01:24:25.722 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-08-19T01:24:27.567 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-08-19T01:24:27.643 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-19T01:24:27.668 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-19T01:24:27.711 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-19T01:24:28.044 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-19T01:24:28.160 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:24:28.282 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:24:28.311 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:28.406 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:28.445 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:24:28.474 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-19T01:24:28.499 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-19T01:24:28.653 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-19T01:24:28.699 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-19T01:24:28.727 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:28.766 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-08-19T01:24:28.831 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:28.861 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:24:28.919 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-19T01:24:28.924 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:24:28.945 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:24:28.969 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:24:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-19T01:24:29.216 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-19T01:24:29.260 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:24:29.295 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:24:29.404 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-19T01:24:29.498 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-19T01:24:29.535 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-19T01:24:29.544 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-19T01:24:29.570 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-19T01:24:29.635 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-19T01:24:29.655 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:24:29.679 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-19T01:24:29.711 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-19T01:24:29.755 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-19T01:24:29.759 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:24:29.787 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-19T01:24:29.798 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:29.815 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-19T01:24:29.884 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-19T01:24:29.893 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:29.931 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:24:29.945 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-19T01:24:29.971 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-19T01:24:29.976 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-19T01:24:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-19T01:24:30.011 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-19T01:24:30.088 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-19T01:24:30.122 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-19T01:24:30.154 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-19T01:24:30.168 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-19T01:24:30.194 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-19T01:24:30.208 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-19T01:24:30.236 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:30.265 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-19T01:24:30.309 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-19T01:24:30.332 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:30.354 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:24:30.384 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-19T01:24:30.417 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:24:30.439 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:24:30.461 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:24:30.518 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-19T01:24:30.570 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-19T01:24:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:24:30.754 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:24:30.862 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-19T01:24:30.966 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-19T01:24:30.995 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-19T01:24:31.030 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-19T01:24:31.095 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-19T01:24:31.096 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-19T01:24:31.140 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-19T01:24:31.179 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-19T01:24:31.183 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-19T01:24:31.211 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-19T01:24:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-19T01:24:31.253 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-19T01:24:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-19T01:24:31.291 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-19T01:24:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-19T01:24:31.324 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-19T01:24:31.361 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-19T01:24:31.380 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-19T01:24:31.422 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-19T01:24:31.453 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-19T01:24:31.454 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-19T01:24:31.484 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-19T01:24:31.496 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-19T01:24:31.513 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-08-19T01:24:31.520 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-19T01:24:31.567 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-19T01:24:31.573 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-19T01:24:31.580 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-19T01:24:31.603 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-19T01:24:31.606 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-19T01:24:31.607 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-19T01:24:31.645 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-19T01:24:31.649 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-19T01:24:31.653 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-19T01:24:31.679 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-19T01:24:31.687 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-19T01:24:31.750 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-19T01:24:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-19T01:24:31.825 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-19T01:24:31.864 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-19T01:24:31.869 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-19T01:24:31.905 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-19T01:24:31.979 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-19T01:24:32.003 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-19T01:24:32.008 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-19T01:24:32.055 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-19T01:24:32.056 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-19T01:24:32.096 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-19T01:24:32.107 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:24:32.121 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-19T01:24:32.146 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-19T01:24:32.221 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-19T01:24:32.230 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-19T01:24:32.258 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:32.349 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-19T01:24:32.354 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:24:32.386 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-19T01:24:32.400 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:24:32.414 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-19T01:24:32.429 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-19T01:24:32.448 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-19T01:24:32.455 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-19T01:24:32.492 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-19T01:24:32.539 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-19T01:24:32.570 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-19T01:24:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-19T01:24:32.600 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-19T01:24:32.600 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-19T01:24:32.634 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-19T01:24:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-19T01:24:32.668 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-19T01:24:32.696 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-19T01:24:32.708 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-19T01:24:32.730 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-19T01:24:32.743 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-19T01:24:32.768 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-19T01:24:32.777 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-19T01:24:32.801 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-19T01:24:32.806 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:32.832 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-19T01:24:32.857 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-19T01:24:32.873 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-19T01:24:32.893 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:24:32.905 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-19T01:24:32.923 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:24:32.930 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-19T01:24:32.970 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-19T01:24:32.986 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-19T01:24:33.005 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-19T01:24:33.008 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:24:33.009 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-19T01:24:33.030 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-19T01:24:33.053 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-19T01:24:33.064 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-19T01:24:33.088 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-19T01:24:33.132 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-19T01:24:33.174 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-19T01:24:33.311 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-19T01:24:33.321 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:24:33.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-19T01:24:33.357 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-19T01:24:33.392 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-19T01:24:33.465 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-19T01:24:33.495 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-19T01:24:33.550 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-19T01:24:33.567 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-19T01:24:33.590 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-19T01:24:33.599 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-19T01:24:33.615 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-19T01:24:33.634 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-19T01:24:33.640 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-19T01:24:33.710 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-19T01:24:33.715 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-19T01:24:33.761 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-19T01:24:33.801 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-19T01:24:33.801 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-19T01:24:33.843 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-19T01:24:33.846 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-19T01:24:33.853 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-19T01:24:33.879 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-19T01:24:33.880 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-19T01:24:33.885 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-19T01:24:33.900 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-19T01:24:33.913 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-19T01:24:33.934 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-19T01:24:33.979 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-19T01:24:33.979 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-19T01:24:33.987 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-19T01:24:34.016 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-19T01:24:34.025 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-19T01:24:34.045 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-19T01:24:34.047 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-19T01:24:34.070 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-19T01:24:34.074 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-19T01:24:34.086 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-19T01:24:34.119 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-19T01:24:34.146 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-19T01:24:34.160 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-19T01:24:34.178 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-19T01:24:34.198 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-19T01:24:34.216 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-19T01:24:34.220 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-19T01:24:34.254 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-19T01:24:34.262 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-19T01:24:34.308 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-19T01:24:34.334 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-19T01:24:34.341 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-19T01:24:34.374 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-19T01:24:34.385 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-19T01:24:34.404 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-19T01:24:34.407 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-19T01:24:34.417 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-19T01:24:34.450 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-19T01:24:34.481 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-19T01:24:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-19T01:24:34.526 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-19T01:24:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-19T01:24:34.664 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-19T01:24:34.720 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-19T01:24:35.136 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-19T01:24:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-19T01:24:35.282 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-19T01:24:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-19T01:24:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-19T01:24:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-19T01:24:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-19T01:24:35.501 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-19T01:24:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-19T01:24:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-19T01:24:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-19T01:24:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-19T01:24:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-19T01:24:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-19T01:24:35.683 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-19T01:24:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-19T01:24:35.729 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-19T01:24:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-19T01:24:35.774 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-19T01:24:35.794 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-19T01:24:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-19T01:24:35.862 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-19T01:24:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-19T01:24:35.893 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-19T01:24:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-19T01:24:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-19T01:24:36.068 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-19T01:24:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-19T01:24:36.099 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-19T01:24:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-19T01:24:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-19T01:24:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-19T01:24:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-19T01:24:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-19T01:24:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-19T01:24:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-19T01:24:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-19T01:24:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-19T01:24:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-19T01:24:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-19T01:24:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-19T01:24:36.787 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-19T01:24:36.826 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-19T01:24:36.874 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:24:36.902 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-19T01:24:36.935 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-19T01:24:36.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-19T01:24:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-19T01:24:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-19T01:24:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-19T01:24:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-19T01:24:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-19T01:24:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-19T01:24:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-19T01:24:37.243 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-19T01:24:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-19T01:24:37.281 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:24:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-19T01:24:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-19T01:24:37.376 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:24:37.403 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-19T01:24:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-19T01:24:37.438 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-19T01:24:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-19T01:24:37.913 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-19T01:24:37.951 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:24:38.023 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:24:38.053 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-19T01:24:38.195 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-19T01:24:38.225 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-19T01:24:38.286 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:24:38.314 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-19T01:24:38.347 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-19T01:24:38.508 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-19T01:24:38.541 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-19T01:24:38.643 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-19T01:24:38.654 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-19T01:24:38.688 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-19T01:24:38.701 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:24:38.734 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-19T01:24:38.797 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:24:38.821 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-19T01:24:38.822 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-19T01:24:38.867 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-19T01:24:38.869 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-19T01:24:39.039 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-19T01:24:39.067 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-19T01:24:39.144 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-19T01:24:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-19T01:24:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:24:39.435 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:24:39.464 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-19T01:24:39.803 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-19T01:24:39.917 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-19T01:24:40.339 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:40.382 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:40.977 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:24:41.053 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:24:41.053 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T01:24:41.053 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T01:24:41.053 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:41.087 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:24:41.173 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-19T01:24:41.272 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:24:41.300 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-19T01:24:41.333 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-19T01:24:41.641 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-19T01:24:41.679 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:24:41.728 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:41.769 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:41.789 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-19T01:24:41.828 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-19T01:24:41.870 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-19T01:24:42.310 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:42.344 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-19T01:24:42.355 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:24:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:24:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-19T01:24:42.440 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:24:42.440 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T01:24:42.440 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T01:24:42.441 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:42.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-19T01:24:42.473 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:24:44.737 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:44.764 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:45.316 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-19T01:24:45.351 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:24:45.442 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-19T01:24:45.442 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-19T01:24:45.442 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-19T01:24:45.442 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:45.469 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:24:52.315 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:24:52.315 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-08-19T01:24:52.315 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-08-19T01:24:52.315 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-08-19T01:24:52.316 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-08-19T01:24:52.316 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-08-19T01:24:52.316 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-08-19T01:24:52.316 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-08-19T01:24:52.316 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:24:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:24:52.507 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:52.507 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T01:24:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-08-19T01:24:53.889 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:54.001 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:24:54.028 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:24:54.028 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:54.029 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T01:24:54.029 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:24:54.029 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:24:54.029 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:54.050 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:24:54.066 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:24:54.066 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:54.066 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T01:24:54.067 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:54.252 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:24:54.275 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:24:54.275 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:54.275 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T01:24:54.275 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:24:54.275 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:24:54.275 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:54.711 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:24:54.735 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:24:54.735 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:54.735 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T01:24:54.735 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:24:54.735 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:24:54.735 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:54.849 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-19T01:24:54.906 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-19T01:24:54.961 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-19T01:24:55.145 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-19T01:24:55.194 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-19T01:24:55.272 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-19T01:24:55.361 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-19T01:24:55.491 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-19T01:24:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:24:55.536 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-19T01:24:55.547 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:24:55.547 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:55.547 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T01:24:55.547 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:24:55.547 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:24:55.547 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-19T01:24:55.596 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-19T01:24:55.779 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:24:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:56.075 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:24:56.103 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:24:56.134 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-19T01:24:56.196 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-19T01:24:56.226 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:24:56.264 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:24:56.288 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:24:56.289 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:56.289 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T01:24:56.289 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:24:56.289 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:24:56.289 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:24:56.386 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-19T01:24:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-19T01:24:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-19T01:24:56.692 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-19T01:24:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-19T01:24:56.826 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-19T01:24:56.923 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-19T01:24:57.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-19T01:24:57.090 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-19T01:24:57.090 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-08-19T01:24:57.090 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-08-19T01:24:57.090 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-08-19T01:24:57.090 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-08-19T01:24:57.090 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-08-19T01:24:57.091 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-08-19T01:24:57.091 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-08-19T01:24:57.091 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:57.098 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-19T01:24:57.130 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-19T01:24:57.158 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-19T01:24:57.267 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:24:57.275 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:24:57.293 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-19T01:24:57.293 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:57.293 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-19T01:24:57.293 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:57.311 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:24:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:24:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:24:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-19T01:24:57.734 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-19T01:24:57.771 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:24:58.282 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:24:58.303 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:24:58.406 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:24:58.578 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-19T01:24:58.605 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:24:58.630 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:24:58.630 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:58.630 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T01:24:58.630 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:24:58.630 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:24:58.630 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:58.690 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:24:58.702 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:24:58.762 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:24:58.789 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-19T01:24:58.790 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:58.790 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-19T01:24:58.790 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:24:58.790 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-19T01:24:58.790 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:58.807 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-19T01:24:58.824 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:24:58.854 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-19T01:24:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:24:59.021 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:24:59.045 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-19T01:24:59.045 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:59.045 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-19T01:24:59.045 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:24:59.045 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-19T01:24:59.045 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:59.480 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:24:59.504 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-19T01:24:59.504 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:59.504 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-19T01:24:59.504 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:24:59.504 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-19T01:24:59.504 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:24:59.619 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-19T01:24:59.675 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-19T01:24:59.732 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-19T01:24:59.765 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-19T01:24:59.791 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:24:59.816 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:24:59.816 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:59.816 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T01:24:59.816 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:24:59.816 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:24:59.816 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:59.816 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:24:59.840 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:24:59.859 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:24:59.882 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:24:59.882 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:59.883 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T01:24:59.883 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:59.918 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-19T01:24:59.934 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:24:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:24:59.958 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:24:59.958 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:24:59.958 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-19T01:24:59.959 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:24:59.959 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:24:59.959 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:24:59.964 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-19T01:25:00.042 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-19T01:25:00.120 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-19T01:25:00.140 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-19T01:25:00.158 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:25:00.183 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:25:00.183 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:00.183 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T01:25:00.183 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:25:00.183 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:25:00.183 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:25:00.234 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:25:00.246 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:25:00.268 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:25:00.279 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-19T01:25:00.292 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:25:00.292 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:00.292 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T01:25:00.292 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:25:00.292 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:25:00.292 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:25:00.323 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-19T01:25:00.342 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-19T01:25:00.356 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-19T01:25:00.383 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-19T01:25:00.399 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-19T01:25:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:25:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-19T01:25:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-19T01:25:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-19T01:25:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:25:01.305 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-19T01:25:01.345 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:25:01.369 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:25:01.369 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:01.369 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T01:25:01.369 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:25:01.369 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:25:01.369 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:25:01.421 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:25:01.444 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:25:01.444 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:01.444 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T01:25:01.444 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:25:01.488 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:25:01.511 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:25:01.511 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:01.512 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-19T01:25:01.512 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:25:01.512 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:25:01.512 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:25:01.814 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:25:01.837 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:25:01.837 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:01.837 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T01:25:01.837 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:25:01.838 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:25:01.838 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:25:02.043 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-19T01:25:02.082 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:25:03.046 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-19T01:25:03.083 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:25:03.185 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-19T01:25:03.345 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-19T01:25:03.401 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:25:03.426 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-19T01:25:03.426 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:03.426 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-19T01:25:03.426 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:25:03.426 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-19T01:25:03.426 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:25:03.477 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:25:03.489 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-19T01:25:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-19T01:25:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-19T01:25:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-19T01:25:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:25:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-19T01:25:04.621 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:04.621 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-19T01:25:04.621 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:25:04.621 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-19T01:25:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:25:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:25:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-19T01:25:04.687 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:04.687 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-19T01:25:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:25:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:25:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-19T01:25:04.764 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:04.764 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-19T01:25:04.764 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:25:04.764 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-19T01:25:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:25:04.838 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-19T01:25:04.879 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-19T01:25:04.920 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-19T01:25:04.952 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-19T01:25:04.979 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-19T01:25:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:25:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-19T01:25:05.089 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-19T01:25:05.089 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-19T01:25:05.089 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:25:05.089 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-19T01:25:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:25:05.160 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-19T01:25:05.201 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-19T01:25:05.230 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-19T01:25:05.852 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:05.895 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:05.915 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:06.358 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-19T01:25:06.400 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-19T01:25:06.441 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-19T01:25:06.473 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-19T01:25:06.500 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-19T01:25:06.682 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-19T01:25:06.729 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-19T01:25:06.756 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-19T01:25:06.759 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-19T01:25:06.841 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-19T01:25:06.881 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-19T01:25:06.971 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-19T01:25:07.012 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-19T01:25:07.012 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:25:07.104 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:25:07.104 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:07.384 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:07.424 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:07.444 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:08.269 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-19T01:25:08.361 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-19T01:25:08.402 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-19T01:25:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-19T01:25:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-19T01:25:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:25:08.600 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:25:08.600 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:09.610 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-19T01:25:09.660 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-19T01:25:09.709 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-19T01:25:09.742 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-19T01:25:09.769 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-19T01:25:09.952 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-19T01:25:09.991 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-19T01:25:10.020 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-19T01:25:10.610 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:10.652 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:10.673 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-19T01:25:11.510 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-19T01:25:11.599 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-19T01:25:11.639 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-19T01:25:11.737 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-19T01:25:11.770 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-19T01:25:11.771 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:25:11.854 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-19T01:25:11.854 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:12.926 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:12.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-19T01:25:12.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-19T01:25:12.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-19T01:25:12.926 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-19T01:25:12.927 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-19T01:25:12.928 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-19T01:25:12.929 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-19T01:25:12.930 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-19T01:25:12.931 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-19T01:25:12.932 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-19T01:25:12.933 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-19T01:25:12.934 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-19T01:25:12.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-19T01:25:12.936 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-19T01:25:12.937 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-19T01:25:12.938 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-19T01:25:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-19T01:25:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-19T01:25:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-19T01:25:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-19T01:25:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-19T01:25:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-19T01:25:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-19T01:25:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-19T01:25:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-19T01:25:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-19T01:25:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-19T01:25:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-19T01:25:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-19T01:25:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-19T01:25:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-19T01:25:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-19T01:25:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-19T01:25:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-19T01:25:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-19T01:25:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-19T01:25:15.074 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:15.074 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:25:15.074 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-08-19T01:25:15.074 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.2-8.el9.noarch 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.7.4-5.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.075 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.076 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.4-9.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T01:25:15.077 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T01:25:15.078 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T01:25:15.079 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-13.14-1.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T01:25:15.080 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-19T01:25:15.081 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T01:25:15.082 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T01:25:15.083 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.084 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-19T01:25:15.085 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T01:25:15.086 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-19T01:25:15.087 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-19T01:25:15.087 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T01:25:15.087 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-19T01:25:15.087 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:25:15.087 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-08-19T01:25:15.419 DEBUG:teuthology.parallel:result is None 2024-08-19T01:25:16.439 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.440 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.441 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-08-19T01:25:16.442 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T01:25:16.443 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T01:25:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.14-1.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T01:25:16.445 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T01:25:16.446 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T01:25:16.447 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T01:25:16.448 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T01:25:16.449 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T01:25:16.450 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-19T01:25:16.451 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T01:25:16.452 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-19T01:25:16.452 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:25:16.452 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-08-19T01:25:16.812 DEBUG:teuthology.parallel:result is None 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-19T01:25:17.857 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-19T01:25:17.858 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-19T01:25:17.859 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-19T01:25:17.859 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-19T01:25:17.859 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-19T01:25:17.859 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-19T01:25:17.869 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-19T01:25:17.869 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-19T01:25:17.869 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-19T01:25:17.870 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-19T01:25:17.871 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-19T01:25:17.872 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-19T01:25:17.873 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-19T01:25:17.874 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-19T01:25:17.875 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-19T01:25:17.876 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-19T01:25:17.877 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-19T01:25:17.878 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-19T01:25:20.019 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.020 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.021 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-19T01:25:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-19T01:25:20.023 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-08-19T01:25:20.024 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.14-1.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-19T01:25:20.025 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-19T01:25:20.026 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-19T01:25:20.027 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-19T01:25:20.028 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.029 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-19T01:25:20.030 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-3.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-19T01:25:20.031 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-19T01:25:20.032 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-19T01:25:20.032 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:25:20.032 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-08-19T01:25:20.375 DEBUG:teuthology.parallel:result is None 2024-08-19T01:25:20.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:20.516 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T01:25:20.537 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T01:25:20.538 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T01:25:20.538 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T01:25:20.540 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:20.675 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T01:25:20.701 INFO:teuthology.orchestra.run.smithi146.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T01:25:20.701 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T01:25:20.702 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T01:25:20.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:20.842 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-19T01:25:20.869 INFO:teuthology.orchestra.run.smithi190.stdout:19.1.0-1382.gde09dd01.el9 2024-08-19T01:25:20.869 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-19T01:25:20.869 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-19T01:25:20.874 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-19T01:25:20.874 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:25:20.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T01:25:20.901 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:25:20.901 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T01:25:20.934 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:25:20.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-19T01:25:20.965 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-19T01:25:20.966 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:25:20.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T01:25:20.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T01:25:21.060 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:25:21.060 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T01:25:21.090 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T01:25:21.159 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:25:21.159 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-08-19T01:25:21.190 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-19T01:25:21.258 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-19T01:25:21.258 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:25:21.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T01:25:21.282 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T01:25:21.347 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:25:21.347 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T01:25:21.377 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T01:25:21.446 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:25:21.446 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-19T01:25:21.474 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-19T01:25:21.542 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-19T01:25:21.543 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:25:21.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T01:25:21.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T01:25:21.634 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:25:21.634 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T01:25:21.664 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T01:25:21.734 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:25:21.734 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-08-19T01:25:21.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-19T01:25:21.829 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-19T01:25:21.927 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-08-19T01:25:21.927 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:21.928 INFO:tasks.cephadm:Cluster fsid is e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:21.928 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-19T01:25:21.928 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.103', 'mon.b': '172.21.15.146', 'mon.c': '172.21.15.190'} 2024-08-19T01:25:21.929 INFO:tasks.cephadm:First mon is mon.a on smithi103 2024-08-19T01:25:21.929 INFO:tasks.cephadm:First mgr is x 2024-08-19T01:25:21.929 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-19T01:25:21.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-08-19T01:25:21.954 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname $(hostname -s) 2024-08-19T01:25:21.984 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-08-19T01:25:22.012 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-19T01:25:22.012 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:22.161 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55869/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.0-1382-gde09dd01', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.0-1382.gde09dd01'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-18 16:22:46.893995', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-19T01:25:22.298 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 de09dd018de4baddfa6ca4d43bc8fb784cf75833 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:25:22.303 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm 2024-08-19T01:25:22.303 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm 2024-08-19T01:25:22.303 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:22.694 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 19 01:25 /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:22.695 DEBUG:teuthology.orchestra.run.smithi146:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:23.076 INFO:teuthology.orchestra.run.smithi146.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 19 01:25 /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:23.077 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:23.462 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 19 01:25 /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:23.462 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:23.480 DEBUG:teuthology.orchestra.run.smithi146:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:23.496 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-19T01:25:23.533 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 on all hosts... 2024-08-19T01:25:23.534 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-19T01:25:23.536 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-19T01:25:23.539 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-19T01:25:25.262 INFO:teuthology.orchestra.run.smithi146.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:25:25.277 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:25:25.491 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:25:53.138 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-19T01:25:53.138 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-19T01:25:53.139 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-19T01:25:53.139 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-08-19T01:25:53.139 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-19T01:25:53.139 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-08-19T01:25:53.139 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-19T01:25:55.640 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-19T01:25:55.640 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-19T01:25:55.640 INFO:teuthology.orchestra.run.smithi146.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-19T01:25:55.641 INFO:teuthology.orchestra.run.smithi146.stdout: "repo_digests": [ 2024-08-19T01:25:55.641 INFO:teuthology.orchestra.run.smithi146.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-19T01:25:55.641 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-08-19T01:25:55.641 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-19T01:25:55.959 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:25:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-19T01:25:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-19T01:25:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-08-19T01:25:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-19T01:25:55.959 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:25:55.959 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:25:55.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-08-19T01:25:56.036 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph 2024-08-19T01:25:56.064 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-08-19T01:25:56.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-08-19T01:25:56.122 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /etc/ceph 2024-08-19T01:25:56.146 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-08-19T01:25:56.175 INFO:tasks.cephadm:Writing seed config... 2024-08-19T01:25:56.176 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-08-19T01:25:56.176 INFO:tasks.cephadm: override: [client] debug client = 20 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-08-19T01:25:56.177 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-08-19T01:25:56.178 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-19T01:25:56.179 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-19T01:25:56.180 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-19T01:25:56.180 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-19T01:25:56.180 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-08-19T01:25:56.180 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-19T01:25:56.180 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-08-19T01:25:56.180 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-19T01:25:56.180 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-08-19T01:25:56.181 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:25:56.181 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-19T01:25:56.196 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e721016a-5dc9-11ef-bccf-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-08-19T01:25:56.197 DEBUG:teuthology.orchestra.run.smithi103:mon.a> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a.service 2024-08-19T01:25:56.239 DEBUG:teuthology.orchestra.run.smithi103:mgr.x> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.x.service 2024-08-19T01:25:56.281 INFO:tasks.cephadm:Bootstrapping... 2024-08-19T01:25:56.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 -v bootstrap --fsid e721016a-5dc9-11ef-bccf-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.103 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:25:56.484 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-08-19T01:25:56.484 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833', '-v', 'bootstrap', '--fsid', 'e721016a-5dc9-11ef-bccf-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.103', '--skip-admin-label'] 2024-08-19T01:25:56.484 INFO:teuthology.orchestra.run.smithi103.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-19T01:25:56.485 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying podman|docker is present... 2024-08-19T01:25:56.520 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.1.2 2024-08-19T01:25:56.520 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying lvm2 is present... 2024-08-19T01:25:56.520 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying time synchronization is in place... 2024-08-19T01:25:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-19T01:25:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-19T01:25:56.534 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-19T01:25:56.534 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-08-19T01:25:56.540 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-08-19T01:25:56.547 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-08-19T01:25:56.547 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-08-19T01:25:56.547 INFO:teuthology.orchestra.run.smithi103.stdout:Repeating the final host check... 2024-08-19T01:25:56.570 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.1.2 2024-08-19T01:25:56.570 INFO:teuthology.orchestra.run.smithi103.stdout:podman (/bin/podman) version 5.1.2 is present 2024-08-19T01:25:56.570 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl is present 2024-08-19T01:25:56.571 INFO:teuthology.orchestra.run.smithi103.stdout:lvcreate is present 2024-08-19T01:25:56.577 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-19T01:25:56.577 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-19T01:25:56.583 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-19T01:25:56.583 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-08-19T01:25:56.589 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-08-19T01:25:56.596 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-08-19T01:25:56.596 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-08-19T01:25:56.596 INFO:teuthology.orchestra.run.smithi103.stdout:Host looks OK 2024-08-19T01:25:56.596 INFO:teuthology.orchestra.run.smithi103.stdout:Cluster fsid: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:25:56.596 INFO:teuthology.orchestra.run.smithi103.stdout:Acquiring lock 140268724615680 on /run/cephadm/e721016a-5dc9-11ef-bccf-c7b262605968.lock 2024-08-19T01:25:56.596 INFO:teuthology.orchestra.run.smithi103.stdout:Lock 140268724615680 acquired on /run/cephadm/e721016a-5dc9-11ef-bccf-c7b262605968.lock 2024-08-19T01:25:56.597 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 3300 ... 2024-08-19T01:25:56.597 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 6789 ... 2024-08-19T01:25:56.597 INFO:teuthology.orchestra.run.smithi103.stdout:Base mon IP(s) is [172.21.15.103:3300, 172.21.15.103:6789], mon addrv is [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-08-19T01:25:56.600 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.103 metric 100 2024-08-19T01:25:56.600 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.103 metric 100 2024-08-19T01:25:56.603 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-19T01:25:56.603 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-08-19T01:25:56.603 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-08-19T01:25:56.605 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-19T01:25:56.605 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-19T01:25:56.606 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-19T01:25:56.606 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-08-19T01:25:56.606 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-08-19T01:25:56.606 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-19T01:25:56.606 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-08-19T01:25:56.607 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-08-19T01:25:56.607 INFO:teuthology.orchestra.run.smithi103.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-19T01:25:56.607 INFO:teuthology.orchestra.run.smithi103.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-19T01:25:56.607 INFO:teuthology.orchestra.run.smithi103.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:25:57.454 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba 2024-08-19T01:25:57.454 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-19T01:25:57.455 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Getting image source signatures 2024-08-19T01:25:57.455 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:7556061847b3cc957dc118ff2516cc1cd809c4272dcb16074c5c7fe28afc4774 2024-08-19T01:25:57.455 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-19T01:25:57.455 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying config sha256:ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba 2024-08-19T01:25:57.455 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-19T01:25:58.383 INFO:teuthology.orchestra.run.smithi103.stdout:ceph: stdout ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc) 2024-08-19T01:25:58.383 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph version: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc) 2024-08-19T01:25:58.383 INFO:teuthology.orchestra.run.smithi103.stdout:Extracting ceph user uid/gid from container image... 2024-08-19T01:25:59.357 INFO:teuthology.orchestra.run.smithi103.stdout:stat: stdout 167 167 2024-08-19T01:25:59.357 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial keys... 2024-08-19T01:26:00.421 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAnn8Jm/lMGKRAAWx7K7dl2z0F+iFyiuuYaNw== 2024-08-19T01:26:01.351 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAon8JmNddhLBAAZ8Ew3SjbDLYYUpNK5bKs7Q== 2024-08-19T01:26:02.207 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQApn8JmfLfDJhAAtn8uFwwkQ/4WWhcV5PiTeA== 2024-08-19T01:26:02.208 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial monmap... 2024-08-19T01:26:03.139 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-19T01:26:03.139 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-19T01:26:03.139 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:03.139 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-19T01:26:03.139 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool for a [v2:172.21.15.103:3300,v1:172.21.15.103:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-19T01:26:03.140 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-08-19T01:26:03.140 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: set fsid to e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:03.140 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-19T01:26:03.140 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:26:03.140 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mon... 2024-08-19T01:26:04.095 INFO:teuthology.orchestra.run.smithi103.stdout:create mon.a on 2024-08-19T01:26:04.311 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-19T01:26:04.467 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-19T01:26:04.646 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e721016a-5dc9-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-e721016a-5dc9-11ef-bccf-c7b262605968.target. 2024-08-19T01:26:04.646 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e721016a-5dc9-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-e721016a-5dc9-11ef-bccf-c7b262605968.target. 2024-08-19T01:26:04.910 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a 2024-08-19T01:26:04.910 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a.service: Unit ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a.service not loaded. 2024-08-19T01:26:05.075 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e721016a-5dc9-11ef-bccf-c7b262605968.target.wants/ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a.service → /etc/systemd/system/ceph-e721016a-5dc9-11ef-bccf-c7b262605968@.service. 2024-08-19T01:26:05.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 podman[18046]: 2024-08-19 01:26:05.217040253 +0000 UTC m=+0.017184452 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:26:05.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 podman[18046]: 2024-08-19 01:26:05.418517586 +0000 UTC m=+0.218661775 container create 6b261e6be044edf41be197c3e744ac96034806c49824f47ebe706d1ce8ae2421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-08-19T01:26:05.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 podman[18046]: 2024-08-19 01:26:05.587407942 +0000 UTC m=+0.387552134 container init 6b261e6be044edf41be197c3e744ac96034806c49824f47ebe706d1ce8ae2421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-19T01:26:05.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 podman[18046]: 2024-08-19 01:26:05.590474244 +0000 UTC m=+0.390618434 container start 6b261e6be044edf41be197c3e744ac96034806c49824f47ebe706d1ce8ae2421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-08-19T01:26:05.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 ceph-mon[18066]: mkfs e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:05.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 ceph-mon[18066]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-19T01:26:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:26:05.687 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-08-19T01:26:05.694 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:26:05.694 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-08-19T01:26:05.694 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-08-19T01:26:05.694 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-08-19T01:26:05.695 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon to start... 2024-08-19T01:26:05.695 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon... 2024-08-19T01:26:05.925 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 bash[18046]: 6b261e6be044edf41be197c3e744ac96034806c49824f47ebe706d1ce8ae2421 2024-08-19T01:26:05.926 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:05 smithi103 systemd[1]: Started Ceph mon.a for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:26:07.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: monmap epoch 1 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: last_changed 2024-08-19T01:26:02.528201+0000 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: min_mon_release 19 (squid) 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: election_strategy: 1 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: fsmap 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: osdmap e1: 0 total, 0 up, 0 in 2024-08-19T01:26:07.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: mgrmap e1: no daemons active 2024-08-19T01:26:07.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:06 smithi103 ceph-mon[18066]: from='client.? 172.21.15.103:0/204761461' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-08-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout cluster: 2024-08-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout id: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-19T01:26:10.786 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout services: 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.564209s) 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout data: 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pgs: 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:10.787 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.126+0000 7f215055a640 1 Processor -- start 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.126+0000 7f215055a640 1 -- start start 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.126+0000 7f215055a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214810a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.126+0000 7f215055a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f214807bd90 con 0x7f2148107d70 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.126+0000 7f214e2f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214810a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.127+0000 7f214e2f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214810a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48536/0 (socket says 172.21.15.103:48536) 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.127+0000 7f214e2f6640 1 -- 172.21.15.103:0/507603006 learned_addr learned my addr 172.21.15.103:0/507603006 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.127+0000 7f214e2f6640 1 -- 172.21.15.103:0/507603006 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214810a6a0 con 0x7f2148107d70 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.129+0000 7f214e2f6640 1 --2- 172.21.15.103:0/507603006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214810a160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2144009820 tx=0x7f214402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96b198fe70cdf02 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.129+0000 7f214d2f4640 1 -- 172.21.15.103:0/507603006 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2144037070 con 0x7f2148107d70 2024-08-19T01:26:10.788 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.129+0000 7f214d2f4640 1 -- 172.21.15.103:0/507603006 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2144032440 con 0x7f2148107d70 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.129+0000 7f214d2f4640 1 -- 172.21.15.103:0/507603006 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2144032740 con 0x7f2148107d70 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 -- 172.21.15.103:0/507603006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 msgr2=0x7f214810a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 --2- 172.21.15.103:0/507603006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214810a160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2144009820 tx=0x7f214402a270 comp rx=0 tx=0).stop 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 -- 172.21.15.103:0/507603006 shutdown_connections 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 --2- 172.21.15.103:0/507603006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214810a160 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 -- 172.21.15.103:0/507603006 >> 172.21.15.103:0/507603006 conn(0x7f2148101930 msgr2=0x7f2148103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 -- 172.21.15.103:0/507603006 shutdown_connections 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 -- 172.21.15.103:0/507603006 wait complete. 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 Processor -- start 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 -- start start 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.130+0000 7f215055a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:10.789 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f215055a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f214810ab20 con 0x7f2148107d70 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f214e2f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f214e2f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48540/0 (socket says 172.21.15.103:48540) 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f214e2f6640 1 -- 172.21.15.103:0/204761461 learned_addr learned my addr 172.21.15.103:0/204761461 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f214e2f6640 1 -- 172.21.15.103:0/204761461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214819af30 con 0x7f2148107d70 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f214e2f6640 1 --2- 172.21.15.103:0/204761461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214819a9f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f21440090b0 tx=0x7f21440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f213f7fe640 1 -- 172.21.15.103:0/204761461 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2144041070 con 0x7f2148107d70 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f213f7fe640 1 -- 172.21.15.103:0/204761461 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214403c7a0 con 0x7f2148107d70 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f213f7fe640 1 -- 172.21.15.103:0/204761461 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2144037040 con 0x7f2148107d70 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214819b1c0 con 0x7f2148107d70 2024-08-19T01:26:10.790 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214819deb0 con 0x7f2148107d70 2024-08-19T01:26:10.791 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.131+0000 7f213f7fe640 1 -- 172.21.15.103:0/204761461 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2144044070 con 0x7f2148107d70 2024-08-19T01:26:10.791 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.132+0000 7f213f7fe640 1 -- 172.21.15.103:0/204761461 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f214403d3f0 con 0x7f2148107d70 2024-08-19T01:26:10.791 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.132+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21481081d0 con 0x7f2148107d70 2024-08-19T01:26:10.793 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.134+0000 7f213f7fe640 1 -- 172.21.15.103:0/204761461 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f2144046450 con 0x7f2148107d70 2024-08-19T01:26:10.793 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.192+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f214802d720 con 0x7f2148107d70 2024-08-19T01:26:10.793 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.192+0000 7f213f7fe640 1 -- 172.21.15.103:0/204761461 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2144046630 con 0x7f2148107d70 2024-08-19T01:26:10.793 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.193+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 msgr2=0x7f214819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.193+0000 7f215055a640 1 --2- 172.21.15.103:0/204761461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214819a9f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f21440090b0 tx=0x7f21440046c0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.193+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 shutdown_connections 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.193+0000 7f215055a640 1 --2- 172.21.15.103:0/204761461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2148107d70 0x7f214819a9f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.193+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 >> 172.21.15.103:0/204761461 conn(0x7f2148101930 msgr2=0x7f21481029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.193+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 shutdown_connections 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:06.193+0000 7f215055a640 1 -- 172.21.15.103:0/204761461 wait complete. 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:mon is available 2024-08-19T01:26:10.794 INFO:teuthology.orchestra.run.smithi103.stdout:Assimilating anything we can from ceph.conf... 2024-08-19T01:26:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:14 smithi103 ceph-mon[18066]: from='client.? 172.21.15.103:0/1729609084' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-19T01:26:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:14 smithi103 ceph-mon[18066]: from='client.? 172.21.15.103:0/1729609084' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-19T01:26:18.182 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:18.183 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-08-19T01:26:18.183 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:18.183 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-08-19T01:26:18.183 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-19T01:26:18.183 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-19T01:26:18.183 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-19T01:26:18.184 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-19T01:26:18.184 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:18.184 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-19T01:26:18.184 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-19T01:26:18.184 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:18.184 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-08-19T01:26:18.184 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-08-19T01:26:18.185 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-19T01:26:18.185 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-19T01:26:18.185 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-19T01:26:18.185 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-19T01:26:18.185 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-19T01:26:18.185 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a09c9f640 1 Processor -- start 2024-08-19T01:26:18.185 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a09c9f640 1 -- start start 2024-08-19T01:26:18.186 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a09c9f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a04105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:18.186 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a09c9f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a040631e0 con 0x7f2a04105b20 2024-08-19T01:26:18.186 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a04105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:18.186 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a04105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46792/0 (socket says 172.21.15.103:46792) 2024-08-19T01:26:18.186 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a037fe640 1 -- 172.21.15.103:0/2724715606 learned_addr learned my addr 172.21.15.103:0/2724715606 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:18.186 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.958+0000 7f2a037fe640 1 -- 172.21.15.103:0/2724715606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a04106460 con 0x7f2a04105b20 2024-08-19T01:26:18.186 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.959+0000 7f2a037fe640 1 --2- 172.21.15.103:0/2724715606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a04105f20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009820 tx=0x7f29f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5d0edcfd8328a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:18.187 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.959+0000 7f2a02ffd640 1 -- 172.21.15.103:0/2724715606 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4037070 con 0x7f2a04105b20 2024-08-19T01:26:18.187 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.959+0000 7f2a02ffd640 1 -- 172.21.15.103:0/2724715606 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4032440 con 0x7f2a04105b20 2024-08-19T01:26:18.187 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.959+0000 7f2a02ffd640 1 -- 172.21.15.103:0/2724715606 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4032740 con 0x7f2a04105b20 2024-08-19T01:26:18.187 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.960+0000 7f2a09c9f640 1 -- 172.21.15.103:0/2724715606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 msgr2=0x7f2a04105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:18.187 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.960+0000 7f2a09c9f640 1 --2- 172.21.15.103:0/2724715606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a04105f20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009820 tx=0x7f29f402a270 comp rx=0 tx=0).stop 2024-08-19T01:26:18.187 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.960+0000 7f2a09c9f640 1 -- 172.21.15.103:0/2724715606 shutdown_connections 2024-08-19T01:26:18.188 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.960+0000 7f2a09c9f640 1 --2- 172.21.15.103:0/2724715606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a04105f20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:18.188 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.960+0000 7f2a09c9f640 1 -- 172.21.15.103:0/2724715606 >> 172.21.15.103:0/2724715606 conn(0x7f2a041018f0 msgr2=0x7f2a04103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:18.188 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.960+0000 7f2a09c9f640 1 -- 172.21.15.103:0/2724715606 shutdown_connections 2024-08-19T01:26:18.188 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.960+0000 7f2a09c9f640 1 -- 172.21.15.103:0/2724715606 wait complete. 2024-08-19T01:26:18.188 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a09c9f640 1 Processor -- start 2024-08-19T01:26:18.188 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a09c9f640 1 -- start start 2024-08-19T01:26:18.188 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a09c9f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a0419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:18.189 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a09c9f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04106a00 con 0x7f2a04105b20 2024-08-19T01:26:18.189 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a0419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:18.189 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a0419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46808/0 (socket says 172.21.15.103:46808) 2024-08-19T01:26:18.189 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a037fe640 1 -- 172.21.15.103:0/1729609084 learned_addr learned my addr 172.21.15.103:0/1729609084 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:18.189 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.961+0000 7f2a037fe640 1 -- 172.21.15.103:0/1729609084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0419d130 con 0x7f2a04105b20 2024-08-19T01:26:18.189 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a037fe640 1 --2- 172.21.15.103:0/1729609084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a0419cbf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f29f40090b0 tx=0x7f29f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:18.189 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4041070 con 0x7f2a04105b20 2024-08-19T01:26:18.190 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f403c7a0 con 0x7f2a04105b20 2024-08-19T01:26:18.190 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0419d3c0 con 0x7f2a04105b20 2024-08-19T01:26:18.190 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4037040 con 0x7f2a04105b20 2024-08-19T01:26:18.190 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a041a00b0 con 0x7f2a04105b20 2024-08-19T01:26:18.190 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f29f4044070 con 0x7f2a04105b20 2024-08-19T01:26:18.190 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.962+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f29f403d3f0 con 0x7f2a04105b20 2024-08-19T01:26:18.191 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.963+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a04105f20 con 0x7f2a04105b20 2024-08-19T01:26:18.191 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:13.965+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f29f4046450 con 0x7f2a04105b20 2024-08-19T01:26:18.191 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.023+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2a041a1790 con 0x7f2a04105b20 2024-08-19T01:26:18.191 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.026+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+456 (secure 0 0 0) 0x7f29f4046630 con 0x7f2a04105b20 2024-08-19T01:26:18.191 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.026+0000 7f2a017fa640 1 -- 172.21.15.103:0/1729609084 <== mon.0 v2:172.21.15.103:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f29f4056070 con 0x7f2a04105b20 2024-08-19T01:26:18.191 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.027+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 msgr2=0x7f2a0419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:18.191 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.027+0000 7f2a09c9f640 1 --2- 172.21.15.103:0/1729609084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a0419cbf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f29f40090b0 tx=0x7f29f40046c0 comp rx=0 tx=0).stop 2024-08-19T01:26:18.192 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.027+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 shutdown_connections 2024-08-19T01:26:18.192 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.027+0000 7f2a09c9f640 1 --2- 172.21.15.103:0/1729609084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a04105b20 0x7f2a0419cbf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:18.192 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.027+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 >> 172.21.15.103:0/1729609084 conn(0x7f2a041018f0 msgr2=0x7f2a04193200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:18.192 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.027+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 shutdown_connections 2024-08-19T01:26:18.192 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:14.027+0000 7f2a09c9f640 1 -- 172.21.15.103:0/1729609084 wait complete. 2024-08-19T01:26:18.192 INFO:teuthology.orchestra.run.smithi103.stdout:Generating new minimal ceph.conf... 2024-08-19T01:26:18.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:18 smithi103 ceph-mon[18066]: from='client.? 172.21.15.103:0/3912461945' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:26:19.225 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.579+0000 7f1df0a8a640 1 Processor -- start 2024-08-19T01:26:19.226 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.580+0000 7f1df0a8a640 1 -- start start 2024-08-19T01:26:19.226 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.580+0000 7f1df0a8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:19.226 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.580+0000 7f1df0a8a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec07c870 con 0x7f1dec07de40 2024-08-19T01:26:19.226 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.580+0000 7f1dea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:19.226 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.580+0000 7f1dea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46818/0 (socket says 172.21.15.103:46818) 2024-08-19T01:26:19.227 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.580+0000 7f1dea59c640 1 -- 172.21.15.103:0/3405132479 learned_addr learned my addr 172.21.15.103:0/3405132479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:19.227 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.580+0000 7f1dea59c640 1 -- 172.21.15.103:0/3405132479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec07c9f0 con 0x7f1dec07de40 2024-08-19T01:26:19.227 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.581+0000 7f1dea59c640 1 --2- 172.21.15.103:0/3405132479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec07c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009820 tx=0x7f1ddc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c91620847311f53 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:19.227 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.581+0000 7f1de9d9b640 1 -- 172.21.15.103:0/3405132479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ddc037070 con 0x7f1dec07de40 2024-08-19T01:26:19.227 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.581+0000 7f1de9d9b640 1 -- 172.21.15.103:0/3405132479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1ddc0328d0 con 0x7f1dec07de40 2024-08-19T01:26:19.227 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.581+0000 7f1de9d9b640 1 -- 172.21.15.103:0/3405132479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ddc0307d0 con 0x7f1dec07de40 2024-08-19T01:26:19.228 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3405132479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 msgr2=0x7f1dec07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:19.228 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 --2- 172.21.15.103:0/3405132479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec07c2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009820 tx=0x7f1ddc02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:19.228 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3405132479 shutdown_connections 2024-08-19T01:26:19.228 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 --2- 172.21.15.103:0/3405132479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec07c2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:19.228 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3405132479 >> 172.21.15.103:0/3405132479 conn(0x7f1dec102a10 msgr2=0x7f1dec104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:19.229 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3405132479 shutdown_connections 2024-08-19T01:26:19.229 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3405132479 wait complete. 2024-08-19T01:26:19.229 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.582+0000 7f1df0a8a640 1 Processor -- start 2024-08-19T01:26:19.229 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1df0a8a640 1 -- start start 2024-08-19T01:26:19.229 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1df0a8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:19.230 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1df0a8a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10d830 con 0x7f1dec07de40 2024-08-19T01:26:19.230 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1dea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:19.230 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1dea59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec19d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46830/0 (socket says 172.21.15.103:46830) 2024-08-19T01:26:19.230 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1dea59c640 1 -- 172.21.15.103:0/3912461945 learned_addr learned my addr 172.21.15.103:0/3912461945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:19.230 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1dea59c640 1 -- 172.21.15.103:0/3912461945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec19d900 con 0x7f1dec07de40 2024-08-19T01:26:19.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1dea59c640 1 --2- 172.21.15.103:0/3912461945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec19d3c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0097f0 tx=0x7f1ddc030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:19.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1de37fe640 1 -- 172.21.15.103:0/3912461945 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ddc037040 con 0x7f1dec07de40 2024-08-19T01:26:19.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1de37fe640 1 -- 172.21.15.103:0/3912461945 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1ddc032bb0 con 0x7f1dec07de40 2024-08-19T01:26:19.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec19db90 con 0x7f1dec07de40 2024-08-19T01:26:19.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.583+0000 7f1de37fe640 1 -- 172.21.15.103:0/3912461945 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ddc03a390 con 0x7f1dec07de40 2024-08-19T01:26:19.231 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.584+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec1a0880 con 0x7f1dec07de40 2024-08-19T01:26:19.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.584+0000 7f1de37fe640 1 -- 172.21.15.103:0/3912461945 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ddc03a580 con 0x7f1dec07de40 2024-08-19T01:26:19.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.584+0000 7f1de37fe640 1 -- 172.21.15.103:0/3912461945 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ddc044b50 con 0x7f1dec07de40 2024-08-19T01:26:19.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.584+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec07c320 con 0x7f1dec07de40 2024-08-19T01:26:19.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.587+0000 7f1de37fe640 1 -- 172.21.15.103:0/3912461945 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f1ddc032d60 con 0x7f1dec07de40 2024-08-19T01:26:19.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.643+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1dec1a0e00 con 0x7f1dec07de40 2024-08-19T01:26:19.232 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.643+0000 7f1de37fe640 1 -- 172.21.15.103:0/3912461945 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f1ddc03aa00 con 0x7f1dec07de40 2024-08-19T01:26:19.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.644+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 msgr2=0x7f1dec19d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:19.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.644+0000 7f1df0a8a640 1 --2- 172.21.15.103:0/3912461945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec19d3c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0097f0 tx=0x7f1ddc030d00 comp rx=0 tx=0).stop 2024-08-19T01:26:19.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.644+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 shutdown_connections 2024-08-19T01:26:19.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.644+0000 7f1df0a8a640 1 --2- 172.21.15.103:0/3912461945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dec07de40 0x7f1dec19d3c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:19.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.644+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 >> 172.21.15.103:0/3912461945 conn(0x7f1dec102a10 msgr2=0x7f1dec103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:19.233 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.644+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 shutdown_connections 2024-08-19T01:26:19.234 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:18.644+0000 7f1df0a8a640 1 -- 172.21.15.103:0/3912461945 wait complete. 2024-08-19T01:26:19.234 INFO:teuthology.orchestra.run.smithi103.stdout:Restarting the monitor... 2024-08-19T01:26:19.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:19 smithi103 systemd[1]: Stopping Ceph mon.a for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:26:19.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:19 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a[18062]: 2024-08-19T01:26:19.499+0000 7fc2dce22640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:26:19.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:19 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a[18062]: 2024-08-19T01:26:19.499+0000 7fc2dce22640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-19T01:26:19.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:19 smithi103 podman[18313]: 2024-08-19 01:26:19.65304877 +0000 UTC m=+0.247177077 container died 6b261e6be044edf41be197c3e744ac96034806c49824f47ebe706d1ce8ae2421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-08-19T01:26:20.159 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:19 smithi103 podman[18313]: 2024-08-19 01:26:19.990873121 +0000 UTC m=+0.585001482 container cleanup 6b261e6be044edf41be197c3e744ac96034806c49824f47ebe706d1ce8ae2421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-19T01:26:20.159 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:19 smithi103 bash[18313]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a 2024-08-19T01:26:20.412 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 podman[18326]: 2024-08-19 01:26:20.158195334 +0000 UTC m=+0.505738969 container remove 6b261e6be044edf41be197c3e744ac96034806c49824f47ebe706d1ce8ae2421 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS) 2024-08-19T01:26:20.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 systemd[1]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-19T01:26:20.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 systemd[1]: Stopped Ceph mon.a for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:26:20.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 systemd[1]: Starting Ceph mon.a for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:26:20.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 podman[18416]: 2024-08-19 01:26:20.500663169 +0000 UTC m=+0.017095293 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:26:20.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 podman[18416]: 2024-08-19 01:26:20.617376744 +0000 UTC m=+0.133808854 container create a8f5c18bf93266b7af249d16335fa092d47e692d65fcf542f5bc308827f2e413 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-19T01:26:20.901 INFO:teuthology.orchestra.run.smithi103.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-19T01:26:21.130 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 podman[18416]: 2024-08-19 01:26:20.811530803 +0000 UTC m=+0.327962913 container init a8f5c18bf93266b7af249d16335fa092d47e692d65fcf542f5bc308827f2e413 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:26:21.130 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 podman[18416]: 2024-08-19 01:26:20.81459711 +0000 UTC m=+0.331029224 container start a8f5c18bf93266b7af249d16335fa092d47e692d65fcf542f5bc308827f2e413 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: set uid:gid to 167:167 (ceph:ceph) 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc), process ceph-mon, pid 2 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: pidfile_write: ignore empty --pid-file 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: load: jerasure load: lrc 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: RocksDB version: 7.9.2 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Git sha 0 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Compile date 2024-08-18 15:19:56 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: DB SUMMARY 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: DB Session ID: 8HV5FCOUJIMY7DA501SH 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: CURRENT file: CURRENT 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: IDENTITY file: IDENTITY 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-19T01:26:21.131 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104726 ; 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.error_if_exists: 0 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.create_if_missing: 0 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.paranoid_checks: 1 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.env: 0x55a18652b7a0 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.fs: PosixFileSystem 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.info_log: 0x55a187665500 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_file_opening_threads: 16 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.statistics: (nil) 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.use_fsync: 0 2024-08-19T01:26:21.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_log_file_size: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.keep_log_file_num: 1000 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.recycle_log_file_num: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.allow_fallocate: 1 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.allow_mmap_reads: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.allow_mmap_writes: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.use_direct_reads: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.create_missing_column_families: 0 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.db_log_dir: 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.wal_dir: 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-19T01:26:21.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.advise_random_on_open: 1 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.db_write_buffer_size: 0 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.write_buffer_manager: 0x55a187674dc0 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.rate_limiter: (nil) 2024-08-19T01:26:21.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.wal_recovery_mode: 2 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.enable_thread_tracking: 0 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.enable_pipelined_write: 0 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.unordered_write: 0 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.row_cache: None 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.wal_filter: None 2024-08-19T01:26:21.135 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-19T01:26:21.136 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.allow_ingest_behind: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.two_write_queues: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.manual_wal_flush: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.wal_compression: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.atomic_flush: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.log_readahead_size: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.best_efforts_recovery: 0 2024-08-19T01:26:21.137 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.allow_data_in_errors: 0 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.db_host_id: __hostname__ 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_background_jobs: 2 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_background_compactions: -1 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_subcompactions: 1 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_total_wal_size: 0 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-19T01:26:21.138 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_open_files: -1 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bytes_per_sync: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_readahead_size: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_background_flushes: -1 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Compression algorithms supported: 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kZSTD supported: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kXpressCompression supported: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kBZip2Compression supported: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kLZ4Compression supported: 1 2024-08-19T01:26:21.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kZlibCompression supported: 1 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kLZ4HCCompression supported: 1 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: kSnappyCompression supported: 1 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.merge_operator: 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_filter: None 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_filter_factory: None 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.sst_partitioner_factory: None 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-19T01:26:21.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a187665120) 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_type: 0 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_index_type: 0 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_shortening: 1 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: checksum: 4 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: no_block_cache: 0 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache: 0x55a187659090 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_options: 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: capacity : 536870912 2024-08-19T01:26:21.141 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_shard_bits : 4 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: strict_capacity_limit : 0 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_compressed: (nil) 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: persistent_cache: (nil) 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size: 4096 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size_deviation: 10 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_restart_interval: 16 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_block_restart_interval: 1 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: metadata_block_size: 4096 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: partition_filters: 0 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: use_delta_encoding: 1 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: filter_policy: bloomfilter 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: whole_key_filtering: 1 2024-08-19T01:26:21.142 INFO:journalctl@ceph.mon.a.smithi103.stdout: verify_compression: 0 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: format_version: 5 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_index_compression: 1 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_align: 0 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: max_auto_readahead_size: 262144 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: prepopulate_block_cache: 0 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: initial_auto_readahead_size: 8192 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.write_buffer_size: 33554432 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_write_buffer_number: 2 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression: NoCompression 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression: Disabled 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.prefix_extractor: nullptr 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-19T01:26:21.143 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.num_levels: 7 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.level: 32767 2024-08-19T01:26:21.144 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.strategy: 0 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.enabled: false 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.target_file_size_base: 67108864 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-19T01:26:21.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.arena_block_size: 1048576 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.disable_auto_compactions: 0 2024-08-19T01:26:21.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.inplace_update_support: 0 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-19T01:26:21.147 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.bloom_locality: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.max_successive_merges: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.paranoid_file_checks: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.force_consistency_checks: 1 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.report_bg_io_stats: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.ttl: 2592000 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.enable_blob_files: false 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.min_blob_size: 0 2024-08-19T01:26:21.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.blob_file_size: 268435456 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.blob_file_starting_level: 0 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6454ab7f-44b4-4c68-8774-ff310d8dd0ec 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030780842512, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030780843556, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99889, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97775, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14161, "raw_average_key_size": 49, "raw_value_size": 90070, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724030780, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6454ab7f-44b4-4c68-8774-ff310d8dd0ec", "db_session_id": "8HV5FCOUJIMY7DA501SH", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-19T01:26:21.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030780843654, "job": 1, "event": "recovery_finished"} 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a187686e00 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: DB pointer 0x55a187780000 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** DB Stats ** 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-19T01:26:21.150 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: L0 2/0 99.41 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: Sum 2/0 99.41 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 142.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-08-19T01:26:21.151 INFO:journalctl@ceph.mon.a.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative compaction: 0.00 GB write, 30.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval compaction: 0.00 GB write, 30.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache BinnedLRUCache@0x55a187659090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-08-19T01:26:21.152 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: starting mon.a rank 0 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???) e1 preinit fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???).mds e1 new map 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???).mds e1 print_map 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout: e1 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout: btime 2024-08-19T01:26:05:629511+0000 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout: legacy client fscid: -1 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout: No filesystems configured 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:26:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 bash[18416]: a8f5c18bf93266b7af249d16335fa092d47e692d65fcf542f5bc308827f2e413 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 systemd[1]: Started Ceph mon.a for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: monmap epoch 1 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: last_changed 2024-08-19T01:26:02.528201+0000 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: min_mon_release 19 (squid) 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: election_strategy: 1 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: fsmap 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: osdmap e1: 0 total, 0 up, 0 in 2024-08-19T01:26:21.154 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:20 smithi103 ceph-mon[18436]: mgrmap e1: no daemons active 2024-08-19T01:26:21.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.322+0000 7f43f996b640 1 Processor -- start 2024-08-19T01:26:21.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.322+0000 7f43f996b640 1 -- start start 2024-08-19T01:26:21.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.322+0000 7f43f996b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:21.992 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.322+0000 7f43f996b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43f407c890 con 0x7f43f407de60 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.323+0000 7f43f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.323+0000 7f43f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46840/0 (socket says 172.21.15.103:46840) 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.323+0000 7f43f2ffd640 1 -- 172.21.15.103:0/177633367 learned_addr learned my addr 172.21.15.103:0/177633367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.323+0000 7f43f2ffd640 1 -- 172.21.15.103:0/177633367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f407ca10 con 0x7f43f407de60 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.325+0000 7f43f2ffd640 1 --2- 172.21.15.103:0/177633367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f407c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009820 tx=0x7f43ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7a3a311b1ef762 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.325+0000 7f43f1ffb640 1 -- 172.21.15.103:0/177633367 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43ec037070 con 0x7f43f407de60 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.325+0000 7f43f1ffb640 1 -- 172.21.15.103:0/177633367 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f43ec0328d0 con 0x7f43f407de60 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.325+0000 7f43f1ffb640 1 -- 172.21.15.103:0/177633367 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43ec0307d0 con 0x7f43f407de60 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 -- 172.21.15.103:0/177633367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 msgr2=0x7f43f407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:21.993 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 --2- 172.21.15.103:0/177633367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f407c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009820 tx=0x7f43ec02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 -- 172.21.15.103:0/177633367 shutdown_connections 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 --2- 172.21.15.103:0/177633367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f407c2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 -- 172.21.15.103:0/177633367 >> 172.21.15.103:0/177633367 conn(0x7f43f4102a10 msgr2=0x7f43f4104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 -- 172.21.15.103:0/177633367 shutdown_connections 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 -- 172.21.15.103:0/177633367 wait complete. 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.326+0000 7f43f996b640 1 Processor -- start 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f996b640 1 -- start start 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f996b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:21.994 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f996b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410d810 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f419d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46854/0 (socket says 172.21.15.103:46854) 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f2ffd640 1 -- 172.21.15.103:0/3707838778 learned_addr learned my addr 172.21.15.103:0/3707838778 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f2ffd640 1 -- 172.21.15.103:0/3707838778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f419d900 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.327+0000 7f43f2ffd640 1 --2- 172.21.15.103:0/3707838778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f419d3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009940 tx=0x7f43ec032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.328+0000 7f43f8969640 1 -- 172.21.15.103:0/3707838778 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43ec042070 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.328+0000 7f43f8969640 1 -- 172.21.15.103:0/3707838778 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f43ec030c80 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.328+0000 7f43f8969640 1 -- 172.21.15.103:0/3707838778 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43ec037040 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.328+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f419db90 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.328+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f41a0880 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.328+0000 7f43f8969640 1 -- 172.21.15.103:0/3707838778 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f43ec045070 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.328+0000 7f43f8969640 1 -- 172.21.15.103:0/3707838778 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f43ec047a40 con 0x7f43f407de60 2024-08-19T01:26:21.995 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.329+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f407c340 con 0x7f43f407de60 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.331+0000 7f43f8969640 1 -- 172.21.15.103:0/3707838778 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f43ec03c700 con 0x7f43f407de60 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.386+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f43f41a0a50 con 0x7f43f407de60 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.389+0000 7f43f8969640 1 -- 172.21.15.103:0/3707838778 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f43ec03c8e0 con 0x7f43f407de60 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.390+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 msgr2=0x7f43f419d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.390+0000 7f43f996b640 1 --2- 172.21.15.103:0/3707838778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f419d3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009940 tx=0x7f43ec032e10 comp rx=0 tx=0).stop 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.390+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 shutdown_connections 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.390+0000 7f43f996b640 1 --2- 172.21.15.103:0/3707838778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43f407de60 0x7f43f419d3c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.390+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 >> 172.21.15.103:0/3707838778 conn(0x7f43f4102a10 msgr2=0x7f43f4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.390+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 shutdown_connections 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:21.390+0000 7f43f996b640 1 -- 172.21.15.103:0/3707838778 wait complete. 2024-08-19T01:26:21.996 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-19T01:26:22.010 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:26:22.010 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mgr... 2024-08-19T01:26:22.010 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-19T01:26:22.011 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-19T01:26:22.267 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.x 2024-08-19T01:26:22.267 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.x.service: Unit ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.x.service not loaded. 2024-08-19T01:26:22.426 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e721016a-5dc9-11ef-bccf-c7b262605968.target.wants/ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-e721016a-5dc9-11ef-bccf-c7b262605968@.service. 2024-08-19T01:26:22.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:22 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3707838778' entity='client.admin' 2024-08-19T01:26:22.555 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:22 smithi103 systemd[1]: Starting Ceph mgr.x for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:26:22.940 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:22 smithi103 podman[18649]: 2024-08-19 01:26:22.65317703 +0000 UTC m=+0.107223041 container create acaf1c3cc543f30c208422422017158e8d3b64bf255cd11c3f2800191007d5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:26:22.940 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:22 smithi103 podman[18649]: 2024-08-19 01:26:22.563338442 +0000 UTC m=+0.017384461 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-19T01:26:22.940 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:22 smithi103 podman[18649]: 2024-08-19 01:26:22.855420618 +0000 UTC m=+0.309466629 container init acaf1c3cc543f30c208422422017158e8d3b64bf255cd11c3f2800191007d5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:26:22.940 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:22 smithi103 podman[18649]: 2024-08-19 01:26:22.858237989 +0000 UTC m=+0.312284008 container start acaf1c3cc543f30c208422422017158e8d3b64bf255cd11c3f2800191007d5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:26:22.940 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:22 smithi103 bash[18649]: acaf1c3cc543f30c208422422017158e8d3b64bf255cd11c3f2800191007d5b8 2024-08-19T01:26:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:26:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-08-19T01:26:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:26:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-08-19T01:26:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-08-19T01:26:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-08-19T01:26:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:26:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-08-19T01:26:22.978 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:26:22.978 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-08-19T01:26:22.978 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-08-19T01:26:22.978 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-19T01:26:22.978 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr to start... 2024-08-19T01:26:22.979 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr... 2024-08-19T01:26:23.190 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:22 smithi103 systemd[1]: Started Ceph mgr.x for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:26:23.190 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.074+0000 7f53bc363040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:26:23.190 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.142+0000 7f53bc363040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:26:23.446 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.288+0000 7f53bc363040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:26:23.447 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.445+0000 7f53bc363040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:26:23.715 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.587+0000 7f53bc363040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:26:23.716 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.713+0000 7f53bc363040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:26:24.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.838+0000 7f53bc363040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:26:24.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:23 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:23.896+0000 7f53bc363040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:26:24.459 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.198+0000 7f53bc363040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:26:24.459 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.259+0000 7f53bc363040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:26:24.459 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.325+0000 7f53bc363040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:26:24.796 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/229204459' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-19T01:26:24.797 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.474+0000 7f53bc363040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:26:24.797 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.536+0000 7f53bc363040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:26:24.797 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.606+0000 7f53bc363040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:26:24.797 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.795+0000 7f53bc363040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:26:25.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.855+0000 7f53bc363040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:26:25.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.916+0000 7f53bc363040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:26:25.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:24.983+0000 7f53bc363040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:26:25.482 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:25 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:25.195+0000 7f53bc363040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:26:25.483 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:25 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:25.253+0000 7f53bc363040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:26:26.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:25 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:25.778+0000 7f53bc363040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:26:26.515 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:26 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:26.090+0000 7f53bc363040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:26:26.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:26.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:26:26.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-19T01:26:26.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-19T01:26:26.534 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-19T01:26:26.535 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:26.535 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-19T01:26:26.535 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:26.535 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-19T01:26:26.535 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-19T01:26:26.542 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-19T01:26:26.542 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-19T01:26:26.543 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-08-19T01:26:05:629511+0000", 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-19T01:26:26.544 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-08-19T01:26:05.630146+0000", 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.538+0000 7f42fce69640 1 Processor -- start 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.538+0000 7f42fce69640 1 -- start start 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.538+0000 7f42fce69640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f810a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.538+0000 7f42fce69640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42f807bde0 con 0x7f42f8107d00 2024-08-19T01:26:26.545 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.539+0000 7f42f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f810a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.539+0000 7f42f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f810a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46474/0 (socket says 172.21.15.103:46474) 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.539+0000 7f42f659c640 1 -- 172.21.15.103:0/298786864 learned_addr learned my addr 172.21.15.103:0/298786864 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.539+0000 7f42f659c640 1 -- 172.21.15.103:0/298786864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f810a630 con 0x7f42f8107d00 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.539+0000 7f42f659c640 1 --2- 172.21.15.103:0/298786864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f810a0f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009820 tx=0x7f42e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ed19e888cddf1d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42f5d9b640 1 -- 172.21.15.103:0/298786864 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42e8037070 con 0x7f42f8107d00 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42f5d9b640 1 -- 172.21.15.103:0/298786864 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f42e80328d0 con 0x7f42f8107d00 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42f5d9b640 1 -- 172.21.15.103:0/298786864 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42e80307d0 con 0x7f42f8107d00 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42fce69640 1 -- 172.21.15.103:0/298786864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 msgr2=0x7f42f810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42fce69640 1 --2- 172.21.15.103:0/298786864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f810a0f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009820 tx=0x7f42e802a270 comp rx=0 tx=0).stop 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42fce69640 1 -- 172.21.15.103:0/298786864 shutdown_connections 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42fce69640 1 --2- 172.21.15.103:0/298786864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f810a0f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.540+0000 7f42fce69640 1 -- 172.21.15.103:0/298786864 >> 172.21.15.103:0/298786864 conn(0x7f42f8101900 msgr2=0x7f42f8103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.541+0000 7f42fce69640 1 -- 172.21.15.103:0/298786864 shutdown_connections 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.541+0000 7f42fce69640 1 -- 172.21.15.103:0/298786864 wait complete. 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.541+0000 7f42fce69640 1 Processor -- start 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.541+0000 7f42fce69640 1 -- start start 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.541+0000 7f42fce69640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.541+0000 7f42fce69640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810b610 con 0x7f42f8107d00 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f819b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46490/0 (socket says 172.21.15.103:46490) 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42f659c640 1 -- 172.21.15.103:0/229204459 learned_addr learned my addr 172.21.15.103:0/229204459 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42f659c640 1 -- 172.21.15.103:0/229204459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f819b6e0 con 0x7f42f8107d00 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42f659c640 1 --2- 172.21.15.103:0/229204459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f819b1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42e8032e10 tx=0x7f42e8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42ef7fe640 1 -- 172.21.15.103:0/229204459 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42e8042070 con 0x7f42f8107d00 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f819b970 con 0x7f42f8107d00 2024-08-19T01:26:26.547 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42ef7fe640 1 -- 172.21.15.103:0/229204459 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f42e8030c80 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42ef7fe640 1 -- 172.21.15.103:0/229204459 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42e8037040 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f819e660 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42ef7fe640 1 -- 172.21.15.103:0/229204459 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f42e8045070 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.542+0000 7f42ef7fe640 1 -- 172.21.15.103:0/229204459 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42e8047a40 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.543+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f8108160 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.545+0000 7f42ef7fe640 1 -- 172.21.15.103:0/229204459 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f42e803c6b0 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.606+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f42f819e830 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.606+0000 7f42ef7fe640 1 -- 172.21.15.103:0/229204459 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f42e803c890 con 0x7f42f8107d00 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.607+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 msgr2=0x7f42f819b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.607+0000 7f42fce69640 1 --2- 172.21.15.103:0/229204459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f819b1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42e8032e10 tx=0x7f42e8032e40 comp rx=0 tx=0).stop 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.607+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 shutdown_connections 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.607+0000 7f42fce69640 1 --2- 172.21.15.103:0/229204459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42f8107d00 0x7f42f819b1a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.607+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 >> 172.21.15.103:0/229204459 conn(0x7f42f8101900 msgr2=0x7f42f8102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.607+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 shutdown_connections 2024-08-19T01:26:26.548 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:24.607+0000 7f42fce69640 1 -- 172.21.15.103:0/229204459 wait complete. 2024-08-19T01:26:26.549 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (1/15)... 2024-08-19T01:26:26.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:26 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:26.512+0000 7f53bc363040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:26:26.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:26 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:26.571+0000 7f53bc363040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:26:26.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:26 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:26.629+0000 7f53bc363040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:26:28.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: Activating manager daemon x 2024-08-19T01:26:28.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: mgrmap e2: x(active, starting, since 0.00298486s) 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: Manager daemon x is now available 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' 2024-08-19T01:26:28.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:27 smithi103 ceph-mon[18436]: from='mgr.14100 172.21.15.103:0/3597904714' entity='mgr.x' 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:29.764 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-19T01:26:29.765 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-19T01:26:29.766 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-08-19T01:26:05:629511+0000", 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-19T01:26:29.767 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-19T01:26:29.768 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-08-19T01:26:05.630146+0000", 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaad225640 1 Processor -- start 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaad225640 1 -- start start 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaad225640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa8107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaad225640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8107c10 con 0x7fdaa8107240 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaa6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa8107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaa6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa8107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46564/0 (socket says 172.21.15.103:46564) 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaa6d9d640 1 -- 172.21.15.103:0/96236523 learned_addr learned my addr 172.21.15.103:0/96236523 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.938+0000 7fdaa6d9d640 1 -- 172.21.15.103:0/96236523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa8108440 con 0x7fdaa8107240 2024-08-19T01:26:29.769 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.939+0000 7fdaa6d9d640 1 --2- 172.21.15.103:0/96236523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa8107640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009820 tx=0x7fdaa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5079fb442ed7c6cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.939+0000 7fdaa5d9b640 1 -- 172.21.15.103:0/96236523 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa0037070 con 0x7fdaa8107240 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.939+0000 7fdaa5d9b640 1 -- 172.21.15.103:0/96236523 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdaa00328d0 con 0x7fdaa8107240 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.939+0000 7fdaa5d9b640 1 -- 172.21.15.103:0/96236523 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa00307d0 con 0x7fdaa8107240 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 -- 172.21.15.103:0/96236523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 msgr2=0x7fdaa8107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 --2- 172.21.15.103:0/96236523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa8107640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009820 tx=0x7fdaa002a270 comp rx=0 tx=0).stop 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 -- 172.21.15.103:0/96236523 shutdown_connections 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 --2- 172.21.15.103:0/96236523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa8107640 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 -- 172.21.15.103:0/96236523 >> 172.21.15.103:0/96236523 conn(0x7fdaa8102a10 msgr2=0x7fdaa8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 -- 172.21.15.103:0/96236523 shutdown_connections 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 -- 172.21.15.103:0/96236523 wait complete. 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.940+0000 7fdaad225640 1 Processor -- start 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaad225640 1 -- start start 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaad225640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:29.770 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaad225640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa810b7f0 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaa6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaa6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa819b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46568/0 (socket says 172.21.15.103:46568) 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaa6d9d640 1 -- 172.21.15.103:0/2229348650 learned_addr learned my addr 172.21.15.103:0/2229348650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaa6d9d640 1 -- 172.21.15.103:0/2229348650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa819b720 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaa6d9d640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa819b1e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009940 tx=0x7fdaa0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fda8ffff640 1 -- 172.21.15.103:0/2229348650 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa0042070 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fda8ffff640 1 -- 172.21.15.103:0/2229348650 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdaa0030c80 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa819b9b0 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.941+0000 7fda8ffff640 1 -- 172.21.15.103:0/2229348650 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa0037040 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.942+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa819e6a0 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.942+0000 7fda8ffff640 1 -- 172.21.15.103:0/2229348650 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fdaa0045070 con 0x7fdaa8107240 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.942+0000 7fda8ffff640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7fda90038e10 0x7fda9003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:29.771 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.943+0000 7fda8ffff640 1 -- 172.21.15.103:0/2229348650 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdaa006bc80 con 0x7fdaa8107240 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.942+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa810b970 con 0x7fdaa8107240 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.943+0000 7fdaa659c640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7fda90038e10 0x7fda9003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.943+0000 7fdaa659c640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7fda90038e10 0x7fda9003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00ad30 tx=0x7fda9c0093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:28.947+0000 7fda8ffff640 1 -- 172.21.15.103:0/2229348650 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdaa0045360 con 0x7fdaa8107240 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.177+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdaa819ecf0 con 0x7fdaa8107240 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.178+0000 7fda8ffff640 1 -- 172.21.15.103:0/2229348650 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fdaa002f2a0 con 0x7fdaa8107240 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7fda90038e10 msgr2=0x7fda9003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7fda90038e10 0x7fda9003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00ad30 tx=0x7fda9c0093f0 comp rx=0 tx=0).stop 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 msgr2=0x7fdaa819b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa819b1e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009940 tx=0x7fdaa0032e10 comp rx=0 tx=0).stop 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 shutdown_connections 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7fda90038e10 0x7fda9003b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:29.772 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 --2- 172.21.15.103:0/2229348650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaa8107240 0x7fdaa819b1e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:29.773 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 >> 172.21.15.103:0/2229348650 conn(0x7fdaa8102a10 msgr2=0x7fdaa81034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:29.773 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 shutdown_connections 2024-08-19T01:26:29.773 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:29.180+0000 7fdaad225640 1 -- 172.21.15.103:0/2229348650 wait complete. 2024-08-19T01:26:29.773 INFO:teuthology.orchestra.run.smithi103.stdout:mgr is available 2024-08-19T01:26:29.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:29 smithi103 ceph-mon[18436]: mgrmap e3: x(active, since 1.00632s) 2024-08-19T01:26:29.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:29 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2229348650' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-19T01:26:30.910 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.169+0000 7f9947ec2640 1 Processor -- start 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.169+0000 7f9947ec2640 1 -- start start 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.169+0000 7f9947ec2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f994010a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:30.911 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.169+0000 7f9947ec2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99400631c0 con 0x7f9940107d50 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.170+0000 7f9945c5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f994010a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.170+0000 7f9945c5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f994010a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46576/0 (socket says 172.21.15.103:46576) 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.170+0000 7f9945c5e640 1 -- 172.21.15.103:0/1371869610 learned_addr learned my addr 172.21.15.103:0/1371869610 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.170+0000 7f9945c5e640 1 -- 172.21.15.103:0/1371869610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994010a680 con 0x7f9940107d50 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9945c5e640 1 --2- 172.21.15.103:0/1371869610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f994010a140 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9930009e80 tx=0x7f993002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57e4c6bf7a118c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9944c5c640 1 -- 172.21.15.103:0/1371869610 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930036070 con 0x7f9940107d50 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9944c5c640 1 -- 172.21.15.103:0/1371869610 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9930031890 con 0x7f9940107d50 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9944c5c640 1 -- 172.21.15.103:0/1371869610 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930007980 con 0x7f9940107d50 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9947ec2640 1 -- 172.21.15.103:0/1371869610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 msgr2=0x7f994010a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9947ec2640 1 --2- 172.21.15.103:0/1371869610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f994010a140 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9930009e80 tx=0x7f993002a7a0 comp rx=0 tx=0).stop 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9947ec2640 1 -- 172.21.15.103:0/1371869610 shutdown_connections 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9947ec2640 1 --2- 172.21.15.103:0/1371869610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f994010a140 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9947ec2640 1 -- 172.21.15.103:0/1371869610 >> 172.21.15.103:0/1371869610 conn(0x7f99401018f0 msgr2=0x7f9940103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:30.912 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.171+0000 7f9947ec2640 1 -- 172.21.15.103:0/1371869610 shutdown_connections 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9947ec2640 1 -- 172.21.15.103:0/1371869610 wait complete. 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9947ec2640 1 Processor -- start 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9947ec2640 1 -- start start 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9947ec2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f99401075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9947ec2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f994010b600 con 0x7f9940107d50 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9945c5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f99401075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9945c5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f99401075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46580/0 (socket says 172.21.15.103:46580) 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.172+0000 7f9945c5e640 1 -- 172.21.15.103:0/2351816874 learned_addr learned my addr 172.21.15.103:0/2351816874 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.173+0000 7f9945c5e640 1 -- 172.21.15.103:0/2351816874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9940107b10 con 0x7f9940107d50 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.173+0000 7f9945c5e640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f99401075d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9930031c70 tx=0x7f9930031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.173+0000 7f993effd640 1 -- 172.21.15.103:0/2351816874 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930041020 con 0x7f9940107d50 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.173+0000 7f993effd640 1 -- 172.21.15.103:0/2351816874 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f993003c900 con 0x7f9940107d50 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.173+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9940105d00 con 0x7f9940107d50 2024-08-19T01:26:30.913 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.173+0000 7f993effd640 1 -- 172.21.15.103:0/2351816874 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930036040 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.173+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9940106240 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.174+0000 7f993effd640 1 -- 172.21.15.103:0/2351816874 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f9930044020 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.174+0000 7f993effd640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f9928038f20 0x7f992803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.174+0000 7f993effd640 1 -- 172.21.15.103:0/2351816874 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f993006d0d0 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.174+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99401081b0 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.174+0000 7f994545d640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f9928038f20 0x7f992803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.175+0000 7f994545d640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f9928038f20 0x7f992803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99340098c0 tx=0x7f9934006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.179+0000 7f993effd640 1 -- 172.21.15.103:0/2351816874 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f993004b020 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.345+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9940106b00 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.346+0000 7f993effd640 1 -- 172.21.15.103:0/2351816874 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+456 (secure 0 0 0) 0x7f993004c260 con 0x7f9940107d50 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f9928038f20 msgr2=0x7f992803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f9928038f20 0x7f992803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99340098c0 tx=0x7f9934006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:30.914 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 msgr2=0x7f99401075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f99401075d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9930031c70 tx=0x7f9930031ca0 comp rx=0 tx=0).stop 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 shutdown_connections 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f9928038f20 0x7f992803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 --2- 172.21.15.103:0/2351816874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9940107d50 0x7f99401075d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 >> 172.21.15.103:0/2351816874 conn(0x7f99401018f0 msgr2=0x7f9940102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 shutdown_connections 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:30.348+0000 7f9947ec2640 1 -- 172.21.15.103:0/2351816874 wait complete. 2024-08-19T01:26:30.915 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling cephadm module... 2024-08-19T01:26:31.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:30 smithi103 ceph-mon[18436]: mgrmap e4: x(active, since 2s) 2024-08-19T01:26:31.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:30 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2351816874' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-19T01:26:32.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:31 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/807837418' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-08-19T01:26:32.002 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:31 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: ignoring --setuser ceph since I am not root 2024-08-19T01:26:32.002 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:31 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: ignoring --setgroup ceph since I am not root 2024-08-19T01:26:32.002 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:31 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:31.933+0000 7fb8f7d5f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:26:32.002 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:31.998+0000 7fb8f7d5f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:26:32.280 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:32.142+0000 7fb8f7d5f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:26:32.287 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.294+0000 7f06ff7a6640 1 Processor -- start 2024-08-19T01:26:32.287 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.295+0000 7f06ff7a6640 1 -- start start 2024-08-19T01:26:32.287 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.295+0000 7f06ff7a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f807c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.295+0000 7f06ff7a6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06f807c830 con 0x7f06f807de00 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.295+0000 7f06fd542640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f807c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.295+0000 7f06fd542640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f807c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46582/0 (socket says 172.21.15.103:46582) 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.295+0000 7f06fd542640 1 -- 172.21.15.103:0/4173269501 learned_addr learned my addr 172.21.15.103:0/4173269501 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.295+0000 7f06fd542640 1 -- 172.21.15.103:0/4173269501 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f807c9b0 con 0x7f06f807de00 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.296+0000 7f06fd542640 1 --2- 172.21.15.103:0/4173269501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f807c260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009820 tx=0x7f06ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2353895df571fa1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.296+0000 7f06ebfff640 1 -- 172.21.15.103:0/4173269501 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec037070 con 0x7f06f807de00 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.296+0000 7f06ebfff640 1 -- 172.21.15.103:0/4173269501 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f06ec0328d0 con 0x7f06f807de00 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.296+0000 7f06ebfff640 1 -- 172.21.15.103:0/4173269501 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec0307d0 con 0x7f06f807de00 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 -- 172.21.15.103:0/4173269501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 msgr2=0x7f06f807c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 --2- 172.21.15.103:0/4173269501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f807c260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009820 tx=0x7f06ec02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 -- 172.21.15.103:0/4173269501 shutdown_connections 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 --2- 172.21.15.103:0/4173269501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f807c260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:32.288 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 -- 172.21.15.103:0/4173269501 >> 172.21.15.103:0/4173269501 conn(0x7f06f8102a10 msgr2=0x7f06f8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 -- 172.21.15.103:0/4173269501 shutdown_connections 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 -- 172.21.15.103:0/4173269501 wait complete. 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 Processor -- start 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.297+0000 7f06ff7a6640 1 -- start start 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06ff7a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06ff7a6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06f810d810 con 0x7f06f807de00 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06fd542640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06fd542640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f819d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46592/0 (socket says 172.21.15.103:46592) 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06fd542640 1 -- 172.21.15.103:0/807837418 learned_addr learned my addr 172.21.15.103:0/807837418 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06fd542640 1 -- 172.21.15.103:0/807837418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f819d8c0 con 0x7f06f807de00 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06fd542640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f819d380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f06ec032e10 tx=0x7f06ec032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec042070 con 0x7f06f807de00 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f819db50 con 0x7f06f807de00 2024-08-19T01:26:32.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f06ec030c80 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec037040 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.298+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f81a0840 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.299+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f06ec045070 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.300+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f807c2e0 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.300+0000 7f06ea7fc640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f06e4038f70 0x7f06e403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.300+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f06ec06be40 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.300+0000 7f06fcd41640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f06e4038f70 0x7f06e403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.300+0000 7f06fcd41640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f06e4038f70 0x7f06e403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f06f40098c0 tx=0x7f06f4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.304+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f06ec045360 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.513+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f06f81a0ec0 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.707+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f06ec06b860 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.709+0000 7f06ea7fc640 1 -- 172.21.15.103:0/807837418 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f06ec0313a0 con 0x7f06f807de00 2024-08-19T01:26:32.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.709+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f06e4038f70 msgr2=0x7f06e403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.709+0000 7f06ff7a6640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f06e4038f70 0x7f06e403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f06f40098c0 tx=0x7f06f4006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 msgr2=0x7f06f819d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f819d380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f06ec032e10 tx=0x7f06ec032e40 comp rx=0 tx=0).stop 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 shutdown_connections 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f06e4038f70 0x7f06e403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 --2- 172.21.15.103:0/807837418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f807de00 0x7f06f819d380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 >> 172.21.15.103:0/807837418 conn(0x7f06f8102a10 msgr2=0x7f06f8103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 shutdown_connections 2024-08-19T01:26:32.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:31.710+0000 7f06ff7a6640 1 -- 172.21.15.103:0/807837418 wait complete. 2024-08-19T01:26:32.554 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:32.303+0000 7fb8f7d5f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:26:32.554 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:32.449+0000 7fb8f7d5f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:26:32.966 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/807837418' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-19T01:26:32.967 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-mon[18436]: mgrmap e5: x(active, since 4s) 2024-08-19T01:26:32.967 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:32.575+0000 7fb8f7d5f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:26:32.967 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:32.700+0000 7fb8f7d5f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:26:32.967 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:32 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:32.762+0000 7fb8f7d5f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:26:33.304 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.078+0000 7fb8f7d5f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:26:33.304 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.139+0000 7fb8f7d5f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:26:33.304 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.207+0000 7fb8f7d5f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:26:33.611 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.356+0000 7fb8f7d5f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:26:33.611 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.415+0000 7fb8f7d5f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:26:33.611 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.480+0000 7fb8f7d5f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:26:33.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:26:33.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-19T01:26:33.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-19T01:26:33.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-19T01:26:33.623 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-19T01:26:33.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:26:33.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.757+0000 7f86c38cd640 1 Processor -- start 2024-08-19T01:26:33.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.758+0000 7f86c38cd640 1 -- start start 2024-08-19T01:26:33.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.758+0000 7f86c38cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc10a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:33.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.758+0000 7f86c38cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc063240 con 0x7f86bc107cd0 2024-08-19T01:26:33.624 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.758+0000 7f86c1669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc10a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:33.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.758+0000 7f86c1669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc10a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37508/0 (socket says 172.21.15.103:37508) 2024-08-19T01:26:33.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.758+0000 7f86c1669640 1 -- 172.21.15.103:0/2315636316 learned_addr learned my addr 172.21.15.103:0/2315636316 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:33.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.758+0000 7f86c1669640 1 -- 172.21.15.103:0/2315636316 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc10a600 con 0x7f86bc107cd0 2024-08-19T01:26:33.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.759+0000 7f86c1669640 1 --2- 172.21.15.103:0/2315636316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc10a0c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009820 tx=0x7f86b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3d1cf50f7d7517 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:33.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.759+0000 7f86b3fff640 1 -- 172.21.15.103:0/2315636316 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b8037070 con 0x7f86bc107cd0 2024-08-19T01:26:33.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.759+0000 7f86b3fff640 1 -- 172.21.15.103:0/2315636316 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f86b80328d0 con 0x7f86bc107cd0 2024-08-19T01:26:33.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.759+0000 7f86b3fff640 1 -- 172.21.15.103:0/2315636316 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b80307d0 con 0x7f86bc107cd0 2024-08-19T01:26:33.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.760+0000 7f86c38cd640 1 -- 172.21.15.103:0/2315636316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 msgr2=0x7f86bc10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:33.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.760+0000 7f86c38cd640 1 --2- 172.21.15.103:0/2315636316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc10a0c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009820 tx=0x7f86b802a270 comp rx=0 tx=0).stop 2024-08-19T01:26:33.626 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.760+0000 7f86c38cd640 1 -- 172.21.15.103:0/2315636316 shutdown_connections 2024-08-19T01:26:33.627 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.760+0000 7f86c38cd640 1 --2- 172.21.15.103:0/2315636316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc10a0c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:33.627 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.760+0000 7f86c38cd640 1 -- 172.21.15.103:0/2315636316 >> 172.21.15.103:0/2315636316 conn(0x7f86bc1018f0 msgr2=0x7f86bc103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:33.627 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.760+0000 7f86c38cd640 1 -- 172.21.15.103:0/2315636316 shutdown_connections 2024-08-19T01:26:33.627 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.760+0000 7f86c38cd640 1 -- 172.21.15.103:0/2315636316 wait complete. 2024-08-19T01:26:33.627 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.761+0000 7f86c38cd640 1 Processor -- start 2024-08-19T01:26:33.627 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.761+0000 7f86c38cd640 1 -- start start 2024-08-19T01:26:33.628 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.761+0000 7f86c38cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:33.628 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.761+0000 7f86c38cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc10b640 con 0x7f86bc107cd0 2024-08-19T01:26:33.628 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.761+0000 7f86c1669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:33.628 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.761+0000 7f86c1669640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37524/0 (socket says 172.21.15.103:37524) 2024-08-19T01:26:33.628 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.761+0000 7f86c1669640 1 -- 172.21.15.103:0/2571058155 learned_addr learned my addr 172.21.15.103:0/2571058155 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:33.628 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.762+0000 7f86c1669640 1 -- 172.21.15.103:0/2571058155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc19b6d0 con 0x7f86bc107cd0 2024-08-19T01:26:33.629 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.762+0000 7f86c1669640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc19b190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86b80097f0 tx=0x7f86b8030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:33.629 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.762+0000 7f86b27fc640 1 -- 172.21.15.103:0/2571058155 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b8037040 con 0x7f86bc107cd0 2024-08-19T01:26:33.629 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.762+0000 7f86b27fc640 1 -- 172.21.15.103:0/2571058155 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f86b8032bb0 con 0x7f86bc107cd0 2024-08-19T01:26:33.629 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.762+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc19b960 con 0x7f86bc107cd0 2024-08-19T01:26:33.629 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.762+0000 7f86b27fc640 1 -- 172.21.15.103:0/2571058155 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86b803a390 con 0x7f86bc107cd0 2024-08-19T01:26:33.629 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.762+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc19e650 con 0x7f86bc107cd0 2024-08-19T01:26:33.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.763+0000 7f86b27fc640 1 -- 172.21.15.103:0/2571058155 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f86b803a580 con 0x7f86bc107cd0 2024-08-19T01:26:33.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.763+0000 7f86b27fc640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 0x7f86a803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:33.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.763+0000 7f86b27fc640 1 -- 172.21.15.103:0/2571058155 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f86b806c1b0 con 0x7f86bc107cd0 2024-08-19T01:26:33.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.763+0000 7f86c0e68640 1 -- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 msgr2=0x7f86a803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/3969074886 2024-08-19T01:26:33.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.763+0000 7f86c0e68640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 0x7f86a803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:33.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.763+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc108130 con 0x7f86bc107cd0 2024-08-19T01:26:33.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.772+0000 7f86b27fc640 1 -- 172.21.15.103:0/2571058155 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f86b8040210 con 0x7f86bc107cd0 2024-08-19T01:26:33.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.964+0000 7f86c0e68640 1 -- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 msgr2=0x7f86a803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/3969074886 2024-08-19T01:26:33.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.964+0000 7f86c0e68640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 0x7f86a803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:26:33.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.993+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f86bc19ebb0 con 0x7f86bc107cd0 2024-08-19T01:26:33.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.994+0000 7f86b27fc640 1 -- 172.21.15.103:0/2571058155 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f86b8031560 con 0x7f86bc107cd0 2024-08-19T01:26:33.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 msgr2=0x7f86a803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 0x7f86a803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 msgr2=0x7f86bc19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc19b190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86b80097f0 tx=0x7f86b8030d00 comp rx=0 tx=0).stop 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 shutdown_connections 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f86a8038fc0 0x7f86a803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 --2- 172.21.15.103:0/2571058155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86bc107cd0 0x7f86bc19b190 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 >> 172.21.15.103:0/2571058155 conn(0x7f86bc1018f0 msgr2=0x7f86bc102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 shutdown_connections 2024-08-19T01:26:33.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:32.996+0000 7f86c38cd640 1 -- 172.21.15.103:0/2571058155 wait complete. 2024-08-19T01:26:33.633 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-08-19T01:26:33.633 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 5... 2024-08-19T01:26:33.861 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2571058155' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-19T01:26:33.861 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.670+0000 7fb8f7d5f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:26:33.861 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.731+0000 7fb8f7d5f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:26:33.862 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.793+0000 7fb8f7d5f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:26:34.143 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:33.863+0000 7fb8f7d5f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:26:34.143 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:34 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:34.082+0000 7fb8f7d5f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:26:34.143 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:34 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:34.140+0000 7fb8f7d5f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:26:34.955 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:34 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:34.649+0000 7fb8f7d5f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:26:35.304 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:34 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:34.952+0000 7fb8f7d5f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:26:35.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:35 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:35.362+0000 7fb8f7d5f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:26:35.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:35 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:35.417+0000 7fb8f7d5f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:26:35.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:35 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:35.477+0000 7fb8f7d5f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:26:37.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: Active manager daemon x restarted 2024-08-19T01:26:37.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: Activating manager daemon x 2024-08-19T01:26:37.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: osdmap e2: 0 total, 0 up, 0 in 2024-08-19T01:26:37.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: mgrmap e6: x(active, starting, since 0.00392408s) 2024-08-19T01:26:37.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:26:37.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-19T01:26:37.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-19T01:26:37.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-19T01:26:37.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-19T01:26:37.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: Manager daemon x is now available 2024-08-19T01:26:37.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:37.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:37.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:37.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:37.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-19T01:26:37.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:36 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-19T01:26:37.917 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:37 smithi103 ceph-mon[18436]: Found migration_current of "None". Setting to last migration. 2024-08-19T01:26:37.917 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:37 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:37.917 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:37 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:37.917 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:37 smithi103 ceph-mon[18436]: mgrmap e7: x(active, since 1.00514s) 2024-08-19T01:26:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:26:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-19T01:26:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-19T01:26:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:26:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.040+0000 7f2273d86640 1 Processor -- start 2024-08-19T01:26:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.040+0000 7f2273d86640 1 -- start start 2024-08-19T01:26:38.237 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.040+0000 7f2273d86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c10a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.040+0000 7f2273d86640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f226c063200 con 0x7f226c107d10 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.040+0000 7f2271b22640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c10a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.040+0000 7f2271b22640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c10a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37536/0 (socket says 172.21.15.103:37536) 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.040+0000 7f2271b22640 1 -- 172.21.15.103:0/1254569034 learned_addr learned my addr 172.21.15.103:0/1254569034 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.041+0000 7f2271b22640 1 -- 172.21.15.103:0/1254569034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c10a640 con 0x7f226c107d10 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.041+0000 7f2271b22640 1 --2- 172.21.15.103:0/1254569034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c10a100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2268009820 tx=0x7f226802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44be099f1e080a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.041+0000 7f2270b20640 1 -- 172.21.15.103:0/1254569034 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2268037070 con 0x7f226c107d10 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.042+0000 7f2270b20640 1 -- 172.21.15.103:0/1254569034 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f22680328d0 con 0x7f226c107d10 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.042+0000 7f2270b20640 1 -- 172.21.15.103:0/1254569034 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22680307d0 con 0x7f226c107d10 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.042+0000 7f2273d86640 1 -- 172.21.15.103:0/1254569034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 msgr2=0x7f226c10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.042+0000 7f2273d86640 1 --2- 172.21.15.103:0/1254569034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c10a100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2268009820 tx=0x7f226802a270 comp rx=0 tx=0).stop 2024-08-19T01:26:38.238 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.043+0000 7f2273d86640 1 -- 172.21.15.103:0/1254569034 shutdown_connections 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.043+0000 7f2273d86640 1 --2- 172.21.15.103:0/1254569034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c10a100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.043+0000 7f2273d86640 1 -- 172.21.15.103:0/1254569034 >> 172.21.15.103:0/1254569034 conn(0x7f226c1018f0 msgr2=0x7f226c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.043+0000 7f2273d86640 1 -- 172.21.15.103:0/1254569034 shutdown_connections 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.043+0000 7f2273d86640 1 -- 172.21.15.103:0/1254569034 wait complete. 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.043+0000 7f2273d86640 1 Processor -- start 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.043+0000 7f2273d86640 1 -- start start 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2273d86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2273d86640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10b620 con 0x7f226c107d10 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2271b22640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2271b22640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37540/0 (socket says 172.21.15.103:37540) 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2271b22640 1 -- 172.21.15.103:0/129888347 learned_addr learned my addr 172.21.15.103:0/129888347 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2271b22640 1 -- 172.21.15.103:0/129888347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c19b6d0 con 0x7f226c107d10 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2271b22640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c19b190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2268032e10 tx=0x7f2268032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:38.239 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2268042070 con 0x7f226c107d10 2024-08-19T01:26:38.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2268030c80 con 0x7f226c107d10 2024-08-19T01:26:38.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c19b960 con 0x7f226c107d10 2024-08-19T01:26:38.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2268037040 con 0x7f226c107d10 2024-08-19T01:26:38.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.044+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c19e650 con 0x7f226c107d10 2024-08-19T01:26:38.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.045+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f2268045070 con 0x7f226c107d10 2024-08-19T01:26:38.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.046+0000 7f2262ffd640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 0x7f225803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:38.240 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.046+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 --> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f225803bb40 con 0x7f2258038f70 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.046+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f226806bef0 con 0x7f226c107d10 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.046+0000 7f2271321640 1 -- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 msgr2=0x7f225803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/3969074886 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.046+0000 7f2271321640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 0x7f225803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.247+0000 7f2271321640 1 -- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 msgr2=0x7f225803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/3969074886 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.247+0000 7f2271321640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 0x7f225803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.648+0000 7f2271321640 1 -- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 msgr2=0x7f225803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/3969074886 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:34.648+0000 7f2271321640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 0x7f225803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:35.449+0000 7f2271321640 1 -- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 msgr2=0x7f225803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/3969074886 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:35.449+0000 7f2271321640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 0x7f225803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:36.590+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f226806a9b0 con 0x7f226c107d10 2024-08-19T01:26:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:36.590+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 msgr2=0x7f225803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:38.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:36.590+0000 7f2262ffd640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 0x7f225803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:38.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.591+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f22680313a0 con 0x7f226c107d10 2024-08-19T01:26:38.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.591+0000 7f2262ffd640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f225803c8a0 0x7f225803ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:38.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.591+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f226802bba0 con 0x7f225803c8a0 2024-08-19T01:26:38.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.591+0000 7f2271321640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f225803c8a0 0x7f225803ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:38.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.592+0000 7f2271321640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f225803c8a0 0x7f225803ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f225c003980 tx=0x7f225c006e10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:38.242 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.593+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f226802bba0 con 0x7f225803c8a0 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.599+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f226c108170 con 0x7f225803c8a0 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2262ffd640 1 -- 172.21.15.103:0/129888347 <== mgr.14116 v2:172.21.15.103:6800/2881014674 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f226c108170 con 0x7f225803c8a0 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f225803c8a0 msgr2=0x7f225803ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f225803c8a0 0x7f225803ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f225c003980 tx=0x7f225c006e10 comp rx=0 tx=0).stop 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 msgr2=0x7f226c19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c19b190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2268032e10 tx=0x7f2268032e40 comp rx=0 tx=0).stop 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 shutdown_connections 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f225803c8a0 0x7f225803ec90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:38.243 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:6800/3969074886,v1:172.21.15.103:6801/3969074886] conn(0x7f2258038f70 0x7f225803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:38.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 --2- 172.21.15.103:0/129888347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226c107d10 0x7f226c19b190 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:38.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 >> 172.21.15.103:0/129888347 conn(0x7f226c1018f0 msgr2=0x7f226c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:38.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 shutdown_connections 2024-08-19T01:26:38.244 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:37.600+0000 7f2273d86640 1 -- 172.21.15.103:0/129888347 wait complete. 2024-08-19T01:26:38.244 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 5 is available 2024-08-19T01:26:38.244 INFO:teuthology.orchestra.run.smithi103.stdout:Setting orchestrator backend to cephadm... 2024-08-19T01:26:38.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-mon[18436]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-19T01:26:38.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-mon[18436]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-19T01:26:38.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:38.587+0000 7fb85f26d640 -1 log_channel(cephadm) log [ERR] : [19/Aug/2024:01:26:38] ENGINE Error in HTTPServer.serve 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: Traceback (most recent call last): 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self._connections.run(self.expiration_interval) 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self._run(expiration_interval) 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:26:38.805 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: s = self.context.wrap_socket( 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: return self.sslsocket_class._create( 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self.do_handshake() 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self._sslobj.do_handshake() 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:26:38.806 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:38 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.648+0000 7f68186a7640 1 Processor -- start 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.648+0000 7f68186a7640 1 -- start start 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.649+0000 7f68186a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f6810107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.649+0000 7f68186a7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6810063240 con 0x7f6810105ac0 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.649+0000 7f6816443640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f6810107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.649+0000 7f6816443640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f6810107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37620/0 (socket says 172.21.15.103:37620) 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.649+0000 7f6816443640 1 -- 172.21.15.103:0/3421677126 learned_addr learned my addr 172.21.15.103:0/3421677126 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:39.338 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.649+0000 7f6816443640 1 -- 172.21.15.103:0/3421677126 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6810108410 con 0x7f6810105ac0 2024-08-19T01:26:39.339 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.650+0000 7f6816443640 1 --2- 172.21.15.103:0/3421677126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f6810107ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f680c009820 tx=0x7f680c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1def38c611345d3d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:39.339 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.650+0000 7f6815441640 1 -- 172.21.15.103:0/3421677126 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f680c037070 con 0x7f6810105ac0 2024-08-19T01:26:39.343 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.650+0000 7f6815441640 1 -- 172.21.15.103:0/3421677126 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f680c0328d0 con 0x7f6810105ac0 2024-08-19T01:26:39.343 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.650+0000 7f6815441640 1 -- 172.21.15.103:0/3421677126 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f680c0307d0 con 0x7f6810105ac0 2024-08-19T01:26:39.343 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.651+0000 7f68186a7640 1 -- 172.21.15.103:0/3421677126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 msgr2=0x7f6810107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:39.343 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.651+0000 7f68186a7640 1 --2- 172.21.15.103:0/3421677126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f6810107ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f680c009820 tx=0x7f680c02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.651+0000 7f68186a7640 1 -- 172.21.15.103:0/3421677126 shutdown_connections 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.651+0000 7f68186a7640 1 --2- 172.21.15.103:0/3421677126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f6810107ed0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.651+0000 7f68186a7640 1 -- 172.21.15.103:0/3421677126 >> 172.21.15.103:0/3421677126 conn(0x7f68101018f0 msgr2=0x7f6810103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.651+0000 7f68186a7640 1 -- 172.21.15.103:0/3421677126 shutdown_connections 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.651+0000 7f68186a7640 1 -- 172.21.15.103:0/3421677126 wait complete. 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68186a7640 1 Processor -- start 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68186a7640 1 -- start start 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68186a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f68101a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68186a7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68101094b0 con 0x7f6810105ac0 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f6816443640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f68101a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f6816443640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f68101a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37630/0 (socket says 172.21.15.103:37630) 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f6816443640 1 -- 172.21.15.103:0/2380414024 learned_addr learned my addr 172.21.15.103:0/2380414024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:39.344 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f6816443640 1 -- 172.21.15.103:0/2380414024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68101a1cb0 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f6816443640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f68101a1770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097f0 tx=0x7f680c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68077fe640 1 -- 172.21.15.103:0/2380414024 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f680c037040 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68077fe640 1 -- 172.21.15.103:0/2380414024 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f680c032bb0 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68101a1f40 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.652+0000 7f68077fe640 1 -- 172.21.15.103:0/2380414024 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f680c03a390 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.653+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68101a4c30 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.653+0000 7f68077fe640 1 -- 172.21.15.103:0/2380414024 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f680c03b3f0 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.654+0000 7f68077fe640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f67fc038fc0 0x7f67fc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.654+0000 7f68077fe640 1 -- 172.21.15.103:0/2380414024 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f680c06c260 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.654+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6810105ec0 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.654+0000 7f6815c42640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f67fc038fc0 0x7f67fc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.654+0000 7f6815c42640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f67fc038fc0 0x7f67fc03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68000098c0 tx=0x7f6800006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.659+0000 7f68077fe640 1 -- 172.21.15.103:0/2380414024 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f680c040210 con 0x7f6810105ac0 2024-08-19T01:26:39.345 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.842+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f681002d8c0 con 0x7f67fc038fc0 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.846+0000 7f68077fe640 1 -- 172.21.15.103:0/2380414024 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f681002d8c0 con 0x7f67fc038fc0 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.848+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f67fc038fc0 msgr2=0x7f67fc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.848+0000 7f68186a7640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f67fc038fc0 0x7f67fc03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68000098c0 tx=0x7f6800006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.848+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 msgr2=0x7f68101a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.848+0000 7f68186a7640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f68101a1770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097f0 tx=0x7f680c030d00 comp rx=0 tx=0).stop 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.849+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 shutdown_connections 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.849+0000 7f68186a7640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f67fc038fc0 0x7f67fc03b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.849+0000 7f68186a7640 1 --2- 172.21.15.103:0/2380414024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6810105ac0 0x7f68101a1770 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.849+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 >> 172.21.15.103:0/2380414024 conn(0x7f68101018f0 msgr2=0x7f6810103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.849+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 shutdown_connections 2024-08-19T01:26:39.346 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:38.849+0000 7f68186a7640 1 -- 172.21.15.103:0/2380414024 wait complete. 2024-08-19T01:26:39.930 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: [19/Aug/2024:01:26:38] ENGINE Bus STARTING 2024-08-19T01:26:39.930 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: [19/Aug/2024:01:26:38] ENGINE Serving on http://172.21.15.103:8765 2024-08-19T01:26:39.930 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: [19/Aug/2024:01:26:38] ENGINE Serving on https://172.21.15.103:7150 2024-08-19T01:26:39.930 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: [19/Aug/2024:01:26:38] ENGINE Bus STARTED 2024-08-19T01:26:39.930 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: [19/Aug/2024:01:26:38] ENGINE Error in HTTPServer.serve 2024-08-19T01:26:39.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-08-19T01:26:39.930 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:26:39.931 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-08-19T01:26:39.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:26:39.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: mgrmap e8: x(active, since 2s) 2024-08-19T01:26:39.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:39.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:39 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:40.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-19T01:26:40.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.751+0000 7f792d4a3640 1 Processor -- start 2024-08-19T01:26:40.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.751+0000 7f792d4a3640 1 -- start start 2024-08-19T01:26:40.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.751+0000 7f792d4a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792810a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.751+0000 7f792d4a3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79280631e0 con 0x7f7928107d30 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.751+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792810a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.751+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792810a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37638/0 (socket says 172.21.15.103:37638) 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.751+0000 7f7926ffd640 1 -- 172.21.15.103:0/2802029191 learned_addr learned my addr 172.21.15.103:0/2802029191 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.752+0000 7f7926ffd640 1 -- 172.21.15.103:0/2802029191 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792810a660 con 0x7f7928107d30 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.752+0000 7f7926ffd640 1 --2- 172.21.15.103:0/2802029191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792810a120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7918009e80 tx=0x7f791802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df81c02a00f08db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f7925ffb640 1 -- 172.21.15.103:0/2802029191 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918036070 con 0x7f7928107d30 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f7925ffb640 1 -- 172.21.15.103:0/2802029191 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7918031890 con 0x7f7928107d30 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f7925ffb640 1 -- 172.21.15.103:0/2802029191 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918007980 con 0x7f7928107d30 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f792d4a3640 1 -- 172.21.15.103:0/2802029191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 msgr2=0x7f792810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f792d4a3640 1 --2- 172.21.15.103:0/2802029191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792810a120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7918009e80 tx=0x7f791802a7a0 comp rx=0 tx=0).stop 2024-08-19T01:26:40.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f792d4a3640 1 -- 172.21.15.103:0/2802029191 shutdown_connections 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f792d4a3640 1 --2- 172.21.15.103:0/2802029191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792810a120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f792d4a3640 1 -- 172.21.15.103:0/2802029191 >> 172.21.15.103:0/2802029191 conn(0x7f79281018f0 msgr2=0x7f7928103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f792d4a3640 1 -- 172.21.15.103:0/2802029191 shutdown_connections 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.753+0000 7f792d4a3640 1 -- 172.21.15.103:0/2802029191 wait complete. 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.754+0000 7f792d4a3640 1 Processor -- start 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.754+0000 7f792d4a3640 1 -- start start 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.754+0000 7f792d4a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.754+0000 7f792d4a3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f792810b640 con 0x7f7928107d30 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.754+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.754+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792819b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37642/0 (socket says 172.21.15.103:37642) 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.754+0000 7f7926ffd640 1 -- 172.21.15.103:0/2907230634 learned_addr learned my addr 172.21.15.103:0/2907230634 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.755+0000 7f7926ffd640 1 -- 172.21.15.103:0/2907230634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792819b6d0 con 0x7f7928107d30 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.755+0000 7f7926ffd640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792819b190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7918009680 tx=0x7f7918031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:40.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.755+0000 7f791ffff640 1 -- 172.21.15.103:0/2907230634 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918041020 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.755+0000 7f791ffff640 1 -- 172.21.15.103:0/2907230634 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f791803c900 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.755+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792819b960 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.755+0000 7f791ffff640 1 -- 172.21.15.103:0/2907230634 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918036040 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.755+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792819e650 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.756+0000 7f791ffff640 1 -- 172.21.15.103:0/2907230634 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7918044020 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.756+0000 7f791ffff640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7914038f70 0x7f791403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.756+0000 7f791ffff640 1 -- 172.21.15.103:0/2907230634 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f791806d510 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.756+0000 7f79267fc640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7914038f70 0x7f791403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.756+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7928108190 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.756+0000 7f79267fc640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7914038f70 0x7f791403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79100098c0 tx=0x7f7910006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.761+0000 7f791ffff640 1 -- 172.21.15.103:0/2907230634 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f791804b020 con 0x7f7928107d30 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.927+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7928102b30 con 0x7f7914038f70 2024-08-19T01:26:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.928+0000 7f791ffff640 1 -- 172.21.15.103:0/2907230634 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7928102b30 con 0x7f7914038f70 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7914038f70 msgr2=0x7f791403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7914038f70 0x7f791403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79100098c0 tx=0x7f7910006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 msgr2=0x7f792819b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792819b190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7918009680 tx=0x7f7918031c70 comp rx=0 tx=0).stop 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 shutdown_connections 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7914038f70 0x7f791403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 --2- 172.21.15.103:0/2907230634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928107d30 0x7f792819b190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 >> 172.21.15.103:0/2907230634 conn(0x7f79281018f0 msgr2=0x7f7928102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 shutdown_connections 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:39.930+0000 7f792d4a3640 1 -- 172.21.15.103:0/2907230634 wait complete. 2024-08-19T01:26:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:Generating ssh key... 2024-08-19T01:26:40.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:40 smithi103 ceph-mon[18436]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:41.769 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-mon[18436]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:41.769 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:41.769 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:41.769 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: Generating public/private rsa key pair. 2024-08-19T01:26:41.769 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: Your identification has been saved in /tmp/tmpmcj3f78g/key 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: Your public key has been saved in /tmp/tmpmcj3f78g/key.pub 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: The key fingerprint is: 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: SHA256:IsmryUwyNQ2d+FYKN+bX2zrMXwJa3pDQabj1b1JYOSI ceph-e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: The key's randomart image is: 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: +---[RSA 3072]----+ 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | | 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | o . o . . | 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | + * + E . + | 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | X = * + + . | 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | o X + S o . | 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | . o + = * o | 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: |o . . .oo = + | 2024-08-19T01:26:41.770 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | * o +. = | 2024-08-19T01:26:41.771 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: | = .o. | 2024-08-19T01:26:41.771 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:41 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: +----[SHA256]-----+ 2024-08-19T01:26:42.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.880+0000 7f41fcc4a640 1 Processor -- start 2024-08-19T01:26:42.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.880+0000 7f41fcc4a640 1 -- start start 2024-08-19T01:26:42.068 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.881+0000 7f41fcc4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f8107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.881+0000 7f41fcc4a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41f8107c10 con 0x7f41f8107240 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.881+0000 7f41f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f8107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.881+0000 7f41f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f8107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37656/0 (socket says 172.21.15.103:37656) 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.881+0000 7f41f659c640 1 -- 172.21.15.103:0/3508925541 learned_addr learned my addr 172.21.15.103:0/3508925541 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.881+0000 7f41f659c640 1 -- 172.21.15.103:0/3508925541 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f8108440 con 0x7f41f8107240 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.882+0000 7f41f659c640 1 --2- 172.21.15.103:0/3508925541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f8107640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009820 tx=0x7f41e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69709231955b4b35 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.882+0000 7f41f559a640 1 -- 172.21.15.103:0/3508925541 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41e8037070 con 0x7f41f8107240 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.882+0000 7f41f559a640 1 -- 172.21.15.103:0/3508925541 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f41e80328d0 con 0x7f41f8107240 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.882+0000 7f41f559a640 1 -- 172.21.15.103:0/3508925541 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41e80307d0 con 0x7f41f8107240 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.882+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3508925541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 msgr2=0x7f41f8107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.882+0000 7f41fcc4a640 1 --2- 172.21.15.103:0/3508925541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f8107640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009820 tx=0x7f41e802a270 comp rx=0 tx=0).stop 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3508925541 shutdown_connections 2024-08-19T01:26:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 --2- 172.21.15.103:0/3508925541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f8107640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3508925541 >> 172.21.15.103:0/3508925541 conn(0x7f41f8102a10 msgr2=0x7f41f8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3508925541 shutdown_connections 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3508925541 wait complete. 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 Processor -- start 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 -- start start 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.883+0000 7f41fcc4a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810b7f0 con 0x7f41f8107240 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:42.070 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41f659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f819b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37664/0 (socket says 172.21.15.103:37664) 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41f659c640 1 -- 172.21.15.103:0/3648296219 learned_addr learned my addr 172.21.15.103:0/3648296219 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41f659c640 1 -- 172.21.15.103:0/3648296219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f819b720 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41f659c640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f819b1e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009940 tx=0x7f41e8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41e77fe640 1 -- 172.21.15.103:0/3648296219 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41e8042070 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41e77fe640 1 -- 172.21.15.103:0/3648296219 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f41e8030c80 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f819b9b0 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41e77fe640 1 -- 172.21.15.103:0/3648296219 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41e8037040 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.884+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f819e6a0 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.885+0000 7f41e77fe640 1 -- 172.21.15.103:0/3648296219 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f41e8045070 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.885+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f810b970 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.885+0000 7f41e77fe640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f41e0038fc0 0x7f41e003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.885+0000 7f41e77fe640 1 -- 172.21.15.103:0/3648296219 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f41e806be60 con 0x7f41f8107240 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.885+0000 7f41f5d9b640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f41e0038fc0 0x7f41e003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:42.071 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.886+0000 7f41f5d9b640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f41e0038fc0 0x7f41e003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098c0 tx=0x7f41ec006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:40.890+0000 7f41e77fe640 1 -- 172.21.15.103:0/3648296219 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f41e8043260 con 0x7f41f8107240 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.056+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f41f8103bf0 con 0x7f41e0038fc0 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.473+0000 7f41e77fe640 1 -- 172.21.15.103:0/3648296219 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41f8103bf0 con 0x7f41e0038fc0 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f41e0038fc0 msgr2=0x7f41e003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f41e0038fc0 0x7f41e003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098c0 tx=0x7f41ec006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 msgr2=0x7f41f819b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f819b1e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009940 tx=0x7f41e8032e10 comp rx=0 tx=0).stop 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 shutdown_connections 2024-08-19T01:26:42.072 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f41e0038fc0 0x7f41e003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:42.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 --2- 172.21.15.103:0/3648296219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f8107240 0x7f41f819b1e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:42.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 >> 172.21.15.103:0/3648296219 conn(0x7f41f8102a10 msgr2=0x7f41f81034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:42.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 shutdown_connections 2024-08-19T01:26:42.073 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:41.475+0000 7f41fcc4a640 1 -- 172.21.15.103:0/3648296219 wait complete. 2024-08-19T01:26:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:42 smithi103 ceph-mon[18436]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:42 smithi103 ceph-mon[18436]: Generating ssh key... 2024-08-19T01:26:43.268 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEZsTwVhFODJwXVyb5nJ3wHPtoGyv4nGrahenkuKazNWGdVWOBQhKaq478T35fgS0mz9OXZAMKpDChB8te3Xzh3fb4wqeWqFmXDunTdAb7UoK39teFSyD1lA/HpYFaPR2aG8NdqrH6MmeqOPdoOtYOsebH4mJnObfJrRa96AXDmFsADvZnYy7xscVe4i5L7UJ3sIfXDZ3qmN3OP++SLHoMAPfR0qibvLPvOJVd8XWPWyz+hY2oJjW3IIaS5ALCqE/mCCVqw1veMXdtJ6UZacA7n7zlPm7vFkeyFuAcnXzfOMEqGD2XMu6oCfborLjphibBMqfCYKQlkSxylLLGBOI+zN/ETakyhRk3q0GyMaBkjb1kFqJI1EvNG2+rTQj7IkZ+Nx0nJr5vhWrbh3Wmek14JyzYbYo3BCTXJ6saLWpy5T+EwUx2zEAYyNR3S1KkkQQrKvf5R8PCyPNFium6B36tdVUIOHjY2M9+hfhZavz8pajG5XC9BPUaQ0mBXm/rsBM= ceph-e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.464+0000 7f739acc8640 1 Processor -- start 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.464+0000 7f739acc8640 1 -- start start 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.464+0000 7f739acc8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f739410a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.464+0000 7f739acc8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7394063200 con 0x7f7394107d10 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.465+0000 7f7398a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f739410a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.465+0000 7f7398a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f739410a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44758/0 (socket says 172.21.15.103:44758) 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.465+0000 7f7398a64640 1 -- 172.21.15.103:0/4287713327 learned_addr learned my addr 172.21.15.103:0/4287713327 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.465+0000 7f7398a64640 1 -- 172.21.15.103:0/4287713327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739410a640 con 0x7f7394107d10 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f7398a64640 1 --2- 172.21.15.103:0/4287713327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f739410a100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7388009820 tx=0x7f738802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3829dafda72a87b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f73937fe640 1 -- 172.21.15.103:0/4287713327 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388037070 con 0x7f7394107d10 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f73937fe640 1 -- 172.21.15.103:0/4287713327 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f73880328d0 con 0x7f7394107d10 2024-08-19T01:26:43.269 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f73937fe640 1 -- 172.21.15.103:0/4287713327 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73880307d0 con 0x7f7394107d10 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f739acc8640 1 -- 172.21.15.103:0/4287713327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 msgr2=0x7f739410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f739acc8640 1 --2- 172.21.15.103:0/4287713327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f739410a100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7388009820 tx=0x7f738802a270 comp rx=0 tx=0).stop 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f739acc8640 1 -- 172.21.15.103:0/4287713327 shutdown_connections 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f739acc8640 1 --2- 172.21.15.103:0/4287713327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f739410a100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f739acc8640 1 -- 172.21.15.103:0/4287713327 >> 172.21.15.103:0/4287713327 conn(0x7f73941018f0 msgr2=0x7f7394103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f739acc8640 1 -- 172.21.15.103:0/4287713327 shutdown_connections 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.466+0000 7f739acc8640 1 -- 172.21.15.103:0/4287713327 wait complete. 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.467+0000 7f739acc8640 1 Processor -- start 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.467+0000 7f739acc8640 1 -- start start 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.467+0000 7f739acc8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f7394107590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.467+0000 7f739acc8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f739410b620 con 0x7f7394107d10 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.467+0000 7f7398a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f7394107590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:43.270 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.467+0000 7f7398a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f7394107590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44766/0 (socket says 172.21.15.103:44766) 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.467+0000 7f7398a64640 1 -- 172.21.15.103:0/734261306 learned_addr learned my addr 172.21.15.103:0/734261306 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.468+0000 7f7398a64640 1 -- 172.21.15.103:0/734261306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7394107ad0 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.468+0000 7f7398a64640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f7394107590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7388032e10 tx=0x7f7388032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.468+0000 7f7391ffb640 1 -- 172.21.15.103:0/734261306 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388042070 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.468+0000 7f7391ffb640 1 -- 172.21.15.103:0/734261306 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7388030c80 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.468+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7394105cc0 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.468+0000 7f7391ffb640 1 -- 172.21.15.103:0/734261306 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7388037040 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.468+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7394106200 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.469+0000 7f7391ffb640 1 -- 172.21.15.103:0/734261306 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7388045070 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.469+0000 7f7391ffb640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7384038fc0 0x7f738403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.469+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7394108170 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.469+0000 7f7391ffb640 1 -- 172.21.15.103:0/734261306 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f738806be90 con 0x7f7394107d10 2024-08-19T01:26:43.271 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.469+0000 7f7393fff640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7384038fc0 0x7f738403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.470+0000 7f7393fff640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7384038fc0 0x7f738403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73800098c0 tx=0x7f7380006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.474+0000 7f7391ffb640 1 -- 172.21.15.103:0/734261306 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7388045360 con 0x7f7394107d10 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.644+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7394106b50 con 0x7f7384038fc0 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.644+0000 7f7391ffb640 1 -- 172.21.15.103:0/734261306 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7394106b50 con 0x7f7384038fc0 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7384038fc0 msgr2=0x7f738403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7384038fc0 0x7f738403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73800098c0 tx=0x7f7380006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 msgr2=0x7f7394107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f7394107590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7388032e10 tx=0x7f7388032e40 comp rx=0 tx=0).stop 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 shutdown_connections 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f7384038fc0 0x7f738403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 --2- 172.21.15.103:0/734261306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394107d10 0x7f7394107590 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 >> 172.21.15.103:0/734261306 conn(0x7f73941018f0 msgr2=0x7f7394102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 shutdown_connections 2024-08-19T01:26:43.272 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:42.647+0000 7f739acc8640 1 -- 172.21.15.103:0/734261306 wait complete. 2024-08-19T01:26:43.273 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:26:43.273 INFO:teuthology.orchestra.run.smithi103.stdout:Adding key to root@localhost authorized_keys... 2024-08-19T01:26:43.273 INFO:teuthology.orchestra.run.smithi103.stdout:Adding host smithi103... 2024-08-19T01:26:43.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:43 smithi103 ceph-mon[18436]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:44.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:44 smithi103 ceph-mon[18436]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:46.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:45 smithi103 ceph-mon[18436]: Deploying cephadm binary to smithi103 2024-08-19T01:26:47.733 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Added host 'smithi103' with addr '172.21.15.103' 2024-08-19T01:26:47.733 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.684+0000 7f7908c6b640 1 Processor -- start 2024-08-19T01:26:47.733 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.684+0000 7f7908c6b640 1 -- start start 2024-08-19T01:26:47.733 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.684+0000 7f7908c6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790410a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:47.733 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.684+0000 7f7908c6b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f790407bdd0 con 0x7f7904107d30 2024-08-19T01:26:47.733 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.685+0000 7f790259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790410a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:47.733 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.685+0000 7f790259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790410a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44774/0 (socket says 172.21.15.103:44774) 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.685+0000 7f790259c640 1 -- 172.21.15.103:0/4040973051 learned_addr learned my addr 172.21.15.103:0/4040973051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.685+0000 7f790259c640 1 -- 172.21.15.103:0/4040973051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790410a660 con 0x7f7904107d30 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f790259c640 1 --2- 172.21.15.103:0/4040973051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790410a120 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009820 tx=0x7f78f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111d34eee487ed4b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7901d9b640 1 -- 172.21.15.103:0/4040973051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78f4037070 con 0x7f7904107d30 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7901d9b640 1 -- 172.21.15.103:0/4040973051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f78f40328d0 con 0x7f7904107d30 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7901d9b640 1 -- 172.21.15.103:0/4040973051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78f40307d0 con 0x7f7904107d30 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7908c6b640 1 -- 172.21.15.103:0/4040973051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 msgr2=0x7f790410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7908c6b640 1 --2- 172.21.15.103:0/4040973051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790410a120 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009820 tx=0x7f78f402a270 comp rx=0 tx=0).stop 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7908c6b640 1 -- 172.21.15.103:0/4040973051 shutdown_connections 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7908c6b640 1 --2- 172.21.15.103:0/4040973051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790410a120 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:47.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7908c6b640 1 -- 172.21.15.103:0/4040973051 >> 172.21.15.103:0/4040973051 conn(0x7f7904101930 msgr2=0x7f7904103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7908c6b640 1 -- 172.21.15.103:0/4040973051 shutdown_connections 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.686+0000 7f7908c6b640 1 -- 172.21.15.103:0/4040973051 wait complete. 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.687+0000 7f7908c6b640 1 Processor -- start 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.687+0000 7f7908c6b640 1 -- start start 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.687+0000 7f7908c6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.687+0000 7f7908c6b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f790410b640 con 0x7f7904107d30 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.687+0000 7f790259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.687+0000 7f790259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790419b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44780/0 (socket says 172.21.15.103:44780) 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.687+0000 7f790259c640 1 -- 172.21.15.103:0/2346375600 learned_addr learned my addr 172.21.15.103:0/2346375600 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.688+0000 7f790259c640 1 -- 172.21.15.103:0/2346375600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790419b6e0 con 0x7f7904107d30 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.688+0000 7f790259c640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790419b1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78f40097f0 tx=0x7f78f4030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.688+0000 7f78fb7fe640 1 -- 172.21.15.103:0/2346375600 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78f4037040 con 0x7f7904107d30 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.688+0000 7f78fb7fe640 1 -- 172.21.15.103:0/2346375600 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f78f4032bb0 con 0x7f7904107d30 2024-08-19T01:26:47.735 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.688+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790419b970 con 0x7f7904107d30 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.688+0000 7f78fb7fe640 1 -- 172.21.15.103:0/2346375600 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78f403a390 con 0x7f7904107d30 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.688+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790419e660 con 0x7f7904107d30 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.689+0000 7f78fb7fe640 1 -- 172.21.15.103:0/2346375600 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f78f403a580 con 0x7f7904107d30 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.689+0000 7f78fb7fe640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f78f0038fc0 0x7f78f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.689+0000 7f78fb7fe640 1 -- 172.21.15.103:0/2346375600 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f78f406c250 con 0x7f7904107d30 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.689+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7904108190 con 0x7f7904107d30 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.689+0000 7f78fbfff640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f78f0038fc0 0x7f78f003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.690+0000 7f78fbfff640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f78f0038fc0 0x7f78f003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0098c0 tx=0x7f78ec006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.694+0000 7f78fb7fe640 1 -- 172.21.15.103:0/2346375600 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f78f4040210 con 0x7f7904107d30 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:43.863+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}) -- 0x7f7904102bf0 con 0x7f78f0038fc0 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.085+0000 7f78fb7fe640 1 -- 172.21.15.103:0/2346375600 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7904102bf0 con 0x7f78f0038fc0 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f78f0038fc0 msgr2=0x7f78f003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:47.736 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f78f0038fc0 0x7f78f003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0098c0 tx=0x7f78ec006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 msgr2=0x7f790419b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790419b1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78f40097f0 tx=0x7f78f4030d00 comp rx=0 tx=0).stop 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 shutdown_connections 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f78f0038fc0 0x7f78f003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 --2- 172.21.15.103:0/2346375600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7904107d30 0x7f790419b1a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.088+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 >> 172.21.15.103:0/2346375600 conn(0x7f7904101930 msgr2=0x7f79041024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.089+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 shutdown_connections 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:47.089+0000 7f7908c6b640 1 -- 172.21.15.103:0/2346375600 wait complete. 2024-08-19T01:26:47.737 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mon service... 2024-08-19T01:26:48.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:48 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:48.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:48 smithi103 ceph-mon[18436]: Added host smithi103 2024-08-19T01:26:48.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:48 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:26:49.153 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.155+0000 7f178ed68640 1 Processor -- start 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.155+0000 7f178ed68640 1 -- start start 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.155+0000 7f178ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f1788107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.155+0000 7f178ed68640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1788107c10 con 0x7f1788107240 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.156+0000 7f178cb04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f1788107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.156+0000 7f178cb04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f1788107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44796/0 (socket says 172.21.15.103:44796) 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.156+0000 7f178cb04640 1 -- 172.21.15.103:0/3939810292 learned_addr learned my addr 172.21.15.103:0/3939810292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.156+0000 7f178cb04640 1 -- 172.21.15.103:0/3939810292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1788108440 con 0x7f1788107240 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.156+0000 7f178cb04640 1 --2- 172.21.15.103:0/3939810292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f1788107640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f177c009820 tx=0x7f177c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1028b8da825d90e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f17877fe640 1 -- 172.21.15.103:0/3939810292 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c037070 con 0x7f1788107240 2024-08-19T01:26:49.154 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f17877fe640 1 -- 172.21.15.103:0/3939810292 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f177c0328d0 con 0x7f1788107240 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f17877fe640 1 -- 172.21.15.103:0/3939810292 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c0307d0 con 0x7f1788107240 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f178ed68640 1 -- 172.21.15.103:0/3939810292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 msgr2=0x7f1788107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f178ed68640 1 --2- 172.21.15.103:0/3939810292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f1788107640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f177c009820 tx=0x7f177c02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f178ed68640 1 -- 172.21.15.103:0/3939810292 shutdown_connections 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f178ed68640 1 --2- 172.21.15.103:0/3939810292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f1788107640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f178ed68640 1 -- 172.21.15.103:0/3939810292 >> 172.21.15.103:0/3939810292 conn(0x7f1788102a10 msgr2=0x7f1788104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f178ed68640 1 -- 172.21.15.103:0/3939810292 shutdown_connections 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.157+0000 7f178ed68640 1 -- 172.21.15.103:0/3939810292 wait complete. 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.158+0000 7f178ed68640 1 Processor -- start 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.158+0000 7f178ed68640 1 -- start start 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.158+0000 7f178ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f178819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.158+0000 7f178ed68640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f178810b7f0 con 0x7f1788107240 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.158+0000 7f178cb04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f178819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:49.155 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.158+0000 7f178cb04640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f178819b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44800/0 (socket says 172.21.15.103:44800) 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.158+0000 7f178cb04640 1 -- 172.21.15.103:0/1444388802 learned_addr learned my addr 172.21.15.103:0/1444388802 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.159+0000 7f178cb04640 1 -- 172.21.15.103:0/1444388802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178819b720 con 0x7f1788107240 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.159+0000 7f178cb04640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f178819b1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f177c009940 tx=0x7f177c032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.159+0000 7f1785ffb640 1 -- 172.21.15.103:0/1444388802 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c042070 con 0x7f1788107240 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.159+0000 7f1785ffb640 1 -- 172.21.15.103:0/1444388802 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f177c030c80 con 0x7f1788107240 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.159+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178819b9b0 con 0x7f1788107240 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.159+0000 7f1785ffb640 1 -- 172.21.15.103:0/1444388802 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c037040 con 0x7f1788107240 2024-08-19T01:26:49.156 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.159+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178819e6a0 con 0x7f1788107240 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.160+0000 7f1785ffb640 1 -- 172.21.15.103:0/1444388802 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f177c045070 con 0x7f1788107240 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.160+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178810b970 con 0x7f1788107240 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.160+0000 7f1785ffb640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f1778038fc0 0x7f177803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.160+0000 7f1785ffb640 1 -- 172.21.15.103:0/1444388802 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f177c06bfe0 con 0x7f1788107240 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.161+0000 7f1787fff640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f1778038fc0 0x7f177803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.161+0000 7f1787fff640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f1778038fc0 0x7f177803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f17740098c0 tx=0x7f1774006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.165+0000 7f1785ffb640 1 -- 172.21.15.103:0/1444388802 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f177c045360 con 0x7f1788107240 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.357+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1788103c30 con 0x7f1778038fc0 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.362+0000 7f1785ffb640 1 -- 172.21.15.103:0/1444388802 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1788103c30 con 0x7f1778038fc0 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f1778038fc0 msgr2=0x7f177803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f1778038fc0 0x7f177803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f17740098c0 tx=0x7f1774006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 msgr2=0x7f178819b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f178819b1e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f177c009940 tx=0x7f177c032e10 comp rx=0 tx=0).stop 2024-08-19T01:26:49.157 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 shutdown_connections 2024-08-19T01:26:49.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f1778038fc0 0x7f177803b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:49.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 --2- 172.21.15.103:0/1444388802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1788107240 0x7f178819b1e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:49.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 >> 172.21.15.103:0/1444388802 conn(0x7f1788102a10 msgr2=0x7f1788103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:49.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 shutdown_connections 2024-08-19T01:26:49.158 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:48.364+0000 7f178ed68640 1 -- 172.21.15.103:0/1444388802 wait complete. 2024-08-19T01:26:49.158 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mgr service... 2024-08-19T01:26:49.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:49 smithi103 ceph-mon[18436]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:49.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:49 smithi103 ceph-mon[18436]: Saving service mon spec with placement count:5 2024-08-19T01:26:49.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:49 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:50.453 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-19T01:26:50.453 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.552+0000 7f5c49869640 1 Processor -- start 2024-08-19T01:26:50.453 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.552+0000 7f5c49869640 1 -- start start 2024-08-19T01:26:50.453 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.552+0000 7f5c49869640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c44107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:50.453 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.552+0000 7f5c49869640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c44107c10 con 0x7f5c44107240 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.552+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c44107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.552+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c44107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44802/0 (socket says 172.21.15.103:44802) 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.552+0000 7f5c42ffd640 1 -- 172.21.15.103:0/3097778529 learned_addr learned my addr 172.21.15.103:0/3097778529 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.553+0000 7f5c42ffd640 1 -- 172.21.15.103:0/3097778529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c44108440 con 0x7f5c44107240 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.553+0000 7f5c42ffd640 1 --2- 172.21.15.103:0/3097778529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c44107640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009e80 tx=0x7f5c3402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27810b067429d4f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.553+0000 7f5c41ffb640 1 -- 172.21.15.103:0/3097778529 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c34036070 con 0x7f5c44107240 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c41ffb640 1 -- 172.21.15.103:0/3097778529 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5c34031890 con 0x7f5c44107240 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c41ffb640 1 -- 172.21.15.103:0/3097778529 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c34007980 con 0x7f5c44107240 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c49869640 1 -- 172.21.15.103:0/3097778529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 msgr2=0x7f5c44107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c49869640 1 --2- 172.21.15.103:0/3097778529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c44107640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009e80 tx=0x7f5c3402a7a0 comp rx=0 tx=0).stop 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c49869640 1 -- 172.21.15.103:0/3097778529 shutdown_connections 2024-08-19T01:26:50.454 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c49869640 1 --2- 172.21.15.103:0/3097778529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c44107640 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c49869640 1 -- 172.21.15.103:0/3097778529 >> 172.21.15.103:0/3097778529 conn(0x7f5c44102a10 msgr2=0x7f5c44104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c49869640 1 -- 172.21.15.103:0/3097778529 shutdown_connections 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.554+0000 7f5c49869640 1 -- 172.21.15.103:0/3097778529 wait complete. 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c49869640 1 Processor -- start 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c49869640 1 -- start start 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c49869640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c4419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c49869640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4410b7f0 con 0x7f5c44107240 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c4419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c4419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44808/0 (socket says 172.21.15.103:44808) 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c42ffd640 1 -- 172.21.15.103:0/3159978803 learned_addr learned my addr 172.21.15.103:0/3159978803 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c42ffd640 1 -- 172.21.15.103:0/3159978803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4419b720 con 0x7f5c44107240 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.555+0000 7f5c42ffd640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c4419b1e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c34007bb0 tx=0x7f5c34031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:50.455 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.556+0000 7f5c48867640 1 -- 172.21.15.103:0/3159978803 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c34041020 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.556+0000 7f5c48867640 1 -- 172.21.15.103:0/3159978803 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5c3403c900 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.556+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4419b9b0 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.556+0000 7f5c48867640 1 -- 172.21.15.103:0/3159978803 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c34036040 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.556+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4419e6a0 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.556+0000 7f5c48867640 1 -- 172.21.15.103:0/3159978803 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5c34044020 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.557+0000 7f5c48867640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f5c2c038f70 0x7f5c2c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.557+0000 7f5c48867640 1 -- 172.21.15.103:0/3159978803 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c3406d460 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.557+0000 7f5c427fc640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f5c2c038f70 0x7f5c2c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.557+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4410b970 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.557+0000 7f5c427fc640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f5c2c038f70 0x7f5c2c03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c380098c0 tx=0x7f5c38006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.561+0000 7f5c48867640 1 -- 172.21.15.103:0/3159978803 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5c3404b020 con 0x7f5c44107240 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.750+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5c44103c30 con 0x7f5c2c038f70 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.754+0000 7f5c48867640 1 -- 172.21.15.103:0/3159978803 <== mgr.14116 v2:172.21.15.103:6800/2881014674 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5c44103c30 con 0x7f5c2c038f70 2024-08-19T01:26:50.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.756+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f5c2c038f70 msgr2=0x7f5c2c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.756+0000 7f5c49869640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f5c2c038f70 0x7f5c2c03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c380098c0 tx=0x7f5c38006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.756+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 msgr2=0x7f5c4419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.756+0000 7f5c49869640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c4419b1e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c34007bb0 tx=0x7f5c34031c70 comp rx=0 tx=0).stop 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.757+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 shutdown_connections 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.757+0000 7f5c49869640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f5c2c038f70 0x7f5c2c03b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.757+0000 7f5c49869640 1 --2- 172.21.15.103:0/3159978803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c44107240 0x7f5c4419b1e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.757+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 >> 172.21.15.103:0/3159978803 conn(0x7f5c44102a10 msgr2=0x7f5c44103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.757+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 shutdown_connections 2024-08-19T01:26:50.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:49.757+0000 7f5c49869640 1 -- 172.21.15.103:0/3159978803 wait complete. 2024-08-19T01:26:51.024 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:50 smithi103 ceph-mon[18436]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:26:51.024 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:50 smithi103 ceph-mon[18436]: Saving service mgr spec with placement count:2 2024-08-19T01:26:51.024 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:50 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:52.294 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:52 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3983764317' entity='client.admin' 2024-08-19T01:26:52.295 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:52 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.859+0000 7f03f1c96640 1 Processor -- start 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.859+0000 7f03f1c96640 1 -- start start 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.859+0000 7f03f1c96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec106560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.859+0000 7f03f1c96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec106b30 con 0x7f03ec106160 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.859+0000 7f03eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.859+0000 7f03eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44822/0 (socket says 172.21.15.103:44822) 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.859+0000 7f03eb7fe640 1 -- 172.21.15.103:0/3938351110 learned_addr learned my addr 172.21.15.103:0/3938351110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:52.302 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.860+0000 7f03eb7fe640 1 -- 172.21.15.103:0/3938351110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec107360 con 0x7f03ec106160 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.860+0000 7f03eb7fe640 1 --2- 172.21.15.103:0/3938351110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec106560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e80 tx=0x7f03dc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775d4befbed5860f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.860+0000 7f03ea7fc640 1 -- 172.21.15.103:0/3938351110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03dc036070 con 0x7f03ec106160 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.860+0000 7f03ea7fc640 1 -- 172.21.15.103:0/3938351110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f03dc031890 con 0x7f03ec106160 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.860+0000 7f03ea7fc640 1 -- 172.21.15.103:0/3938351110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03dc007980 con 0x7f03ec106160 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.861+0000 7f03f1c96640 1 -- 172.21.15.103:0/3938351110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 msgr2=0x7f03ec106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.861+0000 7f03f1c96640 1 --2- 172.21.15.103:0/3938351110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec106560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009e80 tx=0x7f03dc02a7a0 comp rx=0 tx=0).stop 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.861+0000 7f03f1c96640 1 -- 172.21.15.103:0/3938351110 shutdown_connections 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.861+0000 7f03f1c96640 1 --2- 172.21.15.103:0/3938351110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec106560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.861+0000 7f03f1c96640 1 -- 172.21.15.103:0/3938351110 >> 172.21.15.103:0/3938351110 conn(0x7f03ec101930 msgr2=0x7f03ec103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.861+0000 7f03f1c96640 1 -- 172.21.15.103:0/3938351110 shutdown_connections 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.861+0000 7f03f1c96640 1 -- 172.21.15.103:0/3938351110 wait complete. 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03f1c96640 1 Processor -- start 2024-08-19T01:26:52.303 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03f1c96640 1 -- start start 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03f1c96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec07b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03f1c96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec10a710 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec07b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec07b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44830/0 (socket says 172.21.15.103:44830) 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03eb7fe640 1 -- 172.21.15.103:0/3983764317 learned_addr learned my addr 172.21.15.103:0/3983764317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03eb7fe640 1 -- 172.21.15.103:0/3983764317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec19df20 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.862+0000 7f03eb7fe640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec07b9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03dc007bb0 tx=0x7f03dc031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.863+0000 7f03e8ff9640 1 -- 172.21.15.103:0/3983764317 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03dc041020 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.863+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec19e0f0 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.863+0000 7f03e8ff9640 1 -- 172.21.15.103:0/3983764317 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f03dc03c900 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.863+0000 7f03e8ff9640 1 -- 172.21.15.103:0/3983764317 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03dc036040 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.863+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec19e510 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.864+0000 7f03e8ff9640 1 -- 172.21.15.103:0/3983764317 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f03dc044020 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.864+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ec10a890 con 0x7f03ec106160 2024-08-19T01:26:52.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.864+0000 7f03e8ff9640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f03d4038f70 0x7f03d403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.864+0000 7f03e8ff9640 1 -- 172.21.15.103:0/3983764317 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f03dc06d590 con 0x7f03ec106160 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.864+0000 7f03eaffd640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f03d4038f70 0x7f03d403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.865+0000 7f03eaffd640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f03d4038f70 0x7f03d403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03e00098c0 tx=0x7f03e0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:50.869+0000 7f03e8ff9640 1 -- 172.21.15.103:0/3983764317 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f03dc04b020 con 0x7f03ec106160 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.036+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f03ec19ed30 con 0x7f03ec106160 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.039+0000 7f03e8ff9640 1 -- 172.21.15.103:0/3983764317 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f03dc045c40 con 0x7f03ec106160 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f03d4038f70 msgr2=0x7f03d403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f03d4038f70 0x7f03d403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03e00098c0 tx=0x7f03e0006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 msgr2=0x7f03ec07b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec07b9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03dc007bb0 tx=0x7f03dc031c70 comp rx=0 tx=0).stop 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 shutdown_connections 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f03d4038f70 0x7f03d403b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:52.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 --2- 172.21.15.103:0/3983764317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03ec106160 0x7f03ec07b9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:52.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 >> 172.21.15.103:0/3983764317 conn(0x7f03ec101930 msgr2=0x7f03ec102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:52.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 shutdown_connections 2024-08-19T01:26:52.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:51.041+0000 7f03f1c96640 1 -- 172.21.15.103:0/3983764317 wait complete. 2024-08-19T01:26:53.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:53 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:53.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:53 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3706642839' entity='client.admin' 2024-08-19T01:26:54.171 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.759+0000 7f4341b97640 1 Processor -- start 2024-08-19T01:26:54.171 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.759+0000 7f4341b97640 1 -- start start 2024-08-19T01:26:54.171 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.759+0000 7f4341b97640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c07c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:54.171 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.759+0000 7f4341b97640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f433c07c830 con 0x7f433c07de00 2024-08-19T01:26:54.171 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.759+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c07c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.759+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c07c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33592/0 (socket says 172.21.15.103:33592) 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.759+0000 7f433b7fe640 1 -- 172.21.15.103:0/2198819547 learned_addr learned my addr 172.21.15.103:0/2198819547 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.760+0000 7f433b7fe640 1 -- 172.21.15.103:0/2198819547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c07c9b0 con 0x7f433c07de00 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.760+0000 7f433b7fe640 1 --2- 172.21.15.103:0/2198819547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c07c260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f432c009820 tx=0x7f432c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7f2fb59e2efdfa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.760+0000 7f433a7fc640 1 -- 172.21.15.103:0/2198819547 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f432c037070 con 0x7f433c07de00 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f433a7fc640 1 -- 172.21.15.103:0/2198819547 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f432c0328f0 con 0x7f433c07de00 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f433a7fc640 1 -- 172.21.15.103:0/2198819547 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f432c030740 con 0x7f433c07de00 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f4341b97640 1 -- 172.21.15.103:0/2198819547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 msgr2=0x7f433c07c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f4341b97640 1 --2- 172.21.15.103:0/2198819547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c07c260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f432c009820 tx=0x7f432c02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f4341b97640 1 -- 172.21.15.103:0/2198819547 shutdown_connections 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f4341b97640 1 --2- 172.21.15.103:0/2198819547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c07c260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f4341b97640 1 -- 172.21.15.103:0/2198819547 >> 172.21.15.103:0/2198819547 conn(0x7f433c102a10 msgr2=0x7f433c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f4341b97640 1 -- 172.21.15.103:0/2198819547 shutdown_connections 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.761+0000 7f4341b97640 1 -- 172.21.15.103:0/2198819547 wait complete. 2024-08-19T01:26:54.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.762+0000 7f4341b97640 1 Processor -- start 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.762+0000 7f4341b97640 1 -- start start 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.762+0000 7f4341b97640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.762+0000 7f4341b97640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f433c10d870 con 0x7f433c07de00 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.762+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.762+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33600/0 (socket says 172.21.15.103:33600) 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.762+0000 7f433b7fe640 1 -- 172.21.15.103:0/3706642839 learned_addr learned my addr 172.21.15.103:0/3706642839 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.763+0000 7f433b7fe640 1 -- 172.21.15.103:0/3706642839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c19d8f0 con 0x7f433c07de00 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.763+0000 7f433b7fe640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c19d3b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f432c030cc0 tx=0x7f432c030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.763+0000 7f4338ff9640 1 -- 172.21.15.103:0/3706642839 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f432c042070 con 0x7f433c07de00 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.763+0000 7f4338ff9640 1 -- 172.21.15.103:0/3706642839 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f432c030e60 con 0x7f433c07de00 2024-08-19T01:26:54.173 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.763+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c19db80 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.763+0000 7f4338ff9640 1 -- 172.21.15.103:0/3706642839 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f432c037040 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.763+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c1a0870 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.764+0000 7f4338ff9640 1 -- 172.21.15.103:0/3706642839 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f432c045070 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.764+0000 7f4338ff9640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f4324038fc0 0x7f432403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.764+0000 7f4338ff9640 1 -- 172.21.15.103:0/3706642839 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f432c06cf30 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.764+0000 7f433affd640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f4324038fc0 0x7f432403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.764+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c07c2e0 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.765+0000 7f433affd640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f4324038fc0 0x7f432403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f433000ad30 tx=0x7f43300093f0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.769+0000 7f4338ff9640 1 -- 172.21.15.103:0/3706642839 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f432c046e00 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.955+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f433c1a0da0 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.957+0000 7f4338ff9640 1 -- 172.21.15.103:0/3706642839 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f432c045360 con 0x7f433c07de00 2024-08-19T01:26:54.174 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.959+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f4324038fc0 msgr2=0x7f432403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.959+0000 7f4341b97640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f4324038fc0 0x7f432403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f433000ad30 tx=0x7f43300093f0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.959+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 msgr2=0x7f433c19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.959+0000 7f4341b97640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c19d3b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f432c030cc0 tx=0x7f432c030cf0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.960+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 shutdown_connections 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.960+0000 7f4341b97640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f4324038fc0 0x7f432403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.960+0000 7f4341b97640 1 --2- 172.21.15.103:0/3706642839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f433c07de00 0x7f433c19d3b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.960+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 >> 172.21.15.103:0/3706642839 conn(0x7f433c102a10 msgr2=0x7f433c103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.960+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 shutdown_connections 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:52.960+0000 7f4341b97640 1 -- 172.21.15.103:0/3706642839 wait complete. 2024-08-19T01:26:54.175 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling the dashboard module... 2024-08-19T01:26:54.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:54 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:56.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:55 smithi103 ceph-mon[18436]: from='mgr.14116 172.21.15.103:0/4021805' entity='mgr.x' 2024-08-19T01:26:56.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:55 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3615544744' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-08-19T01:26:56.055 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:55 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: ignoring --setuser ceph since I am not root 2024-08-19T01:26:56.055 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:55 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: ignoring --setgroup ceph since I am not root 2024-08-19T01:26:56.055 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.019+0000 7f4ff87fa040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:26:56.392 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.087+0000 7f4ff87fa040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:26:56.392 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.232+0000 7f4ff87fa040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:26:56.393 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.390+0000 7f4ff87fa040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:26:56.666 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.539+0000 7f4ff87fa040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:26:56.853 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.588+0000 7fa0618df640 1 Processor -- start 2024-08-19T01:26:56.854 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.588+0000 7fa0618df640 1 -- start start 2024-08-19T01:26:56.854 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.588+0000 7fa0618df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c10a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:56.854 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.588+0000 7fa0618df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c07bdb0 con 0x7fa05c107d50 2024-08-19T01:26:56.854 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.589+0000 7fa05affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c10a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:56.854 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.589+0000 7fa05affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c10a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33612/0 (socket says 172.21.15.103:33612) 2024-08-19T01:26:56.854 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.589+0000 7fa05affd640 1 -- 172.21.15.103:0/3094903579 learned_addr learned my addr 172.21.15.103:0/3094903579 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:56.854 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.589+0000 7fa05affd640 1 -- 172.21.15.103:0/3094903579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c10a680 con 0x7fa05c107d50 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.590+0000 7fa05affd640 1 --2- 172.21.15.103:0/3094903579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c10a140 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009820 tx=0x7fa04c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53540e1a19aecbf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.590+0000 7fa059ffb640 1 -- 172.21.15.103:0/3094903579 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa04c037070 con 0x7fa05c107d50 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.590+0000 7fa059ffb640 1 -- 172.21.15.103:0/3094903579 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04c0328f0 con 0x7fa05c107d50 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.590+0000 7fa059ffb640 1 -- 172.21.15.103:0/3094903579 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa04c030740 con 0x7fa05c107d50 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 -- 172.21.15.103:0/3094903579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 msgr2=0x7fa05c10a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 --2- 172.21.15.103:0/3094903579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c10a140 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009820 tx=0x7fa04c02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 -- 172.21.15.103:0/3094903579 shutdown_connections 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 --2- 172.21.15.103:0/3094903579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c10a140 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 -- 172.21.15.103:0/3094903579 >> 172.21.15.103:0/3094903579 conn(0x7fa05c101930 msgr2=0x7fa05c103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 -- 172.21.15.103:0/3094903579 shutdown_connections 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 -- 172.21.15.103:0/3094903579 wait complete. 2024-08-19T01:26:56.855 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 Processor -- start 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.591+0000 7fa0618df640 1 -- start start 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa0618df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa0618df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10b640 con 0x7fa05c107d50 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa05affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa05affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c19b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33624/0 (socket says 172.21.15.103:33624) 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa05affd640 1 -- 172.21.15.103:0/3615544744 learned_addr learned my addr 172.21.15.103:0/3615544744 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa05affd640 1 -- 172.21.15.103:0/3615544744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c19b710 con 0x7fa05c107d50 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa05affd640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c19b1d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa04c030cc0 tx=0x7fa04c030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa04c042070 con 0x7fa05c107d50 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04c030e60 con 0x7fa05c107d50 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa04c037040 con 0x7fa05c107d50 2024-08-19T01:26:56.856 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c19b9a0 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.592+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c19e690 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.593+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa04c045070 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.594+0000 7fa0608dd640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7fa044038fc0 0x7fa04403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.594+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c1081b0 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.594+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa04c06c030 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.594+0000 7fa05a7fc640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7fa044038fc0 0x7fa04403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.594+0000 7fa05a7fc640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7fa044038fc0 0x7fa04403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098c0 tx=0x7fa050006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.598+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa04c046e00 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:54.818+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa05c19ec70 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.780+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa04c045360 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.781+0000 7fa0608dd640 1 -- 172.21.15.103:0/3615544744 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa04c03cb20 con 0x7fa05c107d50 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7fa044038fc0 msgr2=0x7fa04403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:56.857 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7fa044038fc0 0x7fa04403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098c0 tx=0x7fa050006db0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 msgr2=0x7fa05c19b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c19b1d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa04c030cc0 tx=0x7fa04c030cf0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 shutdown_connections 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7fa044038fc0 0x7fa04403b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 --2- 172.21.15.103:0/3615544744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa05c107d50 0x7fa05c19b1d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 >> 172.21.15.103:0/3615544744 conn(0x7fa05c101930 msgr2=0x7fa05c1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 shutdown_connections 2024-08-19T01:26:56.858 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:55.783+0000 7fa0618df640 1 -- 172.21.15.103:0/3615544744 wait complete. 2024-08-19T01:26:56.974 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3615544744' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-19T01:26:56.974 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-mon[18436]: mgrmap e9: x(active, since 19s) 2024-08-19T01:26:56.974 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.664+0000 7f4ff87fa040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:26:56.974 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.791+0000 7f4ff87fa040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:26:56.975 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:56 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:56.849+0000 7f4ff87fa040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:26:57.228 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.163+0000 7f4ff87fa040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:26:57.228 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.226+0000 7f4ff87fa040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:26:57.492 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.299+0000 7f4ff87fa040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:26:57.493 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.450+0000 7f4ff87fa040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:26:57.772 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.511+0000 7f4ff87fa040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:26:57.772 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.582+0000 7f4ff87fa040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:26:57.772 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.770+0000 7f4ff87fa040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:26:58.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/455742337' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-19T01:26:58.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.832+0000 7f4ff87fa040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:26:58.055 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.893+0000 7f4ff87fa040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:26:58.055 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:57 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:57.961+0000 7f4ff87fa040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:26:58.111 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:26:58.111 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-19T01:26:58.111 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-19T01:26:58.111 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-19T01:26:58.111 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-19T01:26:58.111 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:26:58.111 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.284+0000 7f83fedf8640 1 Processor -- start 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.285+0000 7f83fedf8640 1 -- start start 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.285+0000 7f83fedf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f810a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.285+0000 7f83fedf8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8063160 con 0x7f83f8107db0 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.285+0000 7f83fcb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f810a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.285+0000 7f83fcb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f810a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33646/0 (socket says 172.21.15.103:33646) 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.285+0000 7f83fcb94640 1 -- 172.21.15.103:0/3770970394 learned_addr learned my addr 172.21.15.103:0/3770970394 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.285+0000 7f83fcb94640 1 -- 172.21.15.103:0/3770970394 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f810a6e0 con 0x7f83f8107db0 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.286+0000 7f83fcb94640 1 --2- 172.21.15.103:0/3770970394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f810a1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009820 tx=0x7f83ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d5a7fc38b91e97 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:58.112 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.286+0000 7f83f77fe640 1 -- 172.21.15.103:0/3770970394 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec037070 con 0x7f83f8107db0 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.286+0000 7f83f77fe640 1 -- 172.21.15.103:0/3770970394 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ec0328f0 con 0x7f83f8107db0 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.286+0000 7f83f77fe640 1 -- 172.21.15.103:0/3770970394 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec030740 con 0x7f83f8107db0 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.287+0000 7f83fedf8640 1 -- 172.21.15.103:0/3770970394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 msgr2=0x7f83f810a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.287+0000 7f83fedf8640 1 --2- 172.21.15.103:0/3770970394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f810a1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009820 tx=0x7f83ec02a270 comp rx=0 tx=0).stop 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.287+0000 7f83fedf8640 1 -- 172.21.15.103:0/3770970394 shutdown_connections 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.287+0000 7f83fedf8640 1 --2- 172.21.15.103:0/3770970394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f810a1a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.287+0000 7f83fedf8640 1 -- 172.21.15.103:0/3770970394 >> 172.21.15.103:0/3770970394 conn(0x7f83f81018f0 msgr2=0x7f83f8103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.287+0000 7f83fedf8640 1 -- 172.21.15.103:0/3770970394 shutdown_connections 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.287+0000 7f83fedf8640 1 -- 172.21.15.103:0/3770970394 wait complete. 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fedf8640 1 Processor -- start 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fedf8640 1 -- start start 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fedf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:58.113 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fedf8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83f810b640 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fcb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fcb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f819b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33654/0 (socket says 172.21.15.103:33654) 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fcb94640 1 -- 172.21.15.103:0/455742337 learned_addr learned my addr 172.21.15.103:0/455742337 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fcb94640 1 -- 172.21.15.103:0/455742337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f819b760 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83fcb94640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f819b220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83ec030cc0 tx=0x7f83ec030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.288+0000 7f83f5ffb640 1 -- 172.21.15.103:0/455742337 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec042070 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.289+0000 7f83f5ffb640 1 -- 172.21.15.103:0/455742337 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ec030e60 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.289+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f819b9f0 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.289+0000 7f83f5ffb640 1 -- 172.21.15.103:0/455742337 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec037040 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.289+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f819e6e0 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.290+0000 7f83f5ffb640 1 -- 172.21.15.103:0/455742337 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f83ec045070 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.290+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f8108210 con 0x7f83f8107db0 2024-08-19T01:26:58.114 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.290+0000 7f83f5ffb640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 0x7f83e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.290+0000 7f83f5ffb640 1 -- 172.21.15.103:0/455742337 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f83ec06c1e0 con 0x7f83f8107db0 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.290+0000 7f83f7fff640 1 -- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 msgr2=0x7f83e803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2881014674 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.290+0000 7f83f7fff640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 0x7f83e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.295+0000 7f83f5ffb640 1 -- 172.21.15.103:0/455742337 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f83ec03cc10 con 0x7f83f8107db0 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.491+0000 7f83f7fff640 1 -- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 msgr2=0x7f83e803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2881014674 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.491+0000 7f83f7fff640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 0x7f83e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.514+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f83f819ecd0 con 0x7f83f8107db0 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.515+0000 7f83f5ffb640 1 -- 172.21.15.103:0/455742337 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f83ec03cdf0 con 0x7f83f8107db0 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 msgr2=0x7f83e803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 0x7f83e803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 msgr2=0x7f83f819b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f819b220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83ec030cc0 tx=0x7f83ec030cf0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.115 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 shutdown_connections 2024-08-19T01:26:58.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f83e8038fc0 0x7f83e803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 --2- 172.21.15.103:0/455742337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83f8107db0 0x7f83f819b220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:26:58.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 >> 172.21.15.103:0/455742337 conn(0x7f83f81018f0 msgr2=0x7f83f81023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:26:58.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 shutdown_connections 2024-08-19T01:26:58.116 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:57.517+0000 7f83fedf8640 1 -- 172.21.15.103:0/455742337 wait complete. 2024-08-19T01:26:58.116 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-08-19T01:26:58.116 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 9... 2024-08-19T01:26:58.363 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:58 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:58.189+0000 7f4ff87fa040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:26:58.363 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:58 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:58.250+0000 7f4ff87fa040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:26:59.054 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:58 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:58.775+0000 7f4ff87fa040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:26:59.502 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:59 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:59.086+0000 7f4ff87fa040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:26:59.502 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:59 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:59.500+0000 7f4ff87fa040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:26:59.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:59 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:59.556+0000 7f4ff87fa040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:26:59.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:26:59 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:26:59.614+0000 7f4ff87fa040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:27:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: Active manager daemon x restarted 2024-08-19T01:27:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: Activating manager daemon x 2024-08-19T01:27:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: osdmap e3: 0 total, 0 up, 0 in 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: mgrmap e10: x(active, starting, since 0.00352613s) 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: Manager daemon x is now available 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-19T01:27:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:27:02.303+0000 7f4f5e477640 -1 log_channel(cephadm) log [ERR] : [19/Aug/2024:01:27:02] ENGINE Error in HTTPServer.serve 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: Traceback (most recent call last): 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self._connections.run(self.expiration_interval) 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self._run(expiration_interval) 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:27:02.448 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: s = self.context.wrap_socket( 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: return self.sslsocket_class._create( 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self.do_handshake() 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: self._sslobj.do_handshake() 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:27:02.449 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:27:02.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-08-19T01:27:02.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-19T01:27:02.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-19T01:27:02.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-08-19T01:27:02.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f330925c640 1 Processor -- start 2024-08-19T01:27:02.456 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f330925c640 1 -- start start 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f330925c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330410a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f330925c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33040631e0 con 0x7f3304107d30 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330410a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330410a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33660/0 (socket says 172.21.15.103:33660) 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f3302d9d640 1 -- 172.21.15.103:0/2858467703 learned_addr learned my addr 172.21.15.103:0/2858467703 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.549+0000 7f3302d9d640 1 -- 172.21.15.103:0/2858467703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330410a660 con 0x7f3304107d30 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.550+0000 7f3302d9d640 1 --2- 172.21.15.103:0/2858467703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330410a120 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009820 tx=0x7f32f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b70d56b4bd52861 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.550+0000 7f330259c640 1 -- 172.21.15.103:0/2858467703 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32f4037070 con 0x7f3304107d30 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.550+0000 7f330259c640 1 -- 172.21.15.103:0/2858467703 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f40328f0 con 0x7f3304107d30 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.550+0000 7f330259c640 1 -- 172.21.15.103:0/2858467703 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32f4030740 con 0x7f3304107d30 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.551+0000 7f330925c640 1 -- 172.21.15.103:0/2858467703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 msgr2=0x7f330410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.551+0000 7f330925c640 1 --2- 172.21.15.103:0/2858467703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330410a120 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009820 tx=0x7f32f402a270 comp rx=0 tx=0).stop 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.551+0000 7f330925c640 1 -- 172.21.15.103:0/2858467703 shutdown_connections 2024-08-19T01:27:02.457 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.551+0000 7f330925c640 1 --2- 172.21.15.103:0/2858467703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330410a120 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.551+0000 7f330925c640 1 -- 172.21.15.103:0/2858467703 >> 172.21.15.103:0/2858467703 conn(0x7f33041018f0 msgr2=0x7f3304103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.551+0000 7f330925c640 1 -- 172.21.15.103:0/2858467703 shutdown_connections 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.551+0000 7f330925c640 1 -- 172.21.15.103:0/2858467703 wait complete. 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f330925c640 1 Processor -- start 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f330925c640 1 -- start start 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f330925c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f330925c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f330410b680 con 0x7f3304107d30 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f3302d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330419d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33666/0 (socket says 172.21.15.103:33666) 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f3302d9d640 1 -- 172.21.15.103:0/3622784008 learned_addr learned my addr 172.21.15.103:0/3622784008 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f3302d9d640 1 -- 172.21.15.103:0/3622784008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330419d900 con 0x7f3304107d30 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f3302d9d640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330419d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32f4030cc0 tx=0x7f32f4030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.552+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32f4042070 con 0x7f3304107d30 2024-08-19T01:27:02.458 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.553+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f4030e60 con 0x7f3304107d30 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.553+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32f4037040 con 0x7f3304107d30 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.553+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330419db90 con 0x7f3304107d30 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.553+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33041a0880 con 0x7f3304107d30 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.554+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f32f4045070 con 0x7f3304107d30 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.554+0000 7f3300d99640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 0x7f32f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.554+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 --> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f32f003bb90 con 0x7f32f0038fc0 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.554+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f32f406c2e0 con 0x7f3304107d30 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.554+0000 7f32fbfff640 1 -- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 msgr2=0x7f32f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2881014674 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.554+0000 7f32fbfff640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 0x7f32f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.755+0000 7f32fbfff640 1 -- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 msgr2=0x7f32f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2881014674 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:58.755+0000 7f32fbfff640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 0x7f32f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:59.156+0000 7f32fbfff640 1 -- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 msgr2=0x7f32f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2881014674 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:59.156+0000 7f32fbfff640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 0x7f32f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:59.957+0000 7f32fbfff640 1 -- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 msgr2=0x7f32f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2881014674 2024-08-19T01:27:02.459 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:26:59.957+0000 7f32fbfff640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 0x7f32f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:00.699+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f32f406ad90 con 0x7f3304107d30 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:00.699+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 msgr2=0x7f32f003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:00.699+0000 7f3300d99640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 0x7f32f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.701+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f32f403cc10 con 0x7f3304107d30 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.701+0000 7f3300d99640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32f003ca80 0x7f32f003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.701+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f32f400af30 con 0x7f32f003ca80 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.701+0000 7f32fbfff640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32f003ca80 0x7f32f003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.702+0000 7f32fbfff640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32f003ca80 0x7f32f003ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f32ec003980 tx=0x7f32ec006e10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.703+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f32f400af30 con 0x7f32f003ca80 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3304108190 con 0x7f32f003ca80 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f3300d99640 1 -- 172.21.15.103:0/3622784008 <== mgr.14148 v2:172.21.15.103:6800/2656970648 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3304108190 con 0x7f32f003ca80 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32f003ca80 msgr2=0x7f32f003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32f003ca80 0x7f32f003ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f32ec003980 tx=0x7f32ec006e10 comp rx=0 tx=0).stop 2024-08-19T01:27:02.460 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 msgr2=0x7f330419d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330419d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32f4030cc0 tx=0x7f32f4030cf0 comp rx=0 tx=0).stop 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 shutdown_connections 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32f003ca80 0x7f32f003ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:6800/2881014674,v1:172.21.15.103:6801/2881014674] conn(0x7f32f0038fc0 0x7f32f003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 --2- 172.21.15.103:0/3622784008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3304107d30 0x7f330419d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.715+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 >> 172.21.15.103:0/3622784008 conn(0x7f33041018f0 msgr2=0x7f33041023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.716+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 shutdown_connections 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:01.716+0000 7f330925c640 1 -- 172.21.15.103:0/3622784008 wait complete. 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 9 is available 2024-08-19T01:27:02.461 INFO:teuthology.orchestra.run.smithi103.stdout:Generating a dashboard self-signed certificate... 2024-08-19T01:27:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-mon[18436]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-19T01:27:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-mon[18436]: mgrmap e11: x(active, since 1.00578s) 2024-08-19T01:27:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-mon[18436]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-19T01:27:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:03.808 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.902+0000 7f00a9fae640 1 Processor -- start 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.902+0000 7f00a9fae640 1 -- start start 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.902+0000 7f00a9fae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a4107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.902+0000 7f00a9fae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f00a4107c10 con 0x7f00a4107240 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.902+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a4107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.903+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a4107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55056/0 (socket says 172.21.15.103:55056) 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.903+0000 7f00a37fe640 1 -- 172.21.15.103:0/1119021105 learned_addr learned my addr 172.21.15.103:0/1119021105 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.903+0000 7f00a37fe640 1 -- 172.21.15.103:0/1119021105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a4108440 con 0x7f00a4107240 2024-08-19T01:27:03.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.903+0000 7f00a37fe640 1 --2- 172.21.15.103:0/1119021105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a4107640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0094009820 tx=0x7f009402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fb5d966390e38f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a27fc640 1 -- 172.21.15.103:0/1119021105 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0094037070 con 0x7f00a4107240 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a27fc640 1 -- 172.21.15.103:0/1119021105 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00940328f0 con 0x7f00a4107240 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a27fc640 1 -- 172.21.15.103:0/1119021105 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0094030740 con 0x7f00a4107240 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a9fae640 1 -- 172.21.15.103:0/1119021105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 msgr2=0x7f00a4107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a9fae640 1 --2- 172.21.15.103:0/1119021105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a4107640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0094009820 tx=0x7f009402a270 comp rx=0 tx=0).stop 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a9fae640 1 -- 172.21.15.103:0/1119021105 shutdown_connections 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a9fae640 1 --2- 172.21.15.103:0/1119021105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a4107640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a9fae640 1 -- 172.21.15.103:0/1119021105 >> 172.21.15.103:0/1119021105 conn(0x7f00a4102a10 msgr2=0x7f00a4104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a9fae640 1 -- 172.21.15.103:0/1119021105 shutdown_connections 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.904+0000 7f00a9fae640 1 -- 172.21.15.103:0/1119021105 wait complete. 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.905+0000 7f00a9fae640 1 Processor -- start 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.905+0000 7f00a9fae640 1 -- start start 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.905+0000 7f00a9fae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:03.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.905+0000 7f00a9fae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f00a410b7f0 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.905+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.905+0000 7f00a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55070/0 (socket says 172.21.15.103:55070) 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.905+0000 7f00a37fe640 1 -- 172.21.15.103:0/638511004 learned_addr learned my addr 172.21.15.103:0/638511004 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.906+0000 7f00a37fe640 1 -- 172.21.15.103:0/638511004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a419b750 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.906+0000 7f00a37fe640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a419b210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0094009940 tx=0x7f0094030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.906+0000 7f00a0ff9640 1 -- 172.21.15.103:0/638511004 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0094042070 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.906+0000 7f00a0ff9640 1 -- 172.21.15.103:0/638511004 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0094030e30 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.906+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a419b9e0 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.906+0000 7f00a0ff9640 1 -- 172.21.15.103:0/638511004 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0094037040 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.906+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a419e6d0 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.907+0000 7f00a0ff9640 1 -- 172.21.15.103:0/638511004 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f0094045070 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.907+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a410b970 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.907+0000 7f00a0ff9640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f008c038eb0 0x7f008c03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.907+0000 7f00a0ff9640 1 -- 172.21.15.103:0/638511004 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f009406c1f0 con 0x7f00a4107240 2024-08-19T01:27:03.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.907+0000 7f00a2ffd640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f008c038eb0 0x7f008c03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.907+0000 7f00a2ffd640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f008c038eb0 0x7f008c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00980098c0 tx=0x7f0098006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:02.912+0000 7f00a0ff9640 1 -- 172.21.15.103:0/638511004 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f009403add0 con 0x7f00a4107240 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.094+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f00a4103c30 con 0x7f008c038eb0 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.214+0000 7f00a0ff9640 1 -- 172.21.15.103:0/638511004 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f00a4103c30 con 0x7f008c038eb0 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f008c038eb0 msgr2=0x7f008c03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f008c038eb0 0x7f008c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f00980098c0 tx=0x7f0098006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 msgr2=0x7f00a419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a419b210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0094009940 tx=0x7f0094030cc0 comp rx=0 tx=0).stop 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 shutdown_connections 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f008c038eb0 0x7f008c03b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 --2- 172.21.15.103:0/638511004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00a4107240 0x7f00a419b210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 >> 172.21.15.103:0/638511004 conn(0x7f00a4102a10 msgr2=0x7f00a4103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 shutdown_connections 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:03.216+0000 7f00a9fae640 1 -- 172.21.15.103:0/638511004 wait complete. 2024-08-19T01:27:03.816 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial admin user... 2024-08-19T01:27:03.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: [19/Aug/2024:01:27:02] ENGINE Bus STARTING 2024-08-19T01:27:03.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: [19/Aug/2024:01:27:02] ENGINE Serving on http://172.21.15.103:8765 2024-08-19T01:27:03.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: [19/Aug/2024:01:27:02] ENGINE Serving on https://172.21.15.103:7150 2024-08-19T01:27:03.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: [19/Aug/2024:01:27:02] ENGINE Bus STARTED 2024-08-19T01:27:03.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: [19/Aug/2024:01:27:02] ENGINE Error in HTTPServer.serve 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-19T01:27:03.932 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:03.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:03 smithi103 ceph-mon[18436]: mgrmap e12: x(active, since 2s) 2024-08-19T01:27:05.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:04 smithi103 ceph-mon[18436]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:05.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:04 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:05.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:04 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:06.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:05 smithi103 ceph-mon[18436]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:06.176 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$bsd6TPoEu5rd/cb6.2F8b.eEGCupV9V/Qp8irMDCsQ47XC64Ap1mS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1724030824, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.198+0000 7fe6abd81640 1 Processor -- start 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.198+0000 7fe6abd81640 1 -- start start 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.198+0000 7fe6abd81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a410a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.198+0000 7fe6abd81640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a40631e0 con 0x7fe6a4107d30 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.199+0000 7fe6a9b1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a410a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.199+0000 7fe6a9b1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a410a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55074/0 (socket says 172.21.15.103:55074) 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.199+0000 7fe6a9b1d640 1 -- 172.21.15.103:0/901149744 learned_addr learned my addr 172.21.15.103:0/901149744 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.199+0000 7fe6a9b1d640 1 -- 172.21.15.103:0/901149744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a410a660 con 0x7fe6a4107d30 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.200+0000 7fe6a9b1d640 1 --2- 172.21.15.103:0/901149744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a410a120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009820 tx=0x7fe6a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec31f683579a50ed server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.200+0000 7fe6a8b1b640 1 -- 172.21.15.103:0/901149744 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a0037070 con 0x7fe6a4107d30 2024-08-19T01:27:06.177 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.200+0000 7fe6a8b1b640 1 -- 172.21.15.103:0/901149744 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a00328f0 con 0x7fe6a4107d30 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.200+0000 7fe6a8b1b640 1 -- 172.21.15.103:0/901149744 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a0030740 con 0x7fe6a4107d30 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.200+0000 7fe6abd81640 1 -- 172.21.15.103:0/901149744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 msgr2=0x7fe6a410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.200+0000 7fe6abd81640 1 --2- 172.21.15.103:0/901149744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a410a120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009820 tx=0x7fe6a002a270 comp rx=0 tx=0).stop 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 -- 172.21.15.103:0/901149744 shutdown_connections 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 --2- 172.21.15.103:0/901149744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a410a120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 -- 172.21.15.103:0/901149744 >> 172.21.15.103:0/901149744 conn(0x7fe6a41018f0 msgr2=0x7fe6a4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 -- 172.21.15.103:0/901149744 shutdown_connections 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 -- 172.21.15.103:0/901149744 wait complete. 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 Processor -- start 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 -- start start 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:06.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.201+0000 7fe6abd81640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a410b680 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe6a9b1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe6a9b1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a419b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55082/0 (socket says 172.21.15.103:55082) 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe6a9b1d640 1 -- 172.21.15.103:0/798551371 learned_addr learned my addr 172.21.15.103:0/798551371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe6a9b1d640 1 -- 172.21.15.103:0/798551371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a419b700 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe6a9b1d640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a419b1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0030cc0 tx=0x7fe6a0030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe69affd640 1 -- 172.21.15.103:0/798551371 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a0042070 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a419b990 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe69affd640 1 -- 172.21.15.103:0/798551371 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a0030e60 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe69affd640 1 -- 172.21.15.103:0/798551371 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a0037040 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.202+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a419e680 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.203+0000 7fe69affd640 1 -- 172.21.15.103:0/798551371 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe6a0045070 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.203+0000 7fe69affd640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe690039020 0x7fe69003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.203+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a4108190 con 0x7fe6a4107d30 2024-08-19T01:27:06.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.203+0000 7fe69affd640 1 -- 172.21.15.103:0/798551371 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe6a006c1b0 con 0x7fe6a4107d30 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.203+0000 7fe6a931c640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe690039020 0x7fe69003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.204+0000 7fe6a931c640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe690039020 0x7fe69003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098c0 tx=0x7fe694006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.212+0000 7fe69affd640 1 -- 172.21.15.103:0/798551371 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe6a0045360 con 0x7fe6a4107d30 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.416+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe6a419e910 con 0x7fe690039020 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.646+0000 7fe69affd640 1 -- 172.21.15.103:0/798551371 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe6a419e910 con 0x7fe690039020 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe690039020 msgr2=0x7fe69003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe690039020 0x7fe69003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098c0 tx=0x7fe694006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 msgr2=0x7fe6a419b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a419b1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0030cc0 tx=0x7fe6a0030cf0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 shutdown_connections 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe690039020 0x7fe69003b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.180 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 --2- 172.21.15.103:0/798551371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6a4107d30 0x7fe6a419b1c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:06.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 >> 172.21.15.103:0/798551371 conn(0x7fe6a41018f0 msgr2=0x7fe6a4102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:06.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 shutdown_connections 2024-08-19T01:27:06.181 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:04.648+0000 7fe6abd81640 1 -- 172.21.15.103:0/798551371 wait complete. 2024-08-19T01:27:06.181 INFO:teuthology.orchestra.run.smithi103.stdout:Fetching dashboard port number... 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 8443 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.558+0000 7f3f5b1b6640 1 Processor -- start 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.558+0000 7f3f5b1b6640 1 -- start start 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.559+0000 7f3f5b1b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54105920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.559+0000 7f3f5b1b6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f54105ef0 con 0x7f3f54105520 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.559+0000 7f3f58f52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.559+0000 7f3f58f52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54105920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55086/0 (socket says 172.21.15.103:55086) 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.559+0000 7f3f58f52640 1 -- 172.21.15.103:0/530381594 learned_addr learned my addr 172.21.15.103:0/530381594 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:07.309 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.559+0000 7f3f58f52640 1 -- 172.21.15.103:0/530381594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f54108500 con 0x7f3f54105520 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f58f52640 1 --2- 172.21.15.103:0/530381594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54105920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009820 tx=0x7f3f4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321dd1f5bf1049d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f537fe640 1 -- 172.21.15.103:0/530381594 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f48037070 con 0x7f3f54105520 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f537fe640 1 -- 172.21.15.103:0/530381594 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f480328f0 con 0x7f3f54105520 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f537fe640 1 -- 172.21.15.103:0/530381594 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f48030740 con 0x7f3f54105520 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/530381594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 msgr2=0x7f3f54105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f5b1b6640 1 --2- 172.21.15.103:0/530381594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54105920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009820 tx=0x7f3f4802a270 comp rx=0 tx=0).stop 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/530381594 shutdown_connections 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f5b1b6640 1 --2- 172.21.15.103:0/530381594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54105920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.560+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/530381594 >> 172.21.15.103:0/530381594 conn(0x7f3f54101070 msgr2=0x7f3f541034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/530381594 shutdown_connections 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/530381594 wait complete. 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f5b1b6640 1 Processor -- start 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f5b1b6640 1 -- start start 2024-08-19T01:27:07.310 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f5b1b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54107d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f5b1b6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410b730 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f58f52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f58f52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54107d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55092/0 (socket says 172.21.15.103:55092) 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.561+0000 7f3f58f52640 1 -- 172.21.15.103:0/998751666 learned_addr learned my addr 172.21.15.103:0/998751666 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.562+0000 7f3f58f52640 1 -- 172.21.15.103:0/998751666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f541082c0 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.562+0000 7f3f58f52640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54107d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009940 tx=0x7f3f48030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.562+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f48042070 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.562+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f541064b0 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.562+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f48030e30 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.562+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f48037040 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.562+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f541069f0 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.563+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3f48045070 con 0x7f3f54105520 2024-08-19T01:27:07.311 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.563+0000 7f3f51ffb640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f44038fc0 0x7f3f4403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.563+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f4806d0b0 con 0x7f3f54105520 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.563+0000 7f3f53fff640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f44038fc0 0x7f3f4403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.563+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f541059a0 con 0x7f3f54105520 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.564+0000 7f3f53fff640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f44038fc0 0x7f3f4403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f400098c0 tx=0x7f3f40006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.568+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3f48045360 con 0x7f3f54105520 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.704+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3f4803b330 con 0x7f3f54105520 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.733+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3f54107070 con 0x7f3f54105520 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.733+0000 7f3f51ffb640 1 -- 172.21.15.103:0/998751666 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3f48043260 con 0x7f3f54105520 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f44038fc0 msgr2=0x7f3f4403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:07.312 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f44038fc0 0x7f3f4403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f400098c0 tx=0x7f3f40006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 msgr2=0x7f3f54107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54107d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009940 tx=0x7f3f48030cc0 comp rx=0 tx=0).stop 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 shutdown_connections 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f44038fc0 0x7f3f4403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 --2- 172.21.15.103:0/998751666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f54105520 0x7f3f54107d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.735+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 >> 172.21.15.103:0/998751666 conn(0x7f3f54101070 msgr2=0x7f3f54101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.736+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 shutdown_connections 2024-08-19T01:27:07.313 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:06.736+0000 7f3f5b1b6640 1 -- 172.21.15.103:0/998751666 wait complete. 2024-08-19T01:27:07.317 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-19T01:27:07.317 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-08-19T01:27:07.325 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-19T01:27:07.325 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-08-19T01:27:07.325 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-08-19T01:27:07.325 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-19T01:27:07.326 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph Dashboard is now available at: 2024-08-19T01:27:07.326 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:07.327 INFO:teuthology.orchestra.run.smithi103.stdout: URL: https://smithi103:8443/ 2024-08-19T01:27:07.327 INFO:teuthology.orchestra.run.smithi103.stdout: User: admin 2024-08-19T01:27:07.327 INFO:teuthology.orchestra.run.smithi103.stdout: Password: zotgy98tlm 2024-08-19T01:27:07.327 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:07.327 INFO:teuthology.orchestra.run.smithi103.stdout:Saving cluster configuration to /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config directory 2024-08-19T01:27:08.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:07 smithi103 ceph-mon[18436]: mgrmap e13: x(active, since 6s) 2024-08-19T01:27:08.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:07 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/998751666' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-08-19T01:27:08.582 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.740+0000 7fdbe118d640 1 Processor -- start 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.740+0000 7fdbe118d640 1 -- start start 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.740+0000 7fdbe118d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.741+0000 7fdbe118d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc07c850 con 0x7fdbdc07de20 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.741+0000 7fdbdad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.741+0000 7fdbdad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55096/0 (socket says 172.21.15.103:55096) 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.741+0000 7fdbdad9d640 1 -- 172.21.15.103:0/909868750 learned_addr learned my addr 172.21.15.103:0/909868750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.741+0000 7fdbdad9d640 1 -- 172.21.15.103:0/909868750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc07c9d0 con 0x7fdbdc07de20 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbdad9d640 1 --2- 172.21.15.103:0/909868750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009820 tx=0x7fdbcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e6634f3b8389 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbd9d9b640 1 -- 172.21.15.103:0/909868750 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc037070 con 0x7fdbdc07de20 2024-08-19T01:27:08.583 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbd9d9b640 1 -- 172.21.15.103:0/909868750 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbcc0328f0 con 0x7fdbdc07de20 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbd9d9b640 1 -- 172.21.15.103:0/909868750 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc030740 con 0x7fdbdc07de20 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbe118d640 1 -- 172.21.15.103:0/909868750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 msgr2=0x7fdbdc07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbe118d640 1 --2- 172.21.15.103:0/909868750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009820 tx=0x7fdbcc02a270 comp rx=0 tx=0).stop 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbe118d640 1 -- 172.21.15.103:0/909868750 shutdown_connections 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbe118d640 1 --2- 172.21.15.103:0/909868750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc07c280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbe118d640 1 -- 172.21.15.103:0/909868750 >> 172.21.15.103:0/909868750 conn(0x7fdbdc102a10 msgr2=0x7fdbdc104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.742+0000 7fdbe118d640 1 -- 172.21.15.103:0/909868750 shutdown_connections 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.743+0000 7fdbe118d640 1 -- 172.21.15.103:0/909868750 wait complete. 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.743+0000 7fdbe118d640 1 Processor -- start 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.743+0000 7fdbe118d640 1 -- start start 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbe118d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbe118d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc10d8b0 con 0x7fdbdc07de20 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbdad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:08.584 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbdad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55110/0 (socket says 172.21.15.103:55110) 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbdad9d640 1 -- 172.21.15.103:0/2185045868 learned_addr learned my addr 172.21.15.103:0/2185045868 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbdad9d640 1 -- 172.21.15.103:0/2185045868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc19d8e0 con 0x7fdbdc07de20 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbdad9d640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19d3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc030cc0 tx=0x7fdbcc030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbd3fff640 1 -- 172.21.15.103:0/2185045868 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc042070 con 0x7fdbdc07de20 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbd3fff640 1 -- 172.21.15.103:0/2185045868 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbcc030e60 con 0x7fdbdc07de20 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc19db70 con 0x7fdbdc07de20 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbd3fff640 1 -- 172.21.15.103:0/2185045868 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc037040 con 0x7fdbdc07de20 2024-08-19T01:27:08.585 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.744+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc1a0860 con 0x7fdbdc07de20 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.745+0000 7fdbd3fff640 1 -- 172.21.15.103:0/2185045868 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fdbcc045070 con 0x7fdbdc07de20 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.745+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc07c300 con 0x7fdbdc07de20 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.746+0000 7fdbd3fff640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdbc8039060 0x7fdbc803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.746+0000 7fdbd3fff640 1 -- 172.21.15.103:0/2185045868 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdbcc06c430 con 0x7fdbdc07de20 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.746+0000 7fdbda59c640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdbc8039060 0x7fdbc803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.746+0000 7fdbda59c640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdbc8039060 0x7fdbc803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098c0 tx=0x7fdbc4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:07.750+0000 7fdbd3fff640 1 -- 172.21.15.103:0/2185045868 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdbcc045360 con 0x7fdbdc07de20 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.004+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fdbdc1a0a30 con 0x7fdbdc07de20 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.007+0000 7fdbd3fff640 1 -- 172.21.15.103:0/2185045868 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fdbcc03ad90 con 0x7fdbdc07de20 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdbc8039060 msgr2=0x7fdbc803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:08.586 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdbc8039060 0x7fdbc803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098c0 tx=0x7fdbc4006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 msgr2=0x7fdbdc19d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19d3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc030cc0 tx=0x7fdbcc030cf0 comp rx=0 tx=0).stop 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 shutdown_connections 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdbc8039060 0x7fdbc803b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 --2- 172.21.15.103:0/2185045868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdbdc07de20 0x7fdbdc19d3a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 >> 172.21.15.103:0/2185045868 conn(0x7fdbdc102a10 msgr2=0x7fdbdc103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 shutdown_connections 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-08-19T01:27:08.009+0000 7fdbe118d640 1 -- 172.21.15.103:0/2185045868 wait complete. 2024-08-19T01:27:08.587 INFO:teuthology.orchestra.run.smithi103.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout:Or, if you are only running a single cluster on this host: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: ceph telemetry on 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout:For more information see: 2024-08-19T01:27:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-19T01:27:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:08.589 INFO:teuthology.orchestra.run.smithi103.stdout:Bootstrap complete. 2024-08-19T01:27:08.621 INFO:tasks.cephadm:Fetching config... 2024-08-19T01:27:08.622 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:27:08.622 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-19T01:27:08.638 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-19T01:27:08.638 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:27:08.638 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-19T01:27:08.694 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-19T01:27:08.697 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:27:08.697 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-19T01:27:08.760 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-19T01:27:08.760 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:27:08.760 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-19T01:27:08.816 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-19T01:27:08.816 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEZsTwVhFODJwXVyb5nJ3wHPtoGyv4nGrahenkuKazNWGdVWOBQhKaq478T35fgS0mz9OXZAMKpDChB8te3Xzh3fb4wqeWqFmXDunTdAb7UoK39teFSyD1lA/HpYFaPR2aG8NdqrH6MmeqOPdoOtYOsebH4mJnObfJrRa96AXDmFsADvZnYy7xscVe4i5L7UJ3sIfXDZ3qmN3OP++SLHoMAPfR0qibvLPvOJVd8XWPWyz+hY2oJjW3IIaS5ALCqE/mCCVqw1veMXdtJ6UZacA7n7zlPm7vFkeyFuAcnXzfOMEqGD2XMu6oCfborLjphibBMqfCYKQlkSxylLLGBOI+zN/ETakyhRk3q0GyMaBkjb1kFqJI1EvNG2+rTQj7IkZ+Nx0nJr5vhWrbh3Wmek14JyzYbYo3BCTXJ6saLWpy5T+EwUx2zEAYyNR3S1KkkQQrKvf5R8PCyPNFium6B36tdVUIOHjY2M9+hfhZavz8pajG5XC9BPUaQ0mBXm/rsBM= ceph-e721016a-5dc9-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-19T01:27:08.891 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEZsTwVhFODJwXVyb5nJ3wHPtoGyv4nGrahenkuKazNWGdVWOBQhKaq478T35fgS0mz9OXZAMKpDChB8te3Xzh3fb4wqeWqFmXDunTdAb7UoK39teFSyD1lA/HpYFaPR2aG8NdqrH6MmeqOPdoOtYOsebH4mJnObfJrRa96AXDmFsADvZnYy7xscVe4i5L7UJ3sIfXDZ3qmN3OP++SLHoMAPfR0qibvLPvOJVd8XWPWyz+hY2oJjW3IIaS5ALCqE/mCCVqw1veMXdtJ6UZacA7n7zlPm7vFkeyFuAcnXzfOMEqGD2XMu6oCfborLjphibBMqfCYKQlkSxylLLGBOI+zN/ETakyhRk3q0GyMaBkjb1kFqJI1EvNG2+rTQj7IkZ+Nx0nJr5vhWrbh3Wmek14JyzYbYo3BCTXJ6saLWpy5T+EwUx2zEAYyNR3S1KkkQQrKvf5R8PCyPNFium6B36tdVUIOHjY2M9+hfhZavz8pajG5XC9BPUaQ0mBXm/rsBM= ceph-e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:08.903 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEZsTwVhFODJwXVyb5nJ3wHPtoGyv4nGrahenkuKazNWGdVWOBQhKaq478T35fgS0mz9OXZAMKpDChB8te3Xzh3fb4wqeWqFmXDunTdAb7UoK39teFSyD1lA/HpYFaPR2aG8NdqrH6MmeqOPdoOtYOsebH4mJnObfJrRa96AXDmFsADvZnYy7xscVe4i5L7UJ3sIfXDZ3qmN3OP++SLHoMAPfR0qibvLPvOJVd8XWPWyz+hY2oJjW3IIaS5ALCqE/mCCVqw1veMXdtJ6UZacA7n7zlPm7vFkeyFuAcnXzfOMEqGD2XMu6oCfborLjphibBMqfCYKQlkSxylLLGBOI+zN/ETakyhRk3q0GyMaBkjb1kFqJI1EvNG2+rTQj7IkZ+Nx0nJr5vhWrbh3Wmek14JyzYbYo3BCTXJ6saLWpy5T+EwUx2zEAYyNR3S1KkkQQrKvf5R8PCyPNFium6B36tdVUIOHjY2M9+hfhZavz8pajG5XC9BPUaQ0mBXm/rsBM= ceph-e721016a-5dc9-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-19T01:27:08.945 INFO:teuthology.orchestra.run.smithi146.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEZsTwVhFODJwXVyb5nJ3wHPtoGyv4nGrahenkuKazNWGdVWOBQhKaq478T35fgS0mz9OXZAMKpDChB8te3Xzh3fb4wqeWqFmXDunTdAb7UoK39teFSyD1lA/HpYFaPR2aG8NdqrH6MmeqOPdoOtYOsebH4mJnObfJrRa96AXDmFsADvZnYy7xscVe4i5L7UJ3sIfXDZ3qmN3OP++SLHoMAPfR0qibvLPvOJVd8XWPWyz+hY2oJjW3IIaS5ALCqE/mCCVqw1veMXdtJ6UZacA7n7zlPm7vFkeyFuAcnXzfOMEqGD2XMu6oCfborLjphibBMqfCYKQlkSxylLLGBOI+zN/ETakyhRk3q0GyMaBkjb1kFqJI1EvNG2+rTQj7IkZ+Nx0nJr5vhWrbh3Wmek14JyzYbYo3BCTXJ6saLWpy5T+EwUx2zEAYyNR3S1KkkQQrKvf5R8PCyPNFium6B36tdVUIOHjY2M9+hfhZavz8pajG5XC9BPUaQ0mBXm/rsBM= ceph-e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:08.958 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEZsTwVhFODJwXVyb5nJ3wHPtoGyv4nGrahenkuKazNWGdVWOBQhKaq478T35fgS0mz9OXZAMKpDChB8te3Xzh3fb4wqeWqFmXDunTdAb7UoK39teFSyD1lA/HpYFaPR2aG8NdqrH6MmeqOPdoOtYOsebH4mJnObfJrRa96AXDmFsADvZnYy7xscVe4i5L7UJ3sIfXDZ3qmN3OP++SLHoMAPfR0qibvLPvOJVd8XWPWyz+hY2oJjW3IIaS5ALCqE/mCCVqw1veMXdtJ6UZacA7n7zlPm7vFkeyFuAcnXzfOMEqGD2XMu6oCfborLjphibBMqfCYKQlkSxylLLGBOI+zN/ETakyhRk3q0GyMaBkjb1kFqJI1EvNG2+rTQj7IkZ+Nx0nJr5vhWrbh3Wmek14JyzYbYo3BCTXJ6saLWpy5T+EwUx2zEAYyNR3S1KkkQQrKvf5R8PCyPNFium6B36tdVUIOHjY2M9+hfhZavz8pajG5XC9BPUaQ0mBXm/rsBM= ceph-e721016a-5dc9-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-19T01:27:08.999 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEZsTwVhFODJwXVyb5nJ3wHPtoGyv4nGrahenkuKazNWGdVWOBQhKaq478T35fgS0mz9OXZAMKpDChB8te3Xzh3fb4wqeWqFmXDunTdAb7UoK39teFSyD1lA/HpYFaPR2aG8NdqrH6MmeqOPdoOtYOsebH4mJnObfJrRa96AXDmFsADvZnYy7xscVe4i5L7UJ3sIfXDZ3qmN3OP++SLHoMAPfR0qibvLPvOJVd8XWPWyz+hY2oJjW3IIaS5ALCqE/mCCVqw1veMXdtJ6UZacA7n7zlPm7vFkeyFuAcnXzfOMEqGD2XMu6oCfborLjphibBMqfCYKQlkSxylLLGBOI+zN/ETakyhRk3q0GyMaBkjb1kFqJI1EvNG2+rTQj7IkZ+Nx0nJr5vhWrbh3Wmek14JyzYbYo3BCTXJ6saLWpy5T+EwUx2zEAYyNR3S1KkkQQrKvf5R8PCyPNFium6B36tdVUIOHjY2M9+hfhZavz8pajG5XC9BPUaQ0mBXm/rsBM= ceph-e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:09.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-19T01:27:09.261 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:09 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2185045868' entity='client.admin' 2024-08-19T01:27:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.775+0000 7fd9093f6640 1 -- 172.21.15.103:0/4029636775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 msgr2=0x7fd904100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.775+0000 7fd9093f6640 1 --2- 172.21.15.103:0/4029636775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd904100660 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009c20 tx=0x7fd8fc02aa90 comp rx=0 tx=0).stop 2024-08-19T01:27:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.775+0000 7fd9093f6640 1 -- 172.21.15.103:0/4029636775 shutdown_connections 2024-08-19T01:27:09.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.775+0000 7fd9093f6640 1 --2- 172.21.15.103:0/4029636775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd904100660 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.775+0000 7fd9093f6640 1 -- 172.21.15.103:0/4029636775 >> 172.21.15.103:0/4029636775 conn(0x7fd9040fb9d0 msgr2=0x7fd9040fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.775+0000 7fd9093f6640 1 -- 172.21.15.103:0/4029636775 shutdown_connections 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.775+0000 7fd9093f6640 1 -- 172.21.15.103:0/4029636775 wait complete. 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd9093f6640 1 Processor -- start 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd9093f6640 1 -- start start 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd9093f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd9040fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd9093f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd904106220 con 0x7fd904100260 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd902ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd9040fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd902ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd9040fd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55130/0 (socket says 172.21.15.103:55130) 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd902ffd640 1 -- 172.21.15.103:0/3560500060 learned_addr learned my addr 172.21.15.103:0/3560500060 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.776+0000 7fd902ffd640 1 -- 172.21.15.103:0/3560500060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9040fddf0 con 0x7fd904100260 2024-08-19T01:27:09.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.777+0000 7fd902ffd640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd9040fd8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc005cc0 tx=0x7fd8fc02b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.777+0000 7fd8fbfff640 1 -- 172.21.15.103:0/3560500060 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8fc047070 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.777+0000 7fd8fbfff640 1 -- 172.21.15.103:0/3560500060 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8fc003e20 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.777+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9040fbfe0 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.777+0000 7fd8fbfff640 1 -- 172.21.15.103:0/3560500060 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8fc033050 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.777+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9040fc520 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.778+0000 7fd8fbfff640 1 -- 172.21.15.103:0/3560500060 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd8fc04a070 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.778+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9041069a0 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.778+0000 7fd8fbfff640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd8f00390c0 0x7fd8f003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.778+0000 7fd8fbfff640 1 -- 172.21.15.103:0/3560500060 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd8fc03e030 con 0x7fd904100260 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.778+0000 7fd9027fc640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd8f00390c0 0x7fd8f003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.779+0000 7fd9027fc640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd8f00390c0 0x7fd8f003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0098c0 tx=0x7fd8ec006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:09.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.782+0000 7fd8fbfff640 1 -- 172.21.15.103:0/3560500060 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd8fc0098b0 con 0x7fd904100260 2024-08-19T01:27:09.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.951+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd90418ff60 con 0x7fd904100260 2024-08-19T01:27:09.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.953+0000 7fd8fbfff640 1 -- 172.21.15.103:0/3560500060 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd8fc038450 con 0x7fd904100260 2024-08-19T01:27:09.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd8f00390c0 msgr2=0x7fd8f003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:09.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd8f00390c0 0x7fd8f003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0098c0 tx=0x7fd8ec006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:09.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 msgr2=0x7fd9040fd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:09.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd9040fd8b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc005cc0 tx=0x7fd8fc02b020 comp rx=0 tx=0).stop 2024-08-19T01:27:09.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 shutdown_connections 2024-08-19T01:27:09.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd8f00390c0 0x7fd8f003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:09.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 --2- 172.21.15.103:0/3560500060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd904100260 0x7fd9040fd8b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:09.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 >> 172.21.15.103:0/3560500060 conn(0x7fd9040fb9d0 msgr2=0x7fd9040698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:09.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 shutdown_connections 2024-08-19T01:27:09.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:09.956+0000 7fd9093f6640 1 -- 172.21.15.103:0/3560500060 wait complete. 2024-08-19T01:27:10.603 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-19T01:27:10.603 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-19T01:27:11.086 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:11.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:10 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3560500060' entity='client.admin' 2024-08-19T01:27:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.749+0000 7f2f925c4640 1 -- 172.21.15.103:0/3191475637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 msgr2=0x7f2f8c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.749+0000 7f2f925c4640 1 --2- 172.21.15.103:0/3191475637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c101750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009880 tx=0x7f2f7c02a5d0 comp rx=0 tx=0).stop 2024-08-19T01:27:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.749+0000 7f2f925c4640 1 -- 172.21.15.103:0/3191475637 shutdown_connections 2024-08-19T01:27:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.749+0000 7f2f925c4640 1 --2- 172.21.15.103:0/3191475637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c101750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.749+0000 7f2f925c4640 1 -- 172.21.15.103:0/3191475637 >> 172.21.15.103:0/3191475637 conn(0x7f2f8c0fcb20 msgr2=0x7f2f8c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.749+0000 7f2f925c4640 1 -- 172.21.15.103:0/3191475637 shutdown_connections 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.749+0000 7f2f925c4640 1 -- 172.21.15.103:0/3191475637 wait complete. 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.750+0000 7f2f925c4640 1 Processor -- start 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.750+0000 7f2f925c4640 1 -- start start 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.750+0000 7f2f925c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.750+0000 7f2f925c4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c107310 con 0x7f2f8c101350 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.750+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.750+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55150/0 (socket says 172.21.15.103:55150) 2024-08-19T01:27:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.750+0000 7f2f8bfff640 1 -- 172.21.15.103:0/1993168336 learned_addr learned my addr 172.21.15.103:0/1993168336 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.751+0000 7f2f8bfff640 1 -- 172.21.15.103:0/1993168336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c196f50 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.751+0000 7f2f8bfff640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c196a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009230 tx=0x7f2f7c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.751+0000 7f2f897fa640 1 -- 172.21.15.103:0/1993168336 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f7c03f070 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.751+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c1971e0 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.751+0000 7f2f897fa640 1 -- 172.21.15.103:0/1993168336 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f7c003aa0 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.751+0000 7f2f897fa640 1 -- 172.21.15.103:0/1993168336 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f7c035030 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.751+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c199ed0 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.752+0000 7f2f897fa640 1 -- 172.21.15.103:0/1993168336 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2f7c042070 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.752+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c107a90 con 0x7f2f8c101350 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.752+0000 7f2f897fa640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f74039060 0x7f2f7403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.752+0000 7f2f897fa640 1 -- 172.21.15.103:0/1993168336 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f7c06ce70 con 0x7f2f8c101350 2024-08-19T01:27:11.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.752+0000 7f2f8b7fe640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f74039060 0x7f2f7403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:11.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.753+0000 7f2f8b7fe640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f74039060 0x7f2f7403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f800098c0 tx=0x7f2f80006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.757+0000 7f2f897fa640 1 -- 172.21.15.103:0/1993168336 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2f7c02f300 con 0x7f2f8c101350 2024-08-19T01:27:11.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.934+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f2f8c02d650 con 0x7f2f74039060 2024-08-19T01:27:11.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.938+0000 7f2f897fa640 1 -- 172.21.15.103:0/1993168336 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f8c02d650 con 0x7f2f74039060 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f74039060 msgr2=0x7f2f7403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f74039060 0x7f2f7403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f800098c0 tx=0x7f2f80006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 msgr2=0x7f2f8c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c196a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009230 tx=0x7f2f7c0048f0 comp rx=0 tx=0).stop 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 shutdown_connections 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f74039060 0x7f2f7403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 --2- 172.21.15.103:0/1993168336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f8c101350 0x7f2f8c196a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 >> 172.21.15.103:0/1993168336 conn(0x7f2f8c0fcb20 msgr2=0x7f2f8c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:11.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 shutdown_connections 2024-08-19T01:27:11.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:11.940+0000 7f2f925c4640 1 -- 172.21.15.103:0/1993168336 wait complete. 2024-08-19T01:27:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:12.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:12.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:12.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:12.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:12.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:12.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:12.734 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi146 2024-08-19T01:27:12.734 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:27:12.734 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.conf 2024-08-19T01:27:12.753 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:27:12.753 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:12.812 INFO:tasks.cephadm:Adding host smithi146 to orchestrator... 2024-08-19T01:27:12.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch host add smithi146 2024-08-19T01:27:13.493 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: Updating smithi103:/etc/ceph/ceph.conf 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: Updating smithi103:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: Updating smithi103:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:13.923 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.920+0000 7fa809ae6640 1 -- 172.21.15.103:0/1201543140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 msgr2=0x7fa804100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.920+0000 7fa809ae6640 1 --2- 172.21.15.103:0/1201543140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804100aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009880 tx=0x7fa7f402a5d0 comp rx=0 tx=0).stop 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.920+0000 7fa809ae6640 1 -- 172.21.15.103:0/1201543140 shutdown_connections 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.920+0000 7fa809ae6640 1 --2- 172.21.15.103:0/1201543140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804100aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.920+0000 7fa809ae6640 1 -- 172.21.15.103:0/1201543140 >> 172.21.15.103:0/1201543140 conn(0x7fa804075f80 msgr2=0x7fa804076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.920+0000 7fa809ae6640 1 -- 172.21.15.103:0/1201543140 shutdown_connections 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.920+0000 7fa809ae6640 1 -- 172.21.15.103:0/1201543140 wait complete. 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.921+0000 7fa809ae6640 1 Processor -- start 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.921+0000 7fa809ae6640 1 -- start start 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.921+0000 7fa809ae6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.921+0000 7fa809ae6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa80410a780 con 0x7fa8041006c0 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.921+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.921+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804101f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49230/0 (socket says 172.21.15.103:49230) 2024-08-19T01:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.921+0000 7fa8037fe640 1 -- 172.21.15.103:0/1830638352 learned_addr learned my addr 172.21.15.103:0/1830638352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.922+0000 7fa8037fe640 1 -- 172.21.15.103:0/1830638352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa804103dd0 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.922+0000 7fa8037fe640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804101f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009230 tx=0x7fa7f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.922+0000 7fa8017fa640 1 -- 172.21.15.103:0/1830638352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7f403f070 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.922+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa804102470 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.922+0000 7fa8017fa640 1 -- 172.21.15.103:0/1830638352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f4004d10 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.922+0000 7fa8017fa640 1 -- 172.21.15.103:0/1830638352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7f4035030 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.922+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa804102930 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.923+0000 7fa8017fa640 1 -- 172.21.15.103:0/1830638352 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa7f4042070 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.923+0000 7fa8017fa640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa7f0039060 0x7fa7f003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.923+0000 7fa8017fa640 1 -- 172.21.15.103:0/1830638352 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa7f406ce90 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.923+0000 7fa7fbfff640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa7f0039060 0x7fa7f003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.924+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa804101970 con 0x7fa8041006c0 2024-08-19T01:27:13.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.924+0000 7fa7fbfff640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa7f0039060 0x7fa7f003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0098c0 tx=0x7fa7ec006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:13.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:13.928+0000 7fa8017fa640 1 -- 172.21.15.103:0/1830638352 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa7f40055a0 con 0x7fa8041006c0 2024-08-19T01:27:14.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:14.099+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}) -- 0x7fa804102c20 con 0x7fa7f0039060 2024-08-19T01:27:15.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:14 smithi103 ceph-mon[18436]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:16.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:15 smithi103 ceph-mon[18436]: Deploying cephadm binary to smithi146 2024-08-19T01:27:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.238+0000 7fa8017fa640 1 -- 172.21.15.103:0/1830638352 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa804102c20 con 0x7fa7f0039060 2024-08-19T01:27:17.240 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi146' with addr '172.21.15.146' 2024-08-19T01:27:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa7f0039060 msgr2=0x7fa7f003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa7f0039060 0x7fa7f003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0098c0 tx=0x7fa7ec006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 msgr2=0x7fa804101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804101f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009230 tx=0x7fa7f40048f0 comp rx=0 tx=0).stop 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 shutdown_connections 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa7f0039060 0x7fa7f003b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 --2- 172.21.15.103:0/1830638352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8041006c0 0x7fa804101f30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 >> 172.21.15.103:0/1830638352 conn(0x7fa804075f80 msgr2=0x7fa804104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 shutdown_connections 2024-08-19T01:27:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:17.241+0000 7fa809ae6640 1 -- 172.21.15.103:0/1830638352 wait complete. 2024-08-19T01:27:17.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-19T01:27:18.109 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:18.489 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:18.489 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:18 smithi103 ceph-mon[18436]: Added host smithi146 2024-08-19T01:27:18.489 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:18.489 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:18.489 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:18.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 -- 172.21.15.103:0/1003251465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 msgr2=0x7f3d90100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:18.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 --2- 172.21.15.103:0/1003251465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90100aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009c20 tx=0x7f3d8002aa90 comp rx=0 tx=0).stop 2024-08-19T01:27:18.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 -- 172.21.15.103:0/1003251465 shutdown_connections 2024-08-19T01:27:18.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 --2- 172.21.15.103:0/1003251465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90100aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:18.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 -- 172.21.15.103:0/1003251465 >> 172.21.15.103:0/1003251465 conn(0x7f3d90075f80 msgr2=0x7f3d90076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:18.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 -- 172.21.15.103:0/1003251465 shutdown_connections 2024-08-19T01:27:18.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 -- 172.21.15.103:0/1003251465 wait complete. 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 Processor -- start 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.569+0000 7f3d95721640 1 -- start start 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d95721640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d95721640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010a780 con 0x7f3d901006c0 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d8effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49258/0 (socket says 172.21.15.103:49258) 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d8effd640 1 -- 172.21.15.103:0/401897567 learned_addr learned my addr 172.21.15.103:0/401897567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d8effd640 1 -- 172.21.15.103:0/401897567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9006a210 con 0x7f3d901006c0 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d8effd640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90195ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005e30 tx=0x7f3d8002b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d77fff640 1 -- 172.21.15.103:0/401897567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d80047070 con 0x7f3d901006c0 2024-08-19T01:27:18.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d90196000 con 0x7f3d901006c0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d77fff640 1 -- 172.21.15.103:0/401897567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d80003e20 con 0x7f3d901006c0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d77fff640 1 -- 172.21.15.103:0/401897567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d80033050 con 0x7f3d901006c0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.570+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d90196460 con 0x7f3d901006c0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.571+0000 7f3d77fff640 1 -- 172.21.15.103:0/401897567 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3d8004a070 con 0x7f3d901006c0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.571+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d90101970 con 0x7f3d901006c0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.571+0000 7f3d77fff640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3d78039060 0x7f3d7803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.571+0000 7f3d77fff640 1 -- 172.21.15.103:0/401897567 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d8003e030 con 0x7f3d901006c0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.572+0000 7f3d8e7fc640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3d78039060 0x7f3d7803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.572+0000 7f3d8e7fc640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3d78039060 0x7f3d7803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3d840098c0 tx=0x7f3d84006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:18.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.576+0000 7f3d77fff640 1 -- 172.21.15.103:0/401897567 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3d8002edf0 con 0x7f3d901006c0 2024-08-19T01:27:18.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.749+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d900699e0 con 0x7f3d78039060 2024-08-19T01:27:18.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.751+0000 7f3d77fff640 1 -- 172.21.15.103:0/401897567 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f3d900699e0 con 0x7f3d78039060 2024-08-19T01:27:18.752 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:18.752 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}] 2024-08-19T01:27:18.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3d78039060 msgr2=0x7f3d7803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:18.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3d78039060 0x7f3d7803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3d840098c0 tx=0x7f3d84006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:18.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 msgr2=0x7f3d90195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90195ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3d80005e30 tx=0x7f3d8002b010 comp rx=0 tx=0).stop 2024-08-19T01:27:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 shutdown_connections 2024-08-19T01:27:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3d78039060 0x7f3d7803b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 --2- 172.21.15.103:0/401897567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d901006c0 0x7f3d90195ac0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 >> 172.21.15.103:0/401897567 conn(0x7f3d90075f80 msgr2=0x7f3d90101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 shutdown_connections 2024-08-19T01:27:18.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:18.753+0000 7f3d95721640 1 -- 172.21.15.103:0/401897567 wait complete. 2024-08-19T01:27:19.381 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-08-19T01:27:19.382 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:27:19.382 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-08-19T01:27:19.401 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:27:19.401 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:19.459 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-08-19T01:27:19.459 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch host add smithi190 2024-08-19T01:27:19.704 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:19.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:19 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:19.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:19 smithi103 ceph-mon[18436]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:27:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.117+0000 7f79b659e640 1 -- 172.21.15.103:0/3998091496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 msgr2=0x7f79b0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:20.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.117+0000 7f79b659e640 1 --2- 172.21.15.103:0/3998091496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b0100830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009880 tx=0x7f79a002a5d0 comp rx=0 tx=0).stop 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.117+0000 7f79b659e640 1 -- 172.21.15.103:0/3998091496 shutdown_connections 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.117+0000 7f79b659e640 1 --2- 172.21.15.103:0/3998091496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b0100830 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.117+0000 7f79b659e640 1 -- 172.21.15.103:0/3998091496 >> 172.21.15.103:0/3998091496 conn(0x7f79b00fbed0 msgr2=0x7f79b00fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.117+0000 7f79b659e640 1 -- 172.21.15.103:0/3998091496 shutdown_connections 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.117+0000 7f79b659e640 1 -- 172.21.15.103:0/3998091496 wait complete. 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79b659e640 1 Processor -- start 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79b659e640 1 -- start start 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79b659e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79b659e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79b010a750 con 0x7f79b0100450 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49278/0 (socket says 172.21.15.103:49278) 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79affff640 1 -- 172.21.15.103:0/3722633773 learned_addr learned my addr 172.21.15.103:0/3722633773 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.118+0000 7f79affff640 1 -- 172.21.15.103:0/3722633773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b019be50 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.119+0000 7f79affff640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b019b910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009230 tx=0x7f79a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.119+0000 7f79ad7fa640 1 -- 172.21.15.103:0/3722633773 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79a003f070 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.119+0000 7f79ad7fa640 1 -- 172.21.15.103:0/3722633773 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79a0004d10 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.119+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b019c140 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.119+0000 7f79ad7fa640 1 -- 172.21.15.103:0/3722633773 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79a0035030 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.119+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b0195ad0 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.120+0000 7f79ad7fa640 1 -- 172.21.15.103:0/3722633773 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f79a0042070 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.120+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b0101930 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.120+0000 7f79ad7fa640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7998039060 0x7f799803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.120+0000 7f79ad7fa640 1 -- 172.21.15.103:0/3722633773 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f79a006ccd0 con 0x7f79b0100450 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.120+0000 7f79af7fe640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7998039060 0x7f799803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:20.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.120+0000 7f79af7fe640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7998039060 0x7f799803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f79a40098c0 tx=0x7f79a4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.125+0000 7f79ad7fa640 1 -- 172.21.15.103:0/3722633773 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f79a0042c60 con 0x7f79b0100450 2024-08-19T01:27:20.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:20.294+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}) -- 0x7f79b0196fc0 con 0x7f7998039060 2024-08-19T01:27:22.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:22.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:22.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:22.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:22.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:22.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:22.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:22.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:22.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:23.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:22 smithi103 ceph-mon[18436]: Deploying cephadm binary to smithi190 2024-08-19T01:27:23.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:22 smithi103 ceph-mon[18436]: Updating smithi146:/etc/ceph/ceph.conf 2024-08-19T01:27:23.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:22 smithi103 ceph-mon[18436]: Updating smithi146:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:27:23.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.477+0000 7f79ad7fa640 1 -- 172.21.15.103:0/3722633773 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f79b0196fc0 con 0x7f7998039060 2024-08-19T01:27:23.481 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-08-19T01:27:23.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.479+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7998039060 msgr2=0x7f799803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:23.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.479+0000 7f79b659e640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7998039060 0x7f799803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f79a40098c0 tx=0x7f79a4006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:23.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.479+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 msgr2=0x7f79b019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:23.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.479+0000 7f79b659e640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b019b910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009230 tx=0x7f79a00048f0 comp rx=0 tx=0).stop 2024-08-19T01:27:23.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.480+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 shutdown_connections 2024-08-19T01:27:23.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.480+0000 7f79b659e640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7998039060 0x7f799803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:23.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.480+0000 7f79b659e640 1 --2- 172.21.15.103:0/3722633773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b0100450 0x7f79b019b910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:23.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.480+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 >> 172.21.15.103:0/3722633773 conn(0x7f79b00fbed0 msgr2=0x7f79b00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:23.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.480+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 shutdown_connections 2024-08-19T01:27:23.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:23.480+0000 7f79b659e640 1 -- 172.21.15.103:0/3722633773 wait complete. 2024-08-19T01:27:23.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:23 smithi103 ceph-mon[18436]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:23.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:23 smithi103 ceph-mon[18436]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:23.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:23.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:23.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:23.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:23.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:24.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-19T01:27:24.290 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:24.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.741+0000 7f489ea3e640 1 -- 172.21.15.103:0/2062582914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 msgr2=0x7f4898101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:24.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.741+0000 7f489ea3e640 1 --2- 172.21.15.103:0/2062582914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898101940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4888009880 tx=0x7f488802a5d0 comp rx=0 tx=0).stop 2024-08-19T01:27:24.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.741+0000 7f489ea3e640 1 -- 172.21.15.103:0/2062582914 shutdown_connections 2024-08-19T01:27:24.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.741+0000 7f489ea3e640 1 --2- 172.21.15.103:0/2062582914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898101940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:24.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.741+0000 7f489ea3e640 1 -- 172.21.15.103:0/2062582914 >> 172.21.15.103:0/2062582914 conn(0x7f48980fcfe0 msgr2=0x7f48980ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:24.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.741+0000 7f489ea3e640 1 -- 172.21.15.103:0/2062582914 shutdown_connections 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.741+0000 7f489ea3e640 1 -- 172.21.15.103:0/2062582914 wait complete. 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.742+0000 7f489ea3e640 1 Processor -- start 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.742+0000 7f489ea3e640 1 -- start start 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.742+0000 7f489ea3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.742+0000 7f489ea3e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f489810b860 con 0x7f4898101560 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f4897fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f4897fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35442/0 (socket says 172.21.15.103:35442) 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f4897fff640 1 -- 172.21.15.103:0/3132217856 learned_addr learned my addr 172.21.15.103:0/3132217856 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f4897fff640 1 -- 172.21.15.103:0/3132217856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48981972f0 con 0x7f4898101560 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f4897fff640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898196db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4888009230 tx=0x7f48880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f48957fa640 1 -- 172.21.15.103:0/3132217856 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f488803f070 con 0x7f4898101560 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f48957fa640 1 -- 172.21.15.103:0/3132217856 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4888004d10 con 0x7f4898101560 2024-08-19T01:27:24.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f48957fa640 1 -- 172.21.15.103:0/3132217856 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4888035030 con 0x7f4898101560 2024-08-19T01:27:24.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48981975e0 con 0x7f4898101560 2024-08-19T01:27:24.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.743+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489819b680 con 0x7f4898101560 2024-08-19T01:27:24.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.744+0000 7f48957fa640 1 -- 172.21.15.103:0/3132217856 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4888042070 con 0x7f4898101560 2024-08-19T01:27:24.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.744+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4898102a40 con 0x7f4898101560 2024-08-19T01:27:24.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.745+0000 7f48957fa640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4880039060 0x7f488003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:24.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.745+0000 7f48957fa640 1 -- 172.21.15.103:0/3132217856 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f488806cc70 con 0x7f4898101560 2024-08-19T01:27:24.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.745+0000 7f48977fe640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4880039060 0x7f488003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:24.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.745+0000 7f48977fe640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4880039060 0x7f488003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098c0 tx=0x7f488c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:24.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.749+0000 7f48957fa640 1 -- 172.21.15.103:0/3132217856 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4888042360 con 0x7f4898101560 2024-08-19T01:27:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.921+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4898198110 con 0x7f4880039060 2024-08-19T01:27:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.923+0000 7f48957fa640 1 -- 172.21.15.103:0/3132217856 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f4898198110 con 0x7f4880039060 2024-08-19T01:27:24.925 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:24.925 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-08-19T01:27:24.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4880039060 msgr2=0x7f488003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:24.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4880039060 0x7f488003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098c0 tx=0x7f488c006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:24.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 msgr2=0x7f4898196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:24.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898196db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4888009230 tx=0x7f48880048f0 comp rx=0 tx=0).stop 2024-08-19T01:27:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 shutdown_connections 2024-08-19T01:27:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4880039060 0x7f488003b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 --2- 172.21.15.103:0/3132217856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4898101560 0x7f4898196db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 >> 172.21.15.103:0/3132217856 conn(0x7f48980fcfe0 msgr2=0x7f48980fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 shutdown_connections 2024-08-19T01:27:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:24.925+0000 7f489ea3e640 1 -- 172.21.15.103:0/3132217856 wait complete. 2024-08-19T01:27:25.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:24 smithi103 ceph-mon[18436]: Updating smithi146:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:27:25.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:24 smithi103 ceph-mon[18436]: Added host smithi190 2024-08-19T01:27:25.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:24 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:25.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:24 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:25.552 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-19T01:27:25.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd crush tunables default 2024-08-19T01:27:25.795 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:26.162 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:25 smithi103 ceph-mon[18436]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:26.162 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:25 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:26.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.240+0000 7f50c6bcb640 1 -- 172.21.15.103:0/2362461272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 msgr2=0x7f50c0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:26.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.240+0000 7f50c6bcb640 1 --2- 172.21.15.103:0/2362461272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0100830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009880 tx=0x7f50bc02a5d0 comp rx=0 tx=0).stop 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.240+0000 7f50c6bcb640 1 -- 172.21.15.103:0/2362461272 shutdown_connections 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.240+0000 7f50c6bcb640 1 --2- 172.21.15.103:0/2362461272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0100830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.240+0000 7f50c6bcb640 1 -- 172.21.15.103:0/2362461272 >> 172.21.15.103:0/2362461272 conn(0x7f50c00fbed0 msgr2=0x7f50c00fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.240+0000 7f50c6bcb640 1 -- 172.21.15.103:0/2362461272 shutdown_connections 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.240+0000 7f50c6bcb640 1 -- 172.21.15.103:0/2362461272 wait complete. 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c6bcb640 1 Processor -- start 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c6bcb640 1 -- start start 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c6bcb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c6bcb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50c010a750 con 0x7f50c0100450 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c4967640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:26.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c4967640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35462/0 (socket says 172.21.15.103:35462) 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c4967640 1 -- 172.21.15.103:0/577175322 learned_addr learned my addr 172.21.15.103:0/577175322 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c4967640 1 -- 172.21.15.103:0/577175322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c0198110 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50c4967640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0197bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009230 tx=0x7f50bc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.241+0000 7f50b5ffb640 1 -- 172.21.15.103:0/577175322 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50bc03f070 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.242+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c0198400 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.242+0000 7f50b5ffb640 1 -- 172.21.15.103:0/577175322 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50bc004d10 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.242+0000 7f50b5ffb640 1 -- 172.21.15.103:0/577175322 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50bc035030 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.242+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c01988c0 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.242+0000 7f50b5ffb640 1 -- 172.21.15.103:0/577175322 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f50bc042070 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.243+0000 7f50b5ffb640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50a8039060 0x7f50a803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.243+0000 7f50b5ffb640 1 -- 172.21.15.103:0/577175322 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50bc06ccb0 con 0x7f50c0100450 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.243+0000 7f50b7fff640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50a8039060 0x7f50a803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:26.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.243+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c0101930 con 0x7f50c0100450 2024-08-19T01:27:26.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.243+0000 7f50b7fff640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50a8039060 0x7f50a803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b00098c0 tx=0x7f50b0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:26.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.248+0000 7f50b5ffb640 1 -- 172.21.15.103:0/577175322 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f50bc0055c0 con 0x7f50c0100450 2024-08-19T01:27:26.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.422+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f50c01008b0 con 0x7f50c0100450 2024-08-19T01:27:26.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.829+0000 7f50b5ffb640 1 -- 172.21.15.103:0/577175322 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f50bc042c60 con 0x7f50c0100450 2024-08-19T01:27:26.831 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50a8039060 msgr2=0x7f50a803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50a8039060 0x7f50a803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b00098c0 tx=0x7f50b0006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 msgr2=0x7f50c0197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0197bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009230 tx=0x7f50bc0048f0 comp rx=0 tx=0).stop 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 shutdown_connections 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50a8039060 0x7f50a803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 --2- 172.21.15.103:0/577175322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50c0100450 0x7f50c0197bd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 >> 172.21.15.103:0/577175322 conn(0x7f50c00fbed0 msgr2=0x7f50c00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 shutdown_connections 2024-08-19T01:27:26.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:26.831+0000 7f50c6bcb640 1 -- 172.21.15.103:0/577175322 wait complete. 2024-08-19T01:27:27.164 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:26 smithi103 ceph-mon[18436]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:27:27.165 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:26 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/577175322' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-19T01:27:27.475 INFO:tasks.cephadm:Adding mon.a on smithi103 2024-08-19T01:27:27.476 INFO:tasks.cephadm:Adding mon.b on smithi146 2024-08-19T01:27:27.476 INFO:tasks.cephadm:Adding mon.c on smithi190 2024-08-19T01:27:27.476 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch apply mon '3;smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c' 2024-08-19T01:27:27.805 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-19T01:27:27.853 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-19T01:27:28.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:28.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/577175322' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-19T01:27:28.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:28.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:28.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:28.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:28.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:28.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:28.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:28.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:28.384 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.332+0000 7f78ef4c5640 1 -- 172.21.15.190:0/3352675066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 msgr2=0x7f78e81041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:28.384 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.332+0000 7f78ef4c5640 1 --2- 172.21.15.190:0/3352675066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e81041e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009880 tx=0x7f78e402a5d0 comp rx=0 tx=0).stop 2024-08-19T01:27:28.384 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.332+0000 7f78ef4c5640 1 -- 172.21.15.190:0/3352675066 shutdown_connections 2024-08-19T01:27:28.384 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.332+0000 7f78ef4c5640 1 --2- 172.21.15.190:0/3352675066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e81041e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:28.384 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.332+0000 7f78ef4c5640 1 -- 172.21.15.190:0/3352675066 >> 172.21.15.190:0/3352675066 conn(0x7f78e80fb9d0 msgr2=0x7f78e80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:28.384 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.332+0000 7f78ef4c5640 1 -- 172.21.15.190:0/3352675066 shutdown_connections 2024-08-19T01:27:28.384 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.332+0000 7f78ef4c5640 1 -- 172.21.15.190:0/3352675066 wait complete. 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ef4c5640 1 Processor -- start 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ef4c5640 1 -- start start 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ef4c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e8101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ef4c5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81072b0 con 0x7f78e8101df0 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ed261640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e8101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ed261640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e8101670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.190:58970/0 (socket says 172.21.15.190:58970) 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ed261640 1 -- 172.21.15.190:0/318956623 learned_addr learned my addr 172.21.15.190:0/318956623 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.333+0000 7f78ed261640 1 -- 172.21.15.190:0/318956623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e8101bb0 con 0x7f78e8101df0 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.334+0000 7f78ed261640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e8101670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009230 tx=0x7f78e40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.334+0000 7f78de7fc640 1 -- 172.21.15.190:0/318956623 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78e403f070 con 0x7f78e8101df0 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.334+0000 7f78ef4c5640 1 -- 172.21.15.190:0/318956623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e80ffda0 con 0x7f78e8101df0 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.334+0000 7f78de7fc640 1 -- 172.21.15.190:0/318956623 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e4003aa0 con 0x7f78e8101df0 2024-08-19T01:27:28.385 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.334+0000 7f78de7fc640 1 -- 172.21.15.190:0/318956623 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78e4035030 con 0x7f78e8101df0 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.334+0000 7f78ef4c5640 1 -- 172.21.15.190:0/318956623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81002e0 con 0x7f78e8101df0 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.335+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e8107a30 con 0x7f78e8101df0 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.336+0000 7f78de7fc640 1 -- 172.21.15.190:0/318956623 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f78e4042070 con 0x7f78e8101df0 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.336+0000 7f78de7fc640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f78d4039060 0x7f78d403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.336+0000 7f78de7fc640 1 -- 172.21.15.190:0/318956623 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f78e406d1e0 con 0x7f78e8101df0 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.337+0000 7f78eca60640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f78d4039060 0x7f78d403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.337+0000 7f78eca60640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f78d4039060 0x7f78d403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78d80098c0 tx=0x7f78d8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.340+0000 7f78de7fc640 1 -- 172.21.15.190:0/318956623 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f78e4042e20 con 0x7f78e8101df0 2024-08-19T01:27:28.529 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.527+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}) -- 0x7f78e81006e0 con 0x7f78d4039060 2024-08-19T01:27:28.534 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.532+0000 7f78de7fc640 1 -- 172.21.15.190:0/318956623 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f78e81006e0 con 0x7f78d4039060 2024-08-19T01:27:28.534 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f78d4039060 msgr2=0x7f78d403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f78d4039060 0x7f78d403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78d80098c0 tx=0x7f78d8006db0 comp rx=0 tx=0).stop 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 msgr2=0x7f78e8101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e8101670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009230 tx=0x7f78e40048f0 comp rx=0 tx=0).stop 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 shutdown_connections 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f78d4039060 0x7f78d403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 --2- 172.21.15.190:0/318956623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78e8101df0 0x7f78e8101670 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 >> 172.21.15.190:0/318956623 conn(0x7f78e80fb9d0 msgr2=0x7f78e80fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 shutdown_connections 2024-08-19T01:27:28.536 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:28.534+0000 7f78d3fff640 1 -- 172.21.15.190:0/318956623 wait complete. 2024-08-19T01:27:29.181 DEBUG:teuthology.orchestra.run.smithi146:mon.b> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.b.service 2024-08-19T01:27:29.184 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.c.service 2024-08-19T01:27:29.187 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:27:29.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:27:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-19T01:27:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: Updating smithi190:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:27:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:29 smithi103 ceph-mon[18436]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:29 smithi103 ceph-mon[18436]: Updating smithi190:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:27:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:29 smithi103 ceph-mon[18436]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:29 smithi103 ceph-mon[18436]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c;count:3 2024-08-19T01:27:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:29 smithi103 ceph-mon[18436]: Deploying daemon mon.c on smithi190 2024-08-19T01:27:30.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:29 smithi103 ceph-mon[18436]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:30.414 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:27:31.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.388+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/1671656156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 msgr2=0x7fc0a01066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:31.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.388+0000 7fc0a4ac3640 1 --2- 172.21.15.190:0/1671656156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a01066e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc098009880 tx=0x7fc09802a5d0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.388+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/1671656156 shutdown_connections 2024-08-19T01:27:31.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.388+0000 7fc0a4ac3640 1 --2- 172.21.15.190:0/1671656156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a01066e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.388+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/1671656156 >> 172.21.15.190:0/1671656156 conn(0x7fc0a0075f80 msgr2=0x7fc0a0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:31.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.388+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/1671656156 shutdown_connections 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.388+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/1671656156 wait complete. 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.389+0000 7fc0a4ac3640 1 Processor -- start 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.389+0000 7fc0a4ac3640 1 -- start start 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.389+0000 7fc0a4ac3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a0100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.389+0000 7fc0a4ac3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a010b8c0 con 0x7fc0a0106300 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.390+0000 7fc09e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a0100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.390+0000 7fc09e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a0100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.190:58988/0 (socket says 172.21.15.190:58988) 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.390+0000 7fc09e59c640 1 -- 172.21.15.190:0/2237747083 learned_addr learned my addr 172.21.15.190:0/2237747083 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.390+0000 7fc09e59c640 1 -- 172.21.15.190:0/2237747083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a0102420 con 0x7fc0a0106300 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.390+0000 7fc09e59c640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a0100530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc098009230 tx=0x7fc0980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.390+0000 7fc08f7fe640 1 -- 172.21.15.190:0/2237747083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc09803f070 con 0x7fc0a0106300 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.391+0000 7fc08f7fe640 1 -- 172.21.15.190:0/2237747083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc098004d10 con 0x7fc0a0106300 2024-08-19T01:27:31.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.391+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a0100a70 con 0x7fc0a0106300 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.391+0000 7fc08f7fe640 1 -- 172.21.15.190:0/2237747083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc098035030 con 0x7fc0a0106300 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.391+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a0100f30 con 0x7fc0a0106300 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.391+0000 7fc08f7fe640 1 -- 172.21.15.190:0/2237747083 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc098042070 con 0x7fc0a0106300 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.392+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a0102a60 con 0x7fc0a0106300 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.392+0000 7fc08f7fe640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc088039060 0x7fc08803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.392+0000 7fc08f7fe640 1 -- 172.21.15.190:0/2237747083 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc09806cd60 con 0x7fc0a0106300 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.392+0000 7fc09dd9b640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc088039060 0x7fc08803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.392+0000 7fc09dd9b640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc088039060 0x7fc08803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc09400ad30 tx=0x7fc0940093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:31.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.397+0000 7fc08f7fe640 1 -- 172.21.15.190:0/2237747083 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc0980372d0 con 0x7fc0a0106300 2024-08-19T01:27:31.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.642+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc0a0101a60 con 0x7fc0a0106300 2024-08-19T01:27:31.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.643+0000 7fc08f7fe640 1 -- 172.21.15.190:0/2237747083 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fc098003920 con 0x7fc0a0106300 2024-08-19T01:27:31.645 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:27:31.645 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:26:02.528201Z","created":"2024-08-19T01:26:02.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-19T01:27:31.645 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc088039060 msgr2=0x7fc08803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc088039060 0x7fc08803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc09400ad30 tx=0x7fc0940093f0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 msgr2=0x7fc0a0100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a0100530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc098009230 tx=0x7fc0980048f0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 shutdown_connections 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc088039060 0x7fc08803b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 --2- 172.21.15.190:0/2237747083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0a0106300 0x7fc0a0100530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 >> 172.21.15.190:0/2237747083 conn(0x7fc0a0075f80 msgr2=0x7fc0a0102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 shutdown_connections 2024-08-19T01:27:31.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:31.645+0000 7fc0a4ac3640 1 -- 172.21.15.190:0/2237747083 wait complete. 2024-08-19T01:27:32.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:31 smithi103 ceph-mon[18436]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:32.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:31 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/2237747083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:33.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 podman[22014]: 2024-08-19 01:27:33.131964131 +0000 UTC m=+0.017418553 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:27:33.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 podman[22014]: 2024-08-19 01:27:33.443232032 +0000 UTC m=+0.328686442 container init 2255ac56c6eb1d7ec9302d80064acc97af52f67022773bce45967bd70ca100eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-19T01:27:33.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 podman[22014]: 2024-08-19 01:27:33.446358001 +0000 UTC m=+0.331812411 container start 2255ac56c6eb1d7ec9302d80064acc97af52f67022773bce45967bd70ca100eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:27:33.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: set uid:gid to 167:167 (ceph:ceph) 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc), process ceph-mon, pid 2 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: pidfile_write: ignore empty --pid-file 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: load: jerasure load: lrc 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: RocksDB version: 7.9.2 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Git sha 0 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Compile date 2024-08-18 15:19:56 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: DB SUMMARY 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: DB Session ID: T0P7V0ACXPP5UJLWJ82F 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: CURRENT file: CURRENT 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: IDENTITY file: IDENTITY 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-08-19T01:27:33.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.error_if_exists: 0 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.create_if_missing: 0 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.paranoid_checks: 1 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.env: 0x558304c657a0 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.fs: PosixFileSystem 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.info_log: 0x558305d07500 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_file_opening_threads: 16 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.statistics: (nil) 2024-08-19T01:27:33.486 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.use_fsync: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_log_file_size: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.keep_log_file_num: 1000 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.recycle_log_file_num: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.allow_fallocate: 1 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.allow_mmap_reads: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.allow_mmap_writes: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.use_direct_reads: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.create_missing_column_families: 0 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.db_log_dir: 2024-08-19T01:27:33.487 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.wal_dir: 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.advise_random_on_open: 1 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.db_write_buffer_size: 0 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.write_buffer_manager: 0x558305d16dc0 2024-08-19T01:27:33.488 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.rate_limiter: (nil) 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.wal_recovery_mode: 2 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.enable_thread_tracking: 0 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.enable_pipelined_write: 0 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.unordered_write: 0 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-19T01:27:33.489 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.row_cache: None 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.wal_filter: None 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.allow_ingest_behind: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.two_write_queues: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.manual_wal_flush: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.wal_compression: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.atomic_flush: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-19T01:27:33.490 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-19T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.log_readahead_size: 0 2024-08-19T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-19T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.best_efforts_recovery: 0 2024-08-19T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-19T01:27:33.491 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-19T01:27:33.492 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.allow_data_in_errors: 0 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.db_host_id: __hostname__ 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_background_jobs: 2 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_background_compactions: -1 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_subcompactions: 1 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_total_wal_size: 0 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-19T01:27:33.493 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_open_files: -1 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bytes_per_sync: 0 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_readahead_size: 0 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_background_flushes: -1 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Compression algorithms supported: 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kZSTD supported: 0 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kXpressCompression supported: 0 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kBZip2Compression supported: 0 2024-08-19T01:27:33.494 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kLZ4Compression supported: 1 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kZlibCompression supported: 1 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kLZ4HCCompression supported: 1 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: kSnappyCompression supported: 1 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.merge_operator: 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_filter: None 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_filter_factory: None 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.sst_partitioner_factory: None 2024-08-19T01:27:33.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558305d07120) 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks: 1 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_top_level_index_and_filter: 1 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_type: 0 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_index_type: 0 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_shortening: 1 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: checksum: 4 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: no_block_cache: 0 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache: 0x558305cfb090 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_name: BinnedLRUCache 2024-08-19T01:27:33.496 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_options: 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: capacity : 536870912 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_shard_bits : 4 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: strict_capacity_limit : 0 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: high_pri_pool_ratio: 0.000 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_compressed: (nil) 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: persistent_cache: (nil) 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size: 4096 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size_deviation: 10 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_restart_interval: 16 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_block_restart_interval: 1 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: metadata_block_size: 4096 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: partition_filters: 0 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: use_delta_encoding: 1 2024-08-19T01:27:33.497 INFO:journalctl@ceph.mon.c.smithi190.stdout: filter_policy: bloomfilter 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: whole_key_filtering: 1 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: verify_compression: 0 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: read_amp_bytes_per_bit: 0 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: format_version: 5 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_index_compression: 1 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_align: 0 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: max_auto_readahead_size: 262144 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: prepopulate_block_cache: 0 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: initial_auto_readahead_size: 8192 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.write_buffer_size: 33554432 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_write_buffer_number: 2 2024-08-19T01:27:33.498 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression: NoCompression 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression: Disabled 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.prefix_extractor: nullptr 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.num_levels: 7 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-19T01:27:33.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.level: 32767 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.strategy: 0 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.enabled: false 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-19T01:27:33.500 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.target_file_size_base: 67108864 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-19T01:27:33.501 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.arena_block_size: 1048576 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.disable_auto_compactions: 0 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-19T01:27:33.502 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.inplace_update_support: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.bloom_locality: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.max_successive_merges: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.paranoid_file_checks: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.force_consistency_checks: 1 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.report_bg_io_stats: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.ttl: 2592000 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-19T01:27:33.503 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.enable_blob_files: false 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.min_blob_size: 0 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.blob_file_size: 268435456 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.blob_file_starting_level: 0 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-19T01:27:33.504 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ade04de8-d2d4-4da2-a04b-9f1bedfd45c3 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030853473786, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030853474297, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724030853, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ade04de8-d2d4-4da2-a04b-9f1bedfd45c3", "db_session_id": "T0P7V0ACXPP5UJLWJ82F", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724030853474415, "job": 1, "event": "recovery_finished"} 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558305d28e00 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: DB pointer 0x558305e22000 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** DB Stats ** 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-19T01:27:33.505 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-19T01:27:33.506 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-19T01:27:33.507 INFO:journalctl@ceph.mon.c.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache BinnedLRUCache@0x558305cfb090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: using public_addr v2:172.21.15.190:0/0 -> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: starting mon.c rank -1 at public addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] at bind addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:33.508 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(???) e0 preinit fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:33.710 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:27:33.710 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:27:33.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).mds e1 new map 2024-08-19T01:27:33.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-19T01:27:33.734 INFO:journalctl@ceph.mon.c.smithi190.stdout: e1 2024-08-19T01:27:33.734 INFO:journalctl@ceph.mon.c.smithi190.stdout: btime 2024-08-19T01:26:05:629511+0000 2024-08-19T01:27:33.734 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: legacy client fscid: -1 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout: No filesystems configured 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:27:33.735 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/577175322' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/577175322' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:33.736 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: Updating smithi190:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:33.737 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: Updating smithi190:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c;count:3 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: Deploying daemon mon.c on smithi190 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/2237747083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 ceph-mon[22032]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 bash[22014]: 2255ac56c6eb1d7ec9302d80064acc97af52f67022773bce45967bd70ca100eb 2024-08-19T01:27:33.738 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:33 smithi190 systemd[1]: Started Ceph mon.c for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:27:33.956 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:27:38.801 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.797+0000 7f0601127640 1 -- 172.21.15.190:0/3470040976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 msgr2=0x7f05e40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.797+0000 7f0601127640 1 --2- 172.21.15.190:0/3470040976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 0x7f05e40059c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05fc06a790 tx=0x7f05f002bfe0 comp rx=0 tx=0).stop 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.797+0000 7f0601127640 1 -- 172.21.15.190:0/3470040976 shutdown_connections 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.797+0000 7f0601127640 1 --2- 172.21.15.190:0/3470040976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 0x7f05e40059c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.797+0000 7f0601127640 1 -- 172.21.15.190:0/3470040976 >> 172.21.15.190:0/3470040976 conn(0x7f05fc0fbf50 msgr2=0x7f05fc0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.797+0000 7f0601127640 1 -- 172.21.15.190:0/3470040976 shutdown_connections 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.797+0000 7f0601127640 1 -- 172.21.15.190:0/3470040976 wait complete. 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.798+0000 7f0601127640 1 Processor -- start 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.798+0000 7f0601127640 1 -- start start 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f0601127640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 0x7f05fc196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f0601127640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 0x7f05fc19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:38.802 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f0601127640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc109360 con 0x7f05e4003530 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f0601127640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1091e0 con 0x7f05fc197490 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fa59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 0x7f05fc19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 0x7f05fc196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fa59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 0x7f05fc19b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47844/0 (socket says 172.21.15.190:47844) 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fa59c640 1 -- 172.21.15.190:0/2934252356 learned_addr learned my addr 172.21.15.190:0/2934252356 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fa59c640 1 -- 172.21.15.190:0/2934252356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 msgr2=0x7f05fc19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fa59c640 1 -- 172.21.15.190:0/2934252356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 msgr2=0x7f05fc19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fa59c640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 0x7f05fc19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fa59c640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 0x7f05fc19b820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fad9d640 1 -- 172.21.15.190:0/2934252356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 msgr2=0x7f05fc19b820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fad9d640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 0x7f05fc19b820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fad9d640 1 -- 172.21.15.190:0/2934252356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc197ab0 con 0x7f05e4003530 2024-08-19T01:27:38.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.799+0000 7f05fad9d640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 0x7f05fc196f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f05f0005cc0 tx=0x7f05f0004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:38.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.800+0000 7f05e3fff640 1 -- 172.21.15.190:0/2934252356 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f05f00348a0 con 0x7f05e4003530 2024-08-19T01:27:38.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.800+0000 7f05e3fff640 1 -- 172.21.15.190:0/2934252356 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f00051d0 con 0x7f05e4003530 2024-08-19T01:27:38.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.800+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc19bd60 con 0x7f05e4003530 2024-08-19T01:27:38.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.800+0000 7f05e3fff640 1 -- 172.21.15.190:0/2934252356 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f05f003d680 con 0x7f05e4003530 2024-08-19T01:27:38.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.800+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc19c2a0 con 0x7f05e4003530 2024-08-19T01:27:38.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.800+0000 7f05e3fff640 1 -- 172.21.15.190:0/2934252356 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f05f0044030 con 0x7f05e4003530 2024-08-19T01:27:38.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.801+0000 7f05e3fff640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f05e8038e50 0x7f05e803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:38.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.801+0000 7f05e3fff640 1 -- 172.21.15.190:0/2934252356 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f05f0042070 con 0x7f05e4003530 2024-08-19T01:27:38.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.801+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc06aa40 con 0x7f05e4003530 2024-08-19T01:27:38.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.801+0000 7f05fa59c640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f05e8038e50 0x7f05e803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:38.805 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.801+0000 7f05fa59c640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f05e8038e50 0x7f05e803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009100 tx=0x7f05ec008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:38.807 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:38.806+0000 7f05e3fff640 1 -- 172.21.15.190:0/2934252356 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f05f004b060 con 0x7f05e4003530 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: mon.a calling monitor election 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: mon.c calling monitor election 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.873 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: monmap epoch 2 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: last_changed 2024-08-19T01:27:33.503300+0000 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: min_mon_release 19 (squid) 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: election_strategy: 1 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: fsmap 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: mgrmap e13: x(active, since 37s) 2024-08-19T01:27:38.874 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: overall HEALTH_OK 2024-08-19T01:27:38.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:38.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:38.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:38.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:38.875 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:38.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:27:38.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: mon.a calling monitor election 2024-08-19T01:27:38.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: mon.c calling monitor election 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: monmap epoch 2 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: last_changed 2024-08-19T01:27:33.503300+0000 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: min_mon_release 19 (squid) 2024-08-19T01:27:38.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: election_strategy: 1 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: fsmap 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: mgrmap e13: x(active, since 37s) 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: overall HEALTH_OK 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:38.883 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:39.042 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.040+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f05fc198340 con 0x7f05e4003530 2024-08-19T01:27:39.043 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.041+0000 7f05e3fff640 1 -- 172.21.15.190:0/2934252356 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f05fc198340 con 0x7f05e4003530 2024-08-19T01:27:39.043 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:27:39.043 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:27:33.503300Z","created":"2024-08-19T01:26:02.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-19T01:27:39.043 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f05e8038e50 msgr2=0x7f05e803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f05e8038e50 0x7f05e803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009100 tx=0x7f05ec008040 comp rx=0 tx=0).stop 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 msgr2=0x7f05fc196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 0x7f05fc196f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f05f0005cc0 tx=0x7f05f0004c10 comp rx=0 tx=0).stop 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 shutdown_connections 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f05e8038e50 0x7f05e803b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc197490 0x7f05fc19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 --2- 172.21.15.190:0/2934252356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05e4003530 0x7f05fc196f50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:39.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 >> 172.21.15.190:0/2934252356 conn(0x7f05fc0fbf50 msgr2=0x7f05fc1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:39.046 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 shutdown_connections 2024-08-19T01:27:39.046 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:39.043+0000 7f0601127640 1 -- 172.21.15.190:0/2934252356 wait complete. 2024-08-19T01:27:39.609 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:39 smithi190 ceph-mon[22032]: Deploying daemon mon.b on smithi146 2024-08-19T01:27:39.609 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:39 smithi190 ceph-mon[22032]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:39.609 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:39 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/2934252356' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:39.609 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:39 smithi103 ceph-mon[18436]: Deploying daemon mon.b on smithi146 2024-08-19T01:27:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:39 smithi103 ceph-mon[18436]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:39 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/2934252356' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:40.640 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:27:40.640 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:27:40.898 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.323+0000 7fcc9315a640 1 -- 172.21.15.190:0/2560578141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 msgr2=0x7fcc8c102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.323+0000 7fcc9315a640 1 --2- 172.21.15.190:0/2560578141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 0x7fcc8c102930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009880 tx=0x7fcc7402a670 comp rx=0 tx=0).stop 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.323+0000 7fcc9315a640 1 -- 172.21.15.190:0/2560578141 shutdown_connections 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.323+0000 7fcc9315a640 1 --2- 172.21.15.190:0/2560578141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 0x7fcc8c102930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.323+0000 7fcc9315a640 1 -- 172.21.15.190:0/2560578141 >> 172.21.15.190:0/2560578141 conn(0x7fcc8c0fbf50 msgr2=0x7fcc8c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.323+0000 7fcc9315a640 1 -- 172.21.15.190:0/2560578141 shutdown_connections 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.323+0000 7fcc9315a640 1 -- 172.21.15.190:0/2560578141 wait complete. 2024-08-19T01:27:41.328 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.324+0000 7fcc9315a640 1 Processor -- start 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.324+0000 7fcc9315a640 1 -- start start 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc9315a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 0x7fcc8c197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc9315a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 0x7fcc8c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc9315a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c10b950 con 0x7fcc8c102550 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc9315a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c10b7d0 con 0x7fcc8c197570 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc8bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 0x7fcc8c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc8bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 0x7fcc8c19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47860/0 (socket says 172.21.15.190:47860) 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc8bfff640 1 -- 172.21.15.190:0/2605848862 learned_addr learned my addr 172.21.15.190:0/2605848862 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc90ef6640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 0x7fcc8c197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc8bfff640 1 -- 172.21.15.190:0/2605848862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 msgr2=0x7fcc8c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc8bfff640 1 -- 172.21.15.190:0/2605848862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 msgr2=0x7fcc8c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc8bfff640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 0x7fcc8c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc8bfff640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 0x7fcc8c19b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:27:41.329 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc90ef6640 1 -- 172.21.15.190:0/2605848862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 msgr2=0x7fcc8c19b900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc90ef6640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 0x7fcc8c19b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.325+0000 7fcc90ef6640 1 -- 172.21.15.190:0/2605848862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c197b90 con 0x7fcc8c102550 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.326+0000 7fcc90ef6640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 0x7fcc8c197030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc74000c00 tx=0x7fcc74004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.326+0000 7fcc89ffb640 1 -- 172.21.15.190:0/2605848862 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcc74035030 con 0x7fcc8c102550 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.326+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c19be40 con 0x7fcc8c102550 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.326+0000 7fcc89ffb640 1 -- 172.21.15.190:0/2605848862 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc740053a0 con 0x7fcc8c102550 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.326+0000 7fcc89ffb640 1 -- 172.21.15.190:0/2605848862 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcc74031830 con 0x7fcc8c102550 2024-08-19T01:27:41.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.326+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c19c320 con 0x7fcc8c102550 2024-08-19T01:27:41.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.327+0000 7fcc89ffb640 1 -- 172.21.15.190:0/2605848862 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcc74042070 con 0x7fcc8c102550 2024-08-19T01:27:41.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.327+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8c06aa40 con 0x7fcc8c102550 2024-08-19T01:27:41.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.327+0000 7fcc89ffb640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc78038e00 0x7fcc7803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:41.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.327+0000 7fcc89ffb640 1 -- 172.21.15.190:0/2605848862 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcc7406ccd0 con 0x7fcc8c102550 2024-08-19T01:27:41.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.328+0000 7fcc8bfff640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc78038e00 0x7fcc7803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:41.331 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.328+0000 7fcc8bfff640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc78038e00 0x7fcc7803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009100 tx=0x7fcc7c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:41.334 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.333+0000 7fcc89ffb640 1 -- 172.21.15.190:0/2605848862 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcc74039350 con 0x7fcc8c102550 2024-08-19T01:27:41.566 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.564+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcc8c19c5d0 con 0x7fcc8c102550 2024-08-19T01:27:41.567 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.565+0000 7fcc89ffb640 1 -- 172.21.15.190:0/2605848862 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fcc74005550 con 0x7fcc8c102550 2024-08-19T01:27:41.567 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:27:41.568 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:27:33.503300Z","created":"2024-08-19T01:26:02.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-19T01:27:41.568 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-08-19T01:27:41.569 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc78038e00 msgr2=0x7fcc7803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:41.569 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc78038e00 0x7fcc7803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009100 tx=0x7fcc7c008040 comp rx=0 tx=0).stop 2024-08-19T01:27:41.570 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 msgr2=0x7fcc8c197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:41.570 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 0x7fcc8c197030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc74000c00 tx=0x7fcc74004e40 comp rx=0 tx=0).stop 2024-08-19T01:27:41.570 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 shutdown_connections 2024-08-19T01:27:41.570 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc78038e00 0x7fcc7803b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:41.571 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc8c197570 0x7fcc8c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:41.571 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 --2- 172.21.15.190:0/2605848862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc8c102550 0x7fcc8c197030 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:41.571 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 >> 172.21.15.190:0/2605848862 conn(0x7fcc8c0fbf50 msgr2=0x7fcc8c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:41.571 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 shutdown_connections 2024-08-19T01:27:41.571 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:41.567+0000 7fcc9315a640 1 -- 172.21.15.190:0/2605848862 wait complete. 2024-08-19T01:27:41.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:41 smithi190 ceph-mon[22032]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:41.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:41 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/2605848862' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:42.063 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:41 smithi103 ceph-mon[18436]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.063 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:41 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/2605848862' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:42.139 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-19T01:27:42.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-19T01:27:42.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-19T01:27:42.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-19T01:27:42.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-19T01:27:42.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/577175322' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/577175322' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: Updating smithi190:/etc/ceph/ceph.conf 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: Updating smithi190:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:42.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: Updating smithi190:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c;count:3 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: Deploying daemon mon.c on smithi190 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/2237747083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.a calling monitor election 2024-08-19T01:27:42.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.c calling monitor election 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: monmap epoch 2 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: last_changed 2024-08-19T01:27:33.503300+0000 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: min_mon_release 19 (squid) 2024-08-19T01:27:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: election_strategy: 1 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: fsmap 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mgrmap e13: x(active, since 37s) 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: overall HEALTH_OK 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: Deploying daemon mon.b on smithi146 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/2934252356' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:42.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:42.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:42.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/2605848862' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:42.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 ceph-mon[22098]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-19T01:27:42.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 bash[22078]: 1bfba6914e0533316429ae5cfa778a08db61b1fd2862c31619f2ef36070faead 2024-08-19T01:27:42.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:42 smithi146 systemd[1]: Started Ceph mon.b for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:27:43.187 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-19T01:27:43.187 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-19T01:27:43.437 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:27:47.461 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.458+0000 7fa361855640 1 -- 172.21.15.190:0/2689465119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa348003530 msgr2=0x7fa3480059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:47.461 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.458+0000 7fa361855640 1 --2- 172.21.15.190:0/2689465119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa348003530 0x7fa3480059c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa35c101750 tx=0x7fa35402bdd0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.461 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.458+0000 7fa361855640 1 -- 172.21.15.190:0/2689465119 shutdown_connections 2024-08-19T01:27:47.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.458+0000 7fa361855640 1 --2- 172.21.15.190:0/2689465119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa348003530 0x7fa3480059c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.458+0000 7fa361855640 1 -- 172.21.15.190:0/2689465119 >> 172.21.15.190:0/2689465119 conn(0x7fa35c0fbf80 msgr2=0x7fa35c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:47.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.458+0000 7fa361855640 1 -- 172.21.15.190:0/2689465119 shutdown_connections 2024-08-19T01:27:47.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.458+0000 7fa361855640 1 -- 172.21.15.190:0/2689465119 wait complete. 2024-08-19T01:27:47.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.459+0000 7fa361855640 1 Processor -- start 2024-08-19T01:27:47.462 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:27:47.462 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: mon.a calling monitor election 2024-08-19T01:27:47.462 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: mon.c calling monitor election 2024-08-19T01:27:47.462 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: monmap epoch 3 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:47.463 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: last_changed 2024-08-19T01:27:42.143560+0000 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: min_mon_release 19 (squid) 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: election_strategy: 1 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: fsmap 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: mgrmap e13: x(active, since 46s) 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: overall HEALTH_OK 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.464 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.459+0000 7fa361855640 1 -- start start 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa361855640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 0x7fa35c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa361855640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c1983a0 0x7fa35c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa361855640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 0x7fa35c1a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa361855640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c108230 con 0x7fa35c1983a0 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa361855640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1080b0 con 0x7fa348003530 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa361855640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1083b0 con 0x7fa35c1a0ed0 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c1983a0 0x7fa35c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:47.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 0x7fa35c1a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c1983a0 0x7fa35c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.190:44538/0 (socket says 172.21.15.190:44538) 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 -- 172.21.15.190:0/3925839523 learned_addr learned my addr 172.21.15.190:0/3925839523 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35affd640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 0x7fa35c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35affd640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 msgr2=0x7fa35c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35affd640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 msgr2=0x7fa35c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35affd640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 0x7fa35c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35b7fe640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 msgr2=0x7fa35c1a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35b7fe640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 msgr2=0x7fa35c1a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35b7fe640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 0x7fa35c1a32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35affd640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 0x7fa35c197e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35b7fe640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 0x7fa35c1a32c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 msgr2=0x7fa35c1a32c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:27:47.468 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 0x7fa35c1a32c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 msgr2=0x7fa35c197e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 0x7fa35c197e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 -- 172.21.15.190:0/3925839523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c1a3b10 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.460+0000 7fa35a7fc640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c1983a0 0x7fa35c198800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa34400b520 tx=0x7fa34400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.461+0000 7fa360853640 1 -- 172.21.15.190:0/3925839523 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344004230 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.461+0000 7fa360853640 1 -- 172.21.15.190:0/3925839523 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3440043d0 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.461+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c195cb0 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.461+0000 7fa360853640 1 -- 172.21.15.190:0/3925839523 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344003cf0 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.461+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c196240 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.462+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c109360 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.462+0000 7fa360853640 1 -- 172.21.15.190:0/3925839523 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa34400e070 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.462+0000 7fa360853640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa33c0394f0 0x7fa33c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.462+0000 7fa360853640 1 -- 172.21.15.190:0/3925839523 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa34404b940 con 0x7fa35c1983a0 2024-08-19T01:27:47.469 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.462+0000 7fa35affd640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa33c0394f0 0x7fa33c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:47.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.463+0000 7fa35affd640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa33c0394f0 0x7fa33c03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa3540093b0 tx=0x7fa35402e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:47.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.467+0000 7fa360853640 1 -- 172.21.15.190:0/3925839523 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa344014020 con 0x7fa35c1983a0 2024-08-19T01:27:47.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: mon.a calling monitor election 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: mon.c calling monitor election 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:47.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: monmap epoch 3 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: last_changed 2024-08-19T01:27:42.143560+0000 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: min_mon_release 19 (squid) 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: election_strategy: 1 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: fsmap 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: mgrmap e13: x(active, since 46s) 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: overall HEALTH_OK 2024-08-19T01:27:47.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:47.720 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.718+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa35c18f400 con 0x7fa35c1983a0 2024-08-19T01:27:47.721 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.719+0000 7fa360853640 1 -- 172.21.15.190:0/3925839523 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fa344020350 con 0x7fa35c1983a0 2024-08-19T01:27:47.721 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:27:47.721 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","modified":"2024-08-19T01:27:42.143560Z","created":"2024-08-19T01:26:02.528201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-19T01:27:47.721 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa33c0394f0 msgr2=0x7fa33c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa33c0394f0 0x7fa33c03b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa3540093b0 tx=0x7fa35402e040 comp rx=0 tx=0).stop 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c1983a0 msgr2=0x7fa35c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c1983a0 0x7fa35c198800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa34400b520 tx=0x7fa34400b9e0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 shutdown_connections 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa33c0394f0 0x7fa33c03b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa35c1a0ed0 0x7fa35c1a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa35c1983a0 0x7fa35c198800 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 --2- 172.21.15.190:0/3925839523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa348003530 0x7fa35c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:47.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 >> 172.21.15.190:0/3925839523 conn(0x7fa35c0fbf80 msgr2=0x7fa35c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:47.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 shutdown_connections 2024-08-19T01:27:47.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:47.721+0000 7fa361855640 1 -- 172.21.15.190:0/3925839523 wait complete. 2024-08-19T01:27:48.361 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:48 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:48.362 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:48 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/3925839523' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:48.362 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:48 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:48.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:48 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:48.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:48 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/3925839523' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:48.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:48 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.071 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-19T01:27:49.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph config generate-minimal-conf 2024-08-19T01:27:49.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:27:49.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.a calling monitor election 2024-08-19T01:27:49.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.c calling monitor election 2024-08-19T01:27:49.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: monmap epoch 3 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: last_changed 2024-08-19T01:27:42.143560+0000 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:49.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: min_mon_release 19 (squid) 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: election_strategy: 1 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: fsmap 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mgrmap e13: x(active, since 46s) 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: overall HEALTH_OK 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:49.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:49.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/3925839523' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-19T01:27:49.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:49.892 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:50.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.b calling monitor election 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: Reconfiguring mon.a (unknown last config time)... 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: Reconfiguring daemon mon.a on smithi103 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.b calling monitor election 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.c calling monitor election 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.a calling monitor election 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: monmap epoch 3 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: last_changed 2024-08-19T01:27:42.143560+0000 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:50.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: min_mon_release 19 (squid) 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: election_strategy: 1 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: fsmap 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: mgrmap e13: x(active, since 48s) 2024-08-19T01:27:50.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:49 smithi146 ceph-mon[22098]: overall HEALTH_OK 2024-08-19T01:27:50.247 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: mon.b calling monitor election 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: Reconfiguring mon.a (unknown last config time)... 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: Reconfiguring daemon mon.a on smithi103 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: mon.b calling monitor election 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: mon.c calling monitor election 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: mon.a calling monitor election 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: monmap epoch 3 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: last_changed 2024-08-19T01:27:42.143560+0000 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: min_mon_release 19 (squid) 2024-08-19T01:27:50.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: election_strategy: 1 2024-08-19T01:27:50.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:50.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:50.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-08-19T01:27:50.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: fsmap 2024-08-19T01:27:50.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:50.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: mgrmap e13: x(active, since 48s) 2024-08-19T01:27:50.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:49 smithi103 ceph-mon[18436]: overall HEALTH_OK 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: mon.b calling monitor election 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: Reconfiguring mon.a (unknown last config time)... 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: Reconfiguring daemon mon.a on smithi103 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: mon.b calling monitor election 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: mon.c calling monitor election 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: mon.a calling monitor election 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-19T01:27:50.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: monmap epoch 3 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: last_changed 2024-08-19T01:27:42.143560+0000 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: created 2024-08-19T01:26:02.528201+0000 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: min_mon_release 19 (squid) 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: election_strategy: 1 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: fsmap 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: osdmap e4: 0 total, 0 up, 0 in 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: mgrmap e13: x(active, since 48s) 2024-08-19T01:27:50.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:49 smithi190 ceph-mon[22032]: overall HEALTH_OK 2024-08-19T01:27:50.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 -- 172.21.15.103:0/3998517213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26101022a0 msgr2=0x7f26101026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:50.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 --2- 172.21.15.103:0/3998517213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26101022a0 0x7f26101026e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2604009900 tx=0x7f260402a480 comp rx=0 tx=0).stop 2024-08-19T01:27:50.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 -- 172.21.15.103:0/3998517213 shutdown_connections 2024-08-19T01:27:50.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 --2- 172.21.15.103:0/3998517213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2610102c20 0x7f261010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 --2- 172.21.15.103:0/3998517213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26101022a0 0x7f26101026e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 --2- 172.21.15.103:0/3998517213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2610100b50 0x7f2610100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 -- 172.21.15.103:0/3998517213 >> 172.21.15.103:0/3998517213 conn(0x7f26100fc500 msgr2=0x7f26100fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 -- 172.21.15.103:0/3998517213 shutdown_connections 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 -- 172.21.15.103:0/3998517213 wait complete. 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.783+0000 7f26162ec640 1 Processor -- start 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f26162ec640 1 -- start start 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f26162ec640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2610100b50 0x7f2610198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f26162ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26101022a0 0x7f26101986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f26162ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2610102c20 0x7f261019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f26162ec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2610110bd0 con 0x7f2610102c20 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f26162ec640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2610110a50 con 0x7f2610100b50 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f26162ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2610110d50 con 0x7f26101022a0 2024-08-19T01:27:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2610102c20 0x7f261019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2610102c20 0x7f261019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49150/0 (socket says 172.21.15.103:49150) 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f260f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26101022a0 0x7f26101986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 -- 172.21.15.103:0/1641612388 learned_addr learned my addr 172.21.15.103:0/1641612388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f260ffff640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2610100b50 0x7f2610198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 -- 172.21.15.103:0/1641612388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26101022a0 msgr2=0x7f26101986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26101022a0 0x7f26101986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 -- 172.21.15.103:0/1641612388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2610100b50 msgr2=0x7f2610198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2610100b50 0x7f2610198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 -- 172.21.15.103:0/1641612388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261019d120 con 0x7f2610102c20 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.784+0000 7f2614889640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2610102c20 0x7f261019ca40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f260800b970 tx=0x7f260800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.785+0000 7f260d7fa640 1 -- 172.21.15.103:0/1641612388 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608019070 con 0x7f2610102c20 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.785+0000 7f260d7fa640 1 -- 172.21.15.103:0/1641612388 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2608014440 con 0x7f2610102c20 2024-08-19T01:27:50.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.785+0000 7f260d7fa640 1 -- 172.21.15.103:0/1641612388 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26080075d0 con 0x7f2610102c20 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.785+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261019d410 con 0x7f2610102c20 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.785+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a4cf0 con 0x7f2610102c20 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.786+0000 7f260d7fa640 1 -- 172.21.15.103:0/1641612388 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f260801c070 con 0x7f2610102c20 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.786+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610100f30 con 0x7f2610102c20 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.786+0000 7f260d7fa640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25f8039290 0x7f25f803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.786+0000 7f260d7fa640 1 -- 172.21.15.103:0/1641612388 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f260804b9c0 con 0x7f2610102c20 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.786+0000 7f260ffff640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25f8039290 0x7f25f803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.786+0000 7f260ffff640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25f8039290 0x7f25f803b750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26000058c0 tx=0x7f260000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:50.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.791+0000 7f260d7fa640 1 -- 172.21.15.103:0/1641612388 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f260800cc30 con 0x7f2610102c20 2024-08-19T01:27:50.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.957+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f26101a4ec0 con 0x7f2610102c20 2024-08-19T01:27:50.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.957+0000 7f260d7fa640 1 -- 172.21.15.103:0/1641612388 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f260800ce10 con 0x7f2610102c20 2024-08-19T01:27:50.961 INFO:teuthology.orchestra.run.smithi103.stdout:# minimal ceph.conf for e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:50.962 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-08-19T01:27:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:27:50.962 INFO:teuthology.orchestra.run.smithi103.stdout: mon_host = [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.959+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25f8039290 msgr2=0x7f25f803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.959+0000 7f26162ec640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25f8039290 0x7f25f803b750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26000058c0 tx=0x7f260000a2d0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.959+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2610102c20 msgr2=0x7f261019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.959+0000 7f26162ec640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2610102c20 0x7f261019ca40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f260800b970 tx=0x7f260800be30 comp rx=0 tx=0).stop 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 shutdown_connections 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25f8039290 0x7f25f803b750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2610102c20 0x7f261019ca40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26101022a0 0x7f26101986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 --2- 172.21.15.103:0/1641612388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2610100b50 0x7f2610198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 >> 172.21.15.103:0/1641612388 conn(0x7f26100fc500 msgr2=0x7f2610069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:50.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 shutdown_connections 2024-08-19T01:27:50.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:50.960+0000 7f26162ec640 1 -- 172.21.15.103:0/1641612388 wait complete. 2024-08-19T01:27:51.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:51.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: Reconfiguring mon.b (monmap changed)... 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: Reconfiguring daemon mon.b on smithi146 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:50 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1641612388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:51.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:51.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: Reconfiguring mon.b (monmap changed)... 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: Reconfiguring daemon mon.b on smithi146 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:50 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1641612388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: Reconfiguring mon.b (monmap changed)... 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: Reconfiguring daemon mon.b on smithi146 2024-08-19T01:27:51.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:51.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:51.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:50 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1641612388' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:51.570 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-19T01:27:51.570 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:27:51.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-08-19T01:27:51.599 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:27:51.599 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:51.665 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:27:51.665 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.conf 2024-08-19T01:27:51.694 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:27:51.695 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:51.762 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:27:51.762 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-08-19T01:27:51.790 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:27:51.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-19T01:27:51.855 INFO:tasks.cephadm:Adding mgr.x on smithi103 2024-08-19T01:27:51.856 INFO:tasks.cephadm:Adding mgr.y on smithi146 2024-08-19T01:27:51.856 INFO:tasks.cephadm:Adding mgr.z on smithi190 2024-08-19T01:27:51.856 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch apply mgr '3;smithi103=x;smithi146=y;smithi190=z' 2024-08-19T01:27:52.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:52.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:52.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:52 smithi190 ceph-mon[22032]: Reconfiguring mon.c (monmap changed)... 2024-08-19T01:27:52.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:52.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:52.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:52.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:52 smithi190 ceph-mon[22032]: Reconfiguring daemon mon.c on smithi190 2024-08-19T01:27:52.937 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:27:52.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:52.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:52.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:52 smithi146 ceph-mon[22098]: Reconfiguring mon.c (monmap changed)... 2024-08-19T01:27:52.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:52.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:52.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:52.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:52 smithi146 ceph-mon[22098]: Reconfiguring daemon mon.c on smithi190 2024-08-19T01:27:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:52 smithi103 ceph-mon[18436]: Reconfiguring mon.c (monmap changed)... 2024-08-19T01:27:53.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-19T01:27:53.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-19T01:27:53.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:52 smithi103 ceph-mon[18436]: Reconfiguring daemon mon.c on smithi190 2024-08-19T01:27:53.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 -- 172.21.15.190:0/2076820199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 msgr2=0x7f155c104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:53.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 --2- 172.21.15.190:0/2076820199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c104820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1558009900 tx=0x7f155802a410 comp rx=0 tx=0).stop 2024-08-19T01:27:53.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 -- 172.21.15.190:0/2076820199 shutdown_connections 2024-08-19T01:27:53.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 --2- 172.21.15.190:0/2076820199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f155c107690 0x7f155c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 --2- 172.21.15.190:0/2076820199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c104d60 0x7f155c107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 --2- 172.21.15.190:0/2076820199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c104820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 -- 172.21.15.190:0/2076820199 >> 172.21.15.190:0/2076820199 conn(0x7f155c0fbfd0 msgr2=0x7f155c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 -- 172.21.15.190:0/2076820199 shutdown_connections 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.398+0000 7f1563147640 1 -- 172.21.15.190:0/2076820199 wait complete. 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 Processor -- start 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 -- start start 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c104d60 0x7f155c1003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f155c107690 0x7f155c100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f155c10d480 con 0x7f155c107690 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f155c10d300 con 0x7f155c104d60 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1563147640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f155c10d600 con 0x7f155c102430 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:53.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:51104/0 (socket says 172.21.15.190:51104) 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 -- 172.21.15.190:0/1774719361 learned_addr learned my addr 172.21.15.190:0/1774719361 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f15616e4640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f155c107690 0x7f155c100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1553fff640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c104d60 0x7f155c1003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 -- 172.21.15.190:0/1774719361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c104d60 msgr2=0x7f155c1003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c104d60 0x7f155c1003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 -- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f155c107690 msgr2=0x7f155c100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f155c107690 0x7f155c100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.399+0000 7f1560ee3640 1 -- 172.21.15.190:0/1774719361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c1011e0 con 0x7f155c102430 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.400+0000 7f1553fff640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c104d60 0x7f155c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.400+0000 7f1560ee3640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c101d90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1558004770 tx=0x7f15580042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.400+0000 7f1551ffb640 1 -- 172.21.15.190:0/1774719361 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155803f030 con 0x7f155c102430 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.400+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c1a1c60 con 0x7f155c102430 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.400+0000 7f1551ffb640 1 -- 172.21.15.190:0/1774719361 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1558003cb0 con 0x7f155c102430 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.400+0000 7f1551ffb640 1 -- 172.21.15.190:0/1774719361 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558035070 con 0x7f155c102430 2024-08-19T01:27:53.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.400+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c1a2160 con 0x7f155c102430 2024-08-19T01:27:53.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.401+0000 7f1551ffb640 1 -- 172.21.15.190:0/1774719361 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1558042030 con 0x7f155c102430 2024-08-19T01:27:53.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.401+0000 7f1551ffb640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1544039290 0x7f154403b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:53.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.401+0000 7f1551ffb640 1 -- 172.21.15.190:0/1774719361 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f155806ce80 con 0x7f155c102430 2024-08-19T01:27:53.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.401+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155c105140 con 0x7f155c102430 2024-08-19T01:27:53.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.402+0000 7f1553fff640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1544039290 0x7f154403b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:53.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.402+0000 7f1553fff640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1544039290 0x7f154403b750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f155c101b10 tx=0x7f154c00a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:53.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.406+0000 7f1551ffb640 1 -- 172.21.15.190:0/1774719361 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f155804d9b0 con 0x7f155c102430 2024-08-19T01:27:53.586 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.583+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi146=y;smithi190=z", "target": ["mon-mgr", ""]}) -- 0x7f155c062360 con 0x7f1544039290 2024-08-19T01:27:53.592 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.589+0000 7f1551ffb640 1 -- 172.21.15.190:0/1774719361 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f155c062360 con 0x7f1544039290 2024-08-19T01:27:53.592 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2024-08-19T01:27:53.594 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1544039290 msgr2=0x7f154403b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:53.594 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1544039290 0x7f154403b750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f155c101b10 tx=0x7f154c00a5c0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.594 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 msgr2=0x7f155c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c101d90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1558004770 tx=0x7f15580042f0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 shutdown_connections 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1544039290 0x7f154403b750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f155c107690 0x7f155c100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f155c104d60 0x7f155c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 --2- 172.21.15.190:0/1774719361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f155c102430 0x7f155c101d90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.591+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 >> 172.21.15.190:0/1774719361 conn(0x7f155c0fbfd0 msgr2=0x7f155c102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.592+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 shutdown_connections 2024-08-19T01:27:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:27:53.592+0000 7f1563147640 1 -- 172.21.15.190:0/1774719361 wait complete. 2024-08-19T01:27:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:53.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:53.953 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:53.953 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:53.953 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:53.953 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:54.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:54.191 DEBUG:teuthology.orchestra.run.smithi146:mgr.y> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.y.service 2024-08-19T01:27:54.194 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.z.service 2024-08-19T01:27:54.196 INFO:tasks.cephadm:Deploying OSDs... 2024-08-19T01:27:54.196 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:27:54.196 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T01:27:54.215 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T01:27:54.215 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-08-19T01:27:54.273 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T01:27:54.273 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:54.273 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-19T01:27:54.273 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:54.273 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:54.273 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 01:27:12.279176031 +0000 2024-08-19T01:27:54.274 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 01:23:21.830105849 +0000 2024-08-19T01:27:54.274 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 01:23:21.830105849 +0000 2024-08-19T01:27:54.274 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 01:23:21.830105849 +0000 2024-08-19T01:27:54.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T01:27:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T01:27:54.342 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T01:27:54.342 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000223825 s, 2.3 MB/s 2024-08-19T01:27:54.343 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T01:27:54.400 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-08-19T01:27:54.458 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T01:27:54.458 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:54.458 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-19T01:27:54.458 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:54.458 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:54.458 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 01:27:12.309175474 +0000 2024-08-19T01:27:54.459 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 01:23:22.167102166 +0000 2024-08-19T01:27:54.459 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 01:23:22.167102166 +0000 2024-08-19T01:27:54.459 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 01:23:22.167102166 +0000 2024-08-19T01:27:54.459 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T01:27:54.523 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T01:27:54.523 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T01:27:54.523 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.00022054 s, 2.3 MB/s 2024-08-19T01:27:54.525 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T01:27:54.582 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-08-19T01:27:54.639 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T01:27:54.639 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:54.639 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-19T01:27:54.639 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:54.640 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:54.640 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 01:27:12.338174936 +0000 2024-08-19T01:27:54.640 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 01:23:22.502098505 +0000 2024-08-19T01:27:54.640 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 01:23:22.502098505 +0000 2024-08-19T01:27:54.640 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 01:23:22.502098505 +0000 2024-08-19T01:27:54.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T01:27:54.703 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T01:27:54.704 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T01:27:54.704 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000196622 s, 2.6 MB/s 2024-08-19T01:27:54.705 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T01:27:54.762 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-08-19T01:27:54.767 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:54 smithi190 ceph-mon[22032]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi146=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:54.767 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:54 smithi190 ceph-mon[22032]: Saving service mgr spec with placement smithi103=x;smithi146=y;smithi190=z;count:3 2024-08-19T01:27:54.767 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:54 smithi190 ceph-mon[22032]: Deploying daemon mgr.z on smithi190 2024-08-19T01:27:54.817 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 01:27:12.370174342 +0000 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 01:23:22.821095018 +0000 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 01:23:22.821095018 +0000 2024-08-19T01:27:54.818 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-08-19 01:23:22.821095018 +0000 2024-08-19T01:27:54.818 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T01:27:54.882 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-08-19T01:27:54.882 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-08-19T01:27:54.882 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000228836 s, 2.2 MB/s 2024-08-19T01:27:54.884 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T01:27:54.940 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-19T01:27:54.940 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T01:27:54.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:54 smithi146 ceph-mon[22098]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi146=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:54.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:54 smithi146 ceph-mon[22098]: Saving service mgr spec with placement smithi103=x;smithi146=y;smithi190=z;count:3 2024-08-19T01:27:54.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:54 smithi146 ceph-mon[22098]: Deploying daemon mgr.z on smithi190 2024-08-19T01:27:54.960 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T01:27:54.960 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-08-19T01:27:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T01:27:55.019 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:55.019 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-08-19T01:27:55.019 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:55.019 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:55.019 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-19 01:27:41.583221328 +0000 2024-08-19T01:27:55.019 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-19 01:23:22.296271694 +0000 2024-08-19T01:27:55.020 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-19 01:23:22.296271694 +0000 2024-08-19T01:27:55.020 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-08-19 01:23:22.296271694 +0000 2024-08-19T01:27:55.020 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T01:27:55.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:54 smithi103 ceph-mon[18436]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi146=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:27:55.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:54 smithi103 ceph-mon[18436]: Saving service mgr spec with placement smithi103=x;smithi146=y;smithi190=z;count:3 2024-08-19T01:27:55.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:54 smithi103 ceph-mon[18436]: Deploying daemon mgr.z on smithi190 2024-08-19T01:27:55.086 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-19T01:27:55.087 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-19T01:27:55.087 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000233598 s, 2.2 MB/s 2024-08-19T01:27:55.088 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T01:27:55.147 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-08-19T01:27:55.204 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T01:27:55.204 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:55.205 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-19T01:27:55.205 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:55.205 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:55.205 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-19 01:27:41.583221328 +0000 2024-08-19T01:27:55.205 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-19 01:23:22.625266889 +0000 2024-08-19T01:27:55.205 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-19 01:23:22.625266889 +0000 2024-08-19T01:27:55.205 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-08-19 01:23:22.625266889 +0000 2024-08-19T01:27:55.205 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T01:27:55.270 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-19T01:27:55.270 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-19T01:27:55.270 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000328652 s, 1.6 MB/s 2024-08-19T01:27:55.271 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T01:27:55.329 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-19 01:27:41.584221315 +0000 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-19 01:23:22.950262142 +0000 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-19 01:23:22.950262142 +0000 2024-08-19T01:27:55.385 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-08-19 01:23:22.950262142 +0000 2024-08-19T01:27:55.386 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T01:27:55.449 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-19T01:27:55.449 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-19T01:27:55.449 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000235827 s, 2.2 MB/s 2024-08-19T01:27:55.450 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T01:27:55.507 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-08-19T01:27:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T01:27:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:55.563 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-19T01:27:55.563 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:55.563 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:55.564 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-19 01:27:41.584221315 +0000 2024-08-19T01:27:55.564 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-19 01:23:23.280257321 +0000 2024-08-19T01:27:55.564 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-19 01:23:23.280257321 +0000 2024-08-19T01:27:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-08-19 01:23:23.280257321 +0000 2024-08-19T01:27:55.564 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T01:27:55.627 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-19T01:27:55.628 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-19T01:27:55.628 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000178566 s, 2.9 MB/s 2024-08-19T01:27:55.629 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T01:27:55.685 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-08-19T01:27:55.685 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-08-19T01:27:55.701 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-19T01:27:55.701 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-08-19T01:27:55.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:55 smithi146 ceph-mon[22098]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:55.757 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-19T01:27:55.757 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-08-19T01:27:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 01:27:55.621918888 +0000 2024-08-19T01:27:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 01:23:22.377918451 +0000 2024-08-19T01:27:55.758 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 01:23:22.377918451 +0000 2024-08-19T01:27:55.758 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-19 01:23:22.377918451 +0000 2024-08-19T01:27:55.758 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-19T01:27:55.862 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T01:27:55.862 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T01:27:55.862 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000249115 s, 2.1 MB/s 2024-08-19T01:27:55.863 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-19T01:27:55.879 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-08-19T01:27:56.097 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:55 smithi103 ceph-mon[18436]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:56.098 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:55 smithi190 ceph-mon[22032]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:56.098 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:55 smithi190 systemd[1]: Starting Ceph mgr.z for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:27:56.113 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-19T01:27:56.113 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:56.113 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-19T01:27:56.113 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:56.114 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:56.114 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 01:27:55.813913462 +0000 2024-08-19T01:27:56.114 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 01:23:22.713908898 +0000 2024-08-19T01:27:56.114 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 01:23:22.713908898 +0000 2024-08-19T01:27:56.114 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-19 01:23:22.713908898 +0000 2024-08-19T01:27:56.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-19T01:27:56.180 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T01:27:56.180 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T01:27:56.180 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000194109 s, 2.6 MB/s 2024-08-19T01:27:56.181 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-19T01:27:56.240 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 01:27:55.813913462 +0000 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 01:23:23.024900057 +0000 2024-08-19T01:27:56.296 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 01:23:23.024900057 +0000 2024-08-19T01:27:56.297 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-19 01:23:23.024900057 +0000 2024-08-19T01:27:56.297 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-19T01:27:56.359 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 podman[23702]: 2024-08-19 01:27:55.9963512 +0000 UTC m=+0.017257876 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:27:56.359 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 podman[23702]: 2024-08-19 01:27:56.122112302 +0000 UTC m=+0.143018968 container create 1841274b263796dd4a436a523834aa664071dc54d4ba502950a137dd087d3688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z, ceph=True, GIT_CLEAN=True, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:27:56.359 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 podman[23702]: 2024-08-19 01:27:56.315911262 +0000 UTC m=+0.336817938 container init 1841274b263796dd4a436a523834aa664071dc54d4ba502950a137dd087d3688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-08-19T01:27:56.359 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 podman[23702]: 2024-08-19 01:27:56.318553385 +0000 UTC m=+0.339460057 container start 1841274b263796dd4a436a523834aa664071dc54d4ba502950a137dd087d3688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-19T01:27:56.359 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 ceph-mgr[23790]: -- 172.21.15.190:0/4051450086 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f34cc690e0 con 0x55f34cc47000 2024-08-19T01:27:56.361 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T01:27:56.361 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T01:27:56.362 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000227742 s, 2.2 MB/s 2024-08-19T01:27:56.363 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-19T01:27:56.420 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-08-19 01:27:55.814913434 +0000 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-08-19 01:23:23.367890306 +0000 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-08-19 01:23:23.367890306 +0000 2024-08-19T01:27:56.478 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-08-19 01:23:23.367890306 +0000 2024-08-19T01:27:56.479 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-19T01:27:56.544 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-08-19T01:27:56.544 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-08-19T01:27:56.544 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000189672 s, 2.7 MB/s 2024-08-19T01:27:56.545 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-19T01:27:56.603 INFO:tasks.cephadm:Deploying osd.0 on smithi103 with /dev/vg_nvme/lv_4... 2024-08-19T01:27:56.603 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-19T01:27:56.614 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 bash[23702]: 1841274b263796dd4a436a523834aa664071dc54d4ba502950a137dd087d3688 2024-08-19T01:27:56.614 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 systemd[1]: Started Ceph mgr.z for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:27:56.614 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:56.541+0000 7fd390eb8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:27:56.614 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:56.611+0000 7fd390eb8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:27:56.857 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:56.881 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:56.753+0000 7fd390eb8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:27:57.176 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:56.911+0000 7fd390eb8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:27:57.177 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.052+0000 7fd390eb8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:27:57.177 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.174+0000 7fd390eb8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:27:57.489 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.300+0000 7fd390eb8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:27:57.489 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.356+0000 7fd390eb8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:27:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:57.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:57.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:57.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:57.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: Deploying daemon mgr.y on smithi146 2024-08-19T01:27:57.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:57 smithi103 ceph-mon[18436]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:57.768 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: Deploying daemon mgr.y on smithi146 2024-08-19T01:27:57.769 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:57 smithi146 ceph-mon[22098]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.659+0000 7fd390eb8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.719+0000 7fd390eb8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.787+0000 7fd390eb8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-19T01:27:57.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:27:57.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:27:57.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: Deploying daemon mgr.y on smithi146 2024-08-19T01:27:57.790 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-mon[22032]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:58.060 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.934+0000 7fd390eb8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:27:58.060 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:57.992+0000 7fd390eb8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:27:58.060 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:58.058+0000 7fd390eb8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:27:58.366 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:58.243+0000 7fd390eb8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:27:58.366 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:58.303+0000 7fd390eb8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:27:58.366 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:58.364+0000 7fd390eb8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:27:58.631 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:58.431+0000 7fd390eb8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:27:58.801 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:58.730+0000 7f4910739040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-19T01:27:58.801 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:58.799+0000 7f4910739040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-19T01:27:59.131 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:58.642+0000 7fd390eb8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:27:59.131 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:58.702+0000 7fd390eb8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:27:59.139 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:58.950+0000 7f4910739040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:27:59.139 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:59.137+0000 7f4910739040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-19T01:27:59.176 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:27:59.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-08-19T01:27:59.418 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:59.282+0000 7f4910739040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-19T01:27:59.447 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:27:59.562 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:59.262+0000 7fd390eb8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:27:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-mon[22098]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:59.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:59.702 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:59.416+0000 7f4910739040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-19T01:27:59.702 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:59.544+0000 7f4910739040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-19T01:27:59.702 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:59.601+0000 7f4910739040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-19T01:27:59.868 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.868 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.868 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.868 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.868 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:59 smithi103 ceph-mon[18436]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:59.869 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:27:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.874+0000 7ff33b4a0640 1 -- 172.21.15.103:0/2477792861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 msgr2=0x7ff334104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.874+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/2477792861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334104780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff330009900 tx=0x7ff33002a410 comp rx=0 tx=0).stop 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- 172.21.15.103:0/2477792861 shutdown_connections 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/2477792861 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3341075f0 0x7ff334109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/2477792861 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334104cc0 0x7ff3341070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/2477792861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334104780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- 172.21.15.103:0/2477792861 >> 172.21.15.103:0/2477792861 conn(0x7ff3340fbfb0 msgr2=0x7ff3340fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- 172.21.15.103:0/2477792861 shutdown_connections 2024-08-19T01:27:59.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- 172.21.15.103:0/2477792861 wait complete. 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 Processor -- start 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- start start 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334104cc0 0x7ff33419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3341075f0 0x7ff3341a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff33410d4c0 con 0x7ff334102390 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff33410d340 con 0x7ff334104cc0 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.875+0000 7ff33b4a0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff33410d640 con 0x7ff3341075f0 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff33923c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff339a3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3341075f0 0x7ff3341a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff33923c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39986/0 (socket says 172.21.15.103:39986) 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff338a3b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334104cc0 0x7ff33419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:59.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff33923c640 1 -- 172.21.15.103:0/3943345405 learned_addr learned my addr 172.21.15.103:0/3943345405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:27:59.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff338a3b640 1 -- 172.21.15.103:0/3943345405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3341075f0 msgr2=0x7ff3341a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:59.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff338a3b640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3341075f0 0x7ff3341a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:59.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff338a3b640 1 -- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 msgr2=0x7ff334199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:27:59.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff338a3b640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:27:59.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff338a3b640 1 -- 172.21.15.103:0/3943345405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3341a19e0 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff33923c640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff339a3d640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3341075f0 0x7ff3341a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff338a3b640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334104cc0 0x7ff33419a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9c0 tx=0x7ff32800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32800ccc0 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3341a1cd0 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3280108e0 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.876+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3280187a0 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.877+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3341a21b0 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.877+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff328018940 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.878+0000 7ff3267fc640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3200392c0 0x7ff32003b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.878+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff32804c880 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.878+0000 7ff33923c640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3200392c0 0x7ff32003b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.878+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff334109140 con 0x7ff334104cc0 2024-08-19T01:27:59.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.878+0000 7ff33923c640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3200392c0 0x7ff32003b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3300096c0 tx=0x7ff330005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:27:59.882 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:27:59.583+0000 7fd390eb8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:27:59.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:27:59.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-mon[22032]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:27:59.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:27:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:27:59.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:27:59.887+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff3280071c0 con 0x7ff334104cc0 2024-08-19T01:28:00.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:00.095+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff334062360 con 0x7ff3200392c0 2024-08-19T01:28:00.201 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:59.929+0000 7f4910739040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-19T01:28:00.201 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:27:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:27:59.990+0000 7f4910739040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-19T01:28:00.201 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.057+0000 7f4910739040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-19T01:28:00.381 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:28:00.015+0000 7fd390eb8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:28:00.381 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:28:00.071+0000 7fd390eb8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:28:00.381 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z[23751]: 2024-08-19T01:28:00.130+0000 7fd390eb8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:28:00.512 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.203+0000 7f4910739040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-19T01:28:00.513 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.262+0000 7f4910739040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-19T01:28:00.513 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.328+0000 7f4910739040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-19T01:28:00.513 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.510+0000 7f4910739040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-19T01:28:00.814 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.571+0000 7f4910739040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-19T01:28:00.814 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.631+0000 7f4910739040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-19T01:28:00.814 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.699+0000 7f4910739040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-19T01:28:00.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:00.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:00.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:00.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:00.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:00.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:00.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:01.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.201 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.914+0000 7f4910739040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-19T01:28:01.201 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:00.973+0000 7f4910739040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-19T01:28:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:01.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:01.803 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:01.494+0000 7f4910739040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-19T01:28:01.803 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:01.801+0000 7f4910739040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-19T01:28:01.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:01.854+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7ff3280178f0 con 0x7ff334104cc0 2024-08-19T01:28:02.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: Reconfiguring mgr.x (unknown last config time)... 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: Reconfiguring daemon mgr.x on smithi103 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: Standby manager daemon z started 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-19T01:28:02.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:01 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: Reconfiguring mgr.x (unknown last config time)... 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: Reconfiguring daemon mgr.x on smithi103 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: Standby manager daemon z started 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-19T01:28:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:01 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:28:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: Reconfiguring mgr.x (unknown last config time)... 2024-08-19T01:28:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: Reconfiguring daemon mgr.x on smithi103 2024-08-19T01:28:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:02.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:02.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: Standby manager daemon z started 2024-08-19T01:28:02.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-19T01:28:02.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:28:02.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-19T01:28:02.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:01 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.190:0/3370212536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:28:02.701 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:02.220+0000 7f4910739040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-19T01:28:02.701 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:02.277+0000 7f4910739040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-19T01:28:02.701 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y[23499]: 2024-08-19T01:28:02.336+0000 7f4910739040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: mgrmap e14: x(active, since 61s), standbys: z 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:02 smithi103 ceph-mon[18436]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: mgrmap e14: x(active, since 61s), standbys: z 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:02 smithi190 ceph-mon[22032]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: mgrmap e14: x(active, since 61s), standbys: z 2024-08-19T01:28:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-19T01:28:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:03.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:03.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:03.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:03.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:02 smithi146 ceph-mon[22098]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1481148013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0157e941-2ba7-4376-b473-9bcdaace0ef4"}]: dispatch 2024-08-19T01:28:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1481148013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0157e941-2ba7-4376-b473-9bcdaace0ef4"}]': finished 2024-08-19T01:28:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: osdmap e5: 1 total, 0 up, 1 in 2024-08-19T01:28:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: Standby manager daemon y started 2024-08-19T01:28:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-19T01:28:04.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:28:04.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-19T01:28:04.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:28:04.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:03 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3782783165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:04.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1481148013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0157e941-2ba7-4376-b473-9bcdaace0ef4"}]: dispatch 2024-08-19T01:28:04.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1481148013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0157e941-2ba7-4376-b473-9bcdaace0ef4"}]': finished 2024-08-19T01:28:04.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: osdmap e5: 1 total, 0 up, 1 in 2024-08-19T01:28:04.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:04.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: Standby manager daemon y started 2024-08-19T01:28:04.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-19T01:28:04.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:28:04.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-19T01:28:04.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:28:04.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:03 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3782783165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:04.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:04.398+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mon.2 v2:172.21.15.146:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7ff32800fca0 con 0x7ff334104cc0 2024-08-19T01:28:04.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1481148013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0157e941-2ba7-4376-b473-9bcdaace0ef4"}]: dispatch 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1481148013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0157e941-2ba7-4376-b473-9bcdaace0ef4"}]': finished 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: osdmap e5: 1 total, 0 up, 1 in 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: Standby manager daemon y started 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='mgr.? 172.21.15.146:0/2095499405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-19T01:28:04.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:03 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3782783165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:05.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:05 smithi146 ceph-mon[22098]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-08-19T01:28:05.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-19T01:28:05.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:05 smithi146 ceph-mon[22098]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:05.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:05 smithi103 ceph-mon[18436]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-08-19T01:28:05.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-19T01:28:05.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:05 smithi103 ceph-mon[18436]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:05.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:05 smithi190 ceph-mon[22032]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-08-19T01:28:05.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-19T01:28:05.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:05 smithi190 ceph-mon[22032]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:08.044 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:07 smithi103 ceph-mon[18436]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:08.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:07 smithi190 ceph-mon[22032]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:07 smithi146 ceph-mon[22098]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:10.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:09 smithi103 ceph-mon[18436]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:09 smithi190 ceph-mon[22032]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:09 smithi146 ceph-mon[22098]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:11.019 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:10 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-19T01:28:11.019 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:10 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:11.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:10 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-19T01:28:11.109 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:10 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:10 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-19T01:28:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:10 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:11.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:11 smithi103 ceph-mon[18436]: Deploying daemon osd.0 on smithi103 2024-08-19T01:28:11.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:11 smithi103 ceph-mon[18436]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:11 smithi190 ceph-mon[22032]: Deploying daemon osd.0 on smithi103 2024-08-19T01:28:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:11 smithi190 ceph-mon[22032]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:12.190 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:11 smithi146 ceph-mon[22098]: Deploying daemon osd.0 on smithi103 2024-08-19T01:28:12.190 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:11 smithi146 ceph-mon[22098]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:13.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:13 smithi103 ceph-mon[18436]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:13 smithi190 ceph-mon[22032]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:13 smithi146 ceph-mon[22098]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:15.954 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:15 smithi103 ceph-mon[18436]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:15.954 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:15.954 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:15.954 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:15 smithi190 ceph-mon[22032]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:15 smithi146 ceph-mon[22098]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:17.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:17 smithi103 ceph-mon[18436]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:17.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:17.932 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:17 smithi190 ceph-mon[22032]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:17 smithi146 ceph-mon[22098]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:18.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.704+0000 7ff3267fc640 1 -- 172.21.15.103:0/3943345405 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff334062360 con 0x7ff3200392c0 2024-08-19T01:28:18.706 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 0 on host 'smithi103' 2024-08-19T01:28:18.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3200392c0 msgr2=0x7ff32003b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:18.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3200392c0 0x7ff32003b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3300096c0 tx=0x7ff330005a20 comp rx=0 tx=0).stop 2024-08-19T01:28:18.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334104cc0 msgr2=0x7ff33419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334104cc0 0x7ff33419a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9c0 tx=0x7ff32800ee80 comp rx=0 tx=0).stop 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 shutdown_connections 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3200392c0 0x7ff32003b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3341075f0 0x7ff3341a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff334104cc0 0x7ff33419a2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 --2- 172.21.15.103:0/3943345405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff334102390 0x7ff334199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 >> 172.21.15.103:0/3943345405 conn(0x7ff3340fbfb0 msgr2=0x7ff334101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 shutdown_connections 2024-08-19T01:28:18.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:28:18.708+0000 7ff33b4a0640 1 -- 172.21.15.103:0/3943345405 wait complete. 2024-08-19T01:28:19.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:19.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:19.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:19.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.023 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:18 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:19.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:18 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:19.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:18 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:18 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:19.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:18 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:18 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:18 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:19.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:18 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:19.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:18 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:18 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:19.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:18 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:18 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:19.395 DEBUG:teuthology.orchestra.run.smithi103:osd.0> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.0.service 2024-08-19T01:28:19.398 INFO:tasks.cephadm:Deploying osd.1 on smithi146 with /dev/vg_nvme/lv_4... 2024-08-19T01:28:19.398 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-19T01:28:19.641 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:28:19.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:19 smithi103 ceph-mon[18436]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:19.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:19 smithi103 ceph-mon[18436]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:28:20.020 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:19 smithi146 ceph-mon[22098]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:20.020 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:19 smithi146 ceph-mon[22098]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:28:20.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:19 smithi190 ceph-mon[22032]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:20.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:19 smithi190 ceph-mon[22032]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-19T01:28:20.869 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:20 smithi103 ceph-mon[18436]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-19T01:28:20.869 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:20 smithi103 ceph-mon[18436]: osdmap e6: 1 total, 0 up, 1 in 2024-08-19T01:28:20.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:20 smithi103 ceph-mon[18436]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:28:20.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:20.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:20 smithi103 ceph-mon[18436]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:28:20.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:20 smithi103 ceph-mon[18436]: osdmap e7: 1 total, 0 up, 1 in 2024-08-19T01:28:20.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:20.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:20 smithi190 ceph-mon[22032]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-19T01:28:20.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:20 smithi190 ceph-mon[22032]: osdmap e6: 1 total, 0 up, 1 in 2024-08-19T01:28:20.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:20 smithi190 ceph-mon[22032]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:28:20.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:20.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:20 smithi190 ceph-mon[22032]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:28:20.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:20 smithi190 ceph-mon[22032]: osdmap e7: 1 total, 0 up, 1 in 2024-08-19T01:28:20.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:21.022 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:20 smithi146 ceph-mon[22098]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-19T01:28:21.022 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:20 smithi146 ceph-mon[22098]: osdmap e6: 1 total, 0 up, 1 in 2024-08-19T01:28:21.023 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:20 smithi146 ceph-mon[22098]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:28:21.023 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:21.023 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:20 smithi146 ceph-mon[22098]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:28:21.023 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:20 smithi146 ceph-mon[22098]: osdmap e7: 1 total, 0 up, 1 in 2024-08-19T01:28:21.023 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:21.555 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:28:21 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0[31822]: 2024-08-19T01:28:21.187+0000 7f728db23640 -1 osd.0 0 waiting for initial osdmap 2024-08-19T01:28:21.555 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:28:21 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0[31822]: 2024-08-19T01:28:21.191+0000 7f728512f640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:28:21.823 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:28:21.846 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:21 smithi146 ceph-mon[22098]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:21.846 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:21 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:21.846 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:21 smithi146 ceph-mon[22098]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' 2024-08-19T01:28:21.846 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:21 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:21.848 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_4 2024-08-19T01:28:21.922 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:21 smithi103 ceph-mon[18436]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:21.922 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:21.922 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:21 smithi103 ceph-mon[18436]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' 2024-08-19T01:28:21.922 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:21 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:22.092 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:28:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:21 smithi190 ceph-mon[22032]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:21 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:21 smithi190 ceph-mon[22032]: from='osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281]' entity='osd.0' 2024-08-19T01:28:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:21 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1050084097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e68102450 msgr2=0x7f8e68104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1050084097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e68102450 0x7f8e68104840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009ca0 tx=0x7f8e5802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1050084097 shutdown_connections 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1050084097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e681076b0 0x7f8e68109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1050084097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e68104d80 0x7f8e68107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1050084097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e68102450 0x7f8e68104840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1050084097 >> 172.21.15.146:0/1050084097 conn(0x7f8e680fbfb0 msgr2=0x7f8e680fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:22.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1050084097 shutdown_connections 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.550+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1050084097 wait complete. 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 Processor -- start 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 -- start start 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e68102450 0x7f8e68197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e68104d80 0x7f8e681980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e681076b0 0x7f8e6819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6810d470 con 0x7f8e68104d80 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6810d2f0 con 0x7f8e681076b0 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6ce8d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6810d5f0 con 0x7f8e68102450 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e681076b0 0x7f8e6819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:22.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e681076b0 0x7f8e6819f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60928/0 (socket says 172.21.15.146:60928) 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 -- 172.21.15.146:0/1862063783 learned_addr learned my addr 172.21.15.146:0/1862063783 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e65d9b640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e68104d80 0x7f8e681980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e6659c640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e68102450 0x7f8e68197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 -- 172.21.15.146:0/1862063783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e68102450 msgr2=0x7f8e68197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e68102450 0x7f8e68197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 -- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e68104d80 msgr2=0x7f8e681980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e68104d80 0x7f8e681980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 -- 172.21.15.146:0/1862063783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6819f7e0 con 0x7f8e681076b0 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e66d9d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e681076b0 0x7f8e6819f170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000e970 tx=0x7f8e6000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.551+0000 7f8e5f7fe640 1 -- 172.21.15.146:0/1862063783 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000ccc0 con 0x7f8e681076b0 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.552+0000 7f8e5f7fe640 1 -- 172.21.15.146:0/1862063783 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e6000ce60 con 0x7f8e681076b0 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.552+0000 7f8e5f7fe640 1 -- 172.21.15.146:0/1862063783 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60010430 con 0x7f8e681076b0 2024-08-19T01:28:22.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.552+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6819fad0 con 0x7f8e681076b0 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.552+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e681a0010 con 0x7f8e681076b0 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.553+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e680fdaa0 con 0x7f8e681076b0 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.554+0000 7f8e5f7fe640 1 -- 172.21.15.146:0/1862063783 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f8e60004f90 con 0x7f8e681076b0 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.554+0000 7f8e5f7fe640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e540a2ba0 0x7f8e540a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.554+0000 7f8e5f7fe640 1 -- 172.21.15.146:0/1862063783 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f8e600cd950 con 0x7f8e681076b0 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.555+0000 7f8e6659c640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e540a2ba0 0x7f8e540a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.555+0000 7f8e6659c640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e540a2ba0 0x7f8e540a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005e30 tx=0x7f8e58004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:22.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.559+0000 7f8e5f7fe640 1 -- 172.21.15.146:0/1862063783 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8e6009c320 con 0x7f8e681076b0 2024-08-19T01:28:22.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:22.733+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8e68062360 con 0x7f8e540a2ba0 2024-08-19T01:28:23.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:28:23.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:28:23.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] boot 2024-08-19T01:28:23.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: osdmap e8: 1 total, 1 up, 1 in 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:23.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:23.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:28:23.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:28:23.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] boot 2024-08-19T01:28:23.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: osdmap e8: 1 total, 1 up, 1 in 2024-08-19T01:28:23.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:23.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:23.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: osd.0 [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] boot 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: osdmap e8: 1 total, 1 up, 1 in 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:23.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:23.136 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:23.136 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:23.136 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:23 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi103 2024-08-19T01:28:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:23 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi103 to 17088M 2024-08-19T01:28:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:23 smithi103 ceph-mon[18436]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:23 smithi103 ceph-mon[18436]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:23 smithi103 ceph-mon[18436]: osdmap e9: 1 total, 1 up, 1 in 2024-08-19T01:28:24.094 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:23 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi103 2024-08-19T01:28:24.094 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:23 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi103 to 17088M 2024-08-19T01:28:24.094 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:23 smithi146 ceph-mon[22098]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:24.094 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:23 smithi146 ceph-mon[22098]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:24.094 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:23 smithi146 ceph-mon[22098]: osdmap e9: 1 total, 1 up, 1 in 2024-08-19T01:28:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:23 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi103 2024-08-19T01:28:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:23 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi103 to 17088M 2024-08-19T01:28:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:23 smithi190 ceph-mon[22032]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-19T01:28:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:23 smithi190 ceph-mon[22032]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:23 smithi190 ceph-mon[22032]: osdmap e9: 1 total, 1 up, 1 in 2024-08-19T01:28:26.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:25 smithi103 ceph-mon[18436]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:26.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:25 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/612492599' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]: dispatch 2024-08-19T01:28:26.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:25 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]: dispatch 2024-08-19T01:28:26.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:25 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]': finished 2024-08-19T01:28:26.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:25 smithi103 ceph-mon[18436]: osdmap e10: 2 total, 1 up, 2 in 2024-08-19T01:28:26.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:25 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:26.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:25 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/3296791215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:25 smithi190 ceph-mon[22032]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:25 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/612492599' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]: dispatch 2024-08-19T01:28:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:25 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]: dispatch 2024-08-19T01:28:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:25 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]': finished 2024-08-19T01:28:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:25 smithi190 ceph-mon[22032]: osdmap e10: 2 total, 1 up, 2 in 2024-08-19T01:28:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:25 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:25 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/3296791215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:26.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:25 smithi146 ceph-mon[22098]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:26.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:25 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/612492599' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]: dispatch 2024-08-19T01:28:26.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:25 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]: dispatch 2024-08-19T01:28:26.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:25 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc4bc49c-cf46-48e4-98ed-60ad89a2812a"}]': finished 2024-08-19T01:28:26.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:25 smithi146 ceph-mon[22098]: osdmap e10: 2 total, 1 up, 2 in 2024-08-19T01:28:26.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:25 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:26.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:25 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/3296791215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:28.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:27 smithi103 ceph-mon[18436]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:28.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:27 smithi190 ceph-mon[22032]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:27 smithi146 ceph-mon[22098]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:30.077 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:29 smithi146 ceph-mon[22098]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:30.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:29 smithi190 ceph-mon[22032]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:29 smithi103 ceph-mon[18436]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:31.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:31 smithi146 ceph-mon[22098]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:31.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-19T01:28:31.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:31 smithi190 ceph-mon[22032]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-19T01:28:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:32.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:31 smithi103 ceph-mon[18436]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:32.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-19T01:28:32.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:33.128 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:32 smithi146 ceph-mon[22098]: Deploying daemon osd.1 on smithi146 2024-08-19T01:28:33.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:32 smithi190 ceph-mon[22032]: Deploying daemon osd.1 on smithi146 2024-08-19T01:28:33.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:32 smithi103 ceph-mon[18436]: Deploying daemon osd.1 on smithi146 2024-08-19T01:28:34.095 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:33 smithi146 ceph-mon[22098]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:34.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:33 smithi190 ceph-mon[22032]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:34.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:33 smithi103 ceph-mon[18436]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:35.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:35 smithi146 ceph-mon[22098]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:35 smithi190 ceph-mon[22032]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:35 smithi103 ceph-mon[18436]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:37.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:37.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:37.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:37.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:38.076 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:37 smithi146 ceph-mon[22098]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:38.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:37 smithi190 ceph-mon[22032]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:38.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:37 smithi103 ceph-mon[18436]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:39.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.571+0000 7f8e5f7fe640 1 -- 172.21.15.146:0/1862063783 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8e68062360 con 0x7f8e540a2ba0 2024-08-19T01:28:39.574 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 1 on host 'smithi146' 2024-08-19T01:28:39.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e540a2ba0 msgr2=0x7f8e540a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:39.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e540a2ba0 0x7f8e540a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005e30 tx=0x7f8e58004a40 comp rx=0 tx=0).stop 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e681076b0 msgr2=0x7f8e6819f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e681076b0 0x7f8e6819f170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000e970 tx=0x7f8e6000ed70 comp rx=0 tx=0).stop 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 shutdown_connections 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e540a2ba0 0x7f8e540a5060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e681076b0 0x7f8e6819f170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e68104d80 0x7f8e681980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 --2- 172.21.15.146:0/1862063783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e68102450 0x7f8e68197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.573+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 >> 172.21.15.146:0/1862063783 conn(0x7f8e680fbfb0 msgr2=0x7f8e68102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.574+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 shutdown_connections 2024-08-19T01:28:39.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:28:39.574+0000 7f8e6ce8d640 1 -- 172.21.15.146:0/1862063783 wait complete. 2024-08-19T01:28:40.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:39 smithi190 ceph-mon[22032]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:40.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:40.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:39 smithi146 ceph-mon[22098]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:40.241 DEBUG:teuthology.orchestra.run.smithi146:osd.1> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.1.service 2024-08-19T01:28:40.244 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/vg_nvme/lv_4... 2024-08-19T01:28:40.244 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-19T01:28:40.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:39 smithi103 ceph-mon[18436]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:40.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:40.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:40.501 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:28:40.756 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:28:40 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1[26554]: 2024-08-19T01:28:40.502+0000 7fa0a5bd1640 -1 osd.1 0 log_to_monitors true 2024-08-19T01:28:41.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:41 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:41.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:41 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:41.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:41 smithi190 ceph-mon[22032]: from='osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:28:41.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:41 smithi190 ceph-mon[22032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:28:41.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:41 smithi190 ceph-mon[22032]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:41.695 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:41 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:41.695 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:41 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:41.695 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:41 smithi146 ceph-mon[22098]: from='osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:28:41.695 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:41 smithi146 ceph-mon[22098]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:28:41.695 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:41 smithi146 ceph-mon[22098]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:41.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:41 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:41.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:41 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:41.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:41 smithi103 ceph-mon[18436]: from='osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:28:41.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:41 smithi103 ceph-mon[18436]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-19T01:28:41.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:41 smithi103 ceph-mon[18436]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:42.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:42 smithi190 ceph-mon[22032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-19T01:28:42.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:42 smithi190 ceph-mon[22032]: osdmap e11: 2 total, 1 up, 2 in 2024-08-19T01:28:42.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:42 smithi190 ceph-mon[22032]: from='osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:28:42.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:42.629 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:42 smithi190 ceph-mon[22032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:28:42.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:42 smithi146 ceph-mon[22098]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-19T01:28:42.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:42 smithi146 ceph-mon[22098]: osdmap e11: 2 total, 1 up, 2 in 2024-08-19T01:28:42.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:42 smithi146 ceph-mon[22098]: from='osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:28:42.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:42.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:42 smithi146 ceph-mon[22098]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:28:42.702 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:28:42 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1[26554]: 2024-08-19T01:28:42.349+0000 7fa0a1b54640 -1 osd.1 0 waiting for initial osdmap 2024-08-19T01:28:42.702 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:28:42 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1[26554]: 2024-08-19T01:28:42.352+0000 7fa09d97c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:28:42.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:42 smithi103 ceph-mon[18436]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-19T01:28:42.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:42 smithi103 ceph-mon[18436]: osdmap e11: 2 total, 1 up, 2 in 2024-08-19T01:28:42.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:42 smithi103 ceph-mon[18436]: from='osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:28:42.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:42.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:42 smithi103 ceph-mon[18436]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:28:42.875 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:28:42.902 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2024-08-19T01:28:43.149 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:28:43.584 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:43 smithi190 ceph-mon[22032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:28:43.584 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:43 smithi190 ceph-mon[22032]: osdmap e12: 2 total, 1 up, 2 in 2024-08-19T01:28:43.584 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:43.584 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:43.584 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:43 smithi190 ceph-mon[22032]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 -- 172.21.15.190:0/651167128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 msgr2=0x7f28b4101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 --2- 172.21.15.190:0/651167128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4101d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009900 tx=0x7f28a802a410 comp rx=0 tx=0).stop 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 -- 172.21.15.190:0/651167128 shutdown_connections 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 --2- 172.21.15.190:0/651167128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b41034f0 0x7f28b4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 --2- 172.21.15.190:0/651167128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b4102b30 0x7f28b4102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 --2- 172.21.15.190:0/651167128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4101d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 -- 172.21.15.190:0/651167128 >> 172.21.15.190:0/651167128 conn(0x7f28b40fd0c0 msgr2=0x7f28b40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 -- 172.21.15.190:0/651167128 shutdown_connections 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.582+0000 7f28bace6640 1 -- 172.21.15.190:0/651167128 wait complete. 2024-08-19T01:28:43.616 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 Processor -- start 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 -- start start 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b30 0x7f28b419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b41034f0 0x7f28b41a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410d4c0 con 0x7f28b41034f0 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410d340 con 0x7f28b4102b30 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28bace6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410d640 con 0x7f28b4101930 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28b8a82640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28b8a82640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:39052/0 (socket says 172.21.15.190:39052) 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28b8a82640 1 -- 172.21.15.190:0/4293251939 learned_addr learned my addr 172.21.15.190:0/4293251939 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.583+0000 7f28b3fff640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b30 0x7f28b419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b9283640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b41034f0 0x7f28b41a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:43.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b8a82640 1 -- 172.21.15.190:0/4293251939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b30 msgr2=0x7f28b419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b8a82640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b30 0x7f28b419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b8a82640 1 -- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b41034f0 msgr2=0x7f28b41a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b8a82640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b41034f0 0x7f28b41a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b8a82640 1 -- 172.21.15.190:0/4293251939 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41a1960 con 0x7f28b4101930 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b9283640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b41034f0 0x7f28b41a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b8a82640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4199d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28a80314e0 tx=0x7f28a8003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b1ffb640 1 -- 172.21.15.190:0/4293251939 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a803f070 con 0x7f28b4101930 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b1ffb640 1 -- 172.21.15.190:0/4293251939 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28a8031e40 con 0x7f28b4101930 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28b1ffb640 1 -- 172.21.15.190:0/4293251939 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035030 con 0x7f28b4101930 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b41a1bf0 con 0x7f28b4101930 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.584+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a2120 con 0x7f28b4101930 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.585+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b41a23b0 con 0x7f28b4101930 2024-08-19T01:28:43.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.587+0000 7f28b1ffb640 1 -- 172.21.15.190:0/4293251939 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f28a8042070 con 0x7f28b4101930 2024-08-19T01:28:43.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.587+0000 7f28b1ffb640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28a40a2ba0 0x7f28a40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:28:43.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.587+0000 7f28b1ffb640 1 -- 172.21.15.190:0/4293251939 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f28a80ee0b0 con 0x7f28b4101930 2024-08-19T01:28:43.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.587+0000 7f28b3fff640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28a40a2ba0 0x7f28a40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:28:43.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.588+0000 7f28b3fff640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28a40a2ba0 0x7f28a40a5060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f28a0005fd0 tx=0x7f28a0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:28:43.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.592+0000 7f28b1ffb640 1 -- 172.21.15.190:0/4293251939 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f28a80bc860 con 0x7f28b4101930 2024-08-19T01:28:43.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:43 smithi146 ceph-mon[22098]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:28:43.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:43 smithi146 ceph-mon[22098]: osdmap e12: 2 total, 1 up, 2 in 2024-08-19T01:28:43.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:43.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:43.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:43 smithi146 ceph-mon[22098]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:43.769 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:28:43.768+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f28b4075870 con 0x7f28a40a2ba0 2024-08-19T01:28:43.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:43 smithi103 ceph-mon[18436]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:28:43.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:43 smithi103 ceph-mon[18436]: osdmap e12: 2 total, 1 up, 2 in 2024-08-19T01:28:43.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:43.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:43.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:43 smithi103 ceph-mon[18436]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] boot 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: osdmap e13: 2 total, 2 up, 2 in 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi146 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:44.478 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi146 to 17088M 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:44.479 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] boot 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: osdmap e13: 2 total, 2 up, 2 in 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi146 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi146 to 17088M 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:44.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:28:44.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:28:44.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: osd.1 [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] boot 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: osdmap e13: 2 total, 2 up, 2 in 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi146 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi146 to 17088M 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:44.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:28:44.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:44.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:28:44.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:28:44.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:28:44.806 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:45.584 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:45 smithi190 ceph-mon[22032]: osdmap e14: 2 total, 2 up, 2 in 2024-08-19T01:28:45.584 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:45 smithi190 ceph-mon[22032]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:45.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:45 smithi146 ceph-mon[22098]: osdmap e14: 2 total, 2 up, 2 in 2024-08-19T01:28:45.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:45 smithi146 ceph-mon[22098]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:45.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:45 smithi103 ceph-mon[18436]: osdmap e14: 2 total, 2 up, 2 in 2024-08-19T01:28:45.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:45 smithi103 ceph-mon[18436]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:46.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:46 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/151873063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]: dispatch 2024-08-19T01:28:46.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:46 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]: dispatch 2024-08-19T01:28:46.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:46 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]': finished 2024-08-19T01:28:46.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:46 smithi190 ceph-mon[22032]: osdmap e15: 3 total, 2 up, 3 in 2024-08-19T01:28:46.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:46 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:46.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:46 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/151873063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]: dispatch 2024-08-19T01:28:46.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:46 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]: dispatch 2024-08-19T01:28:46.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:46 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]': finished 2024-08-19T01:28:46.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:46 smithi146 ceph-mon[22098]: osdmap e15: 3 total, 2 up, 3 in 2024-08-19T01:28:46.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:46 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:46.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:46 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/151873063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]: dispatch 2024-08-19T01:28:46.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:46 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]: dispatch 2024-08-19T01:28:46.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:46 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5253afd0-394e-4565-b605-fad40ff30406"}]': finished 2024-08-19T01:28:46.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:46 smithi103 ceph-mon[18436]: osdmap e15: 3 total, 2 up, 3 in 2024-08-19T01:28:46.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:46 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:28:47.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:47 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/2088068090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:47.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:47 smithi146 ceph-mon[22098]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:47 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/2088068090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:47 smithi103 ceph-mon[18436]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:47 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/2088068090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:28:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:47 smithi190 ceph-mon[22032]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:50.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:49 smithi103 ceph-mon[18436]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:50.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:49 smithi190 ceph-mon[22032]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:50.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:49 smithi146 ceph-mon[22098]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:52.025 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:51 smithi190 ceph-mon[22032]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:51 smithi103 ceph-mon[18436]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:51 smithi146 ceph-mon[22098]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:52.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-19T01:28:52.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:52.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-19T01:28:52.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-19T01:28:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:28:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:53 smithi103 ceph-mon[18436]: Deploying daemon osd.2 on smithi190 2024-08-19T01:28:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:53 smithi103 ceph-mon[18436]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:54.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:53 smithi190 ceph-mon[22032]: Deploying daemon osd.2 on smithi190 2024-08-19T01:28:54.072 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:53 smithi190 ceph-mon[22032]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:53 smithi146 ceph-mon[22098]: Deploying daemon osd.2 on smithi190 2024-08-19T01:28:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:53 smithi146 ceph-mon[22098]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:56.042 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:55 smithi190 ceph-mon[22032]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:56.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:55 smithi103 ceph-mon[18436]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:55 smithi146 ceph-mon[22098]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:58.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:57 smithi103 ceph-mon[18436]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:58.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:58.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:58.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:58.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:57 smithi190 ceph-mon[22032]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:58.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:58.068 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:58.069 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:57 smithi146 ceph-mon[22098]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:28:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:28:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:28:59.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:28:59 smithi190 ceph-mon[22032]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:28:59 smithi103 ceph-mon[18436]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:28:59 smithi146 ceph-mon[22098]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:00.763 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.761+0000 7f28b1ffb640 1 -- 172.21.15.190:0/4293251939 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f28b4075870 con 0x7f28a40a2ba0 2024-08-19T01:29:00.763 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28a40a2ba0 msgr2=0x7f28a40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28a40a2ba0 0x7f28a40a5060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f28a0005fd0 tx=0x7f28a0005850 comp rx=0 tx=0).stop 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 msgr2=0x7f28b4199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4199d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28a80314e0 tx=0x7f28a8003c80 comp rx=0 tx=0).stop 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 shutdown_connections 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28a40a2ba0 0x7f28a40a5060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b41034f0 0x7f28b41a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:00.765 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28b4102b30 0x7f28b419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:00.766 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 --2- 172.21.15.190:0/4293251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28b4101930 0x7f28b4199d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:00.766 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.763+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 >> 172.21.15.190:0/4293251939 conn(0x7f28b40fd0c0 msgr2=0x7f28b4074150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:00.766 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.764+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 shutdown_connections 2024-08-19T01:29:00.766 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:00.764+0000 7f28bace6640 1 -- 172.21.15.190:0/4293251939 wait complete. 2024-08-19T01:29:01.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:01.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:01.059 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.060 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:01.060 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:01.060 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.060 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:01.410 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.2.service 2024-08-19T01:29:01.412 INFO:tasks.cephadm:Deploying osd.3 on smithi103 with /dev/vg_nvme/lv_3... 2024-08-19T01:29:01.413 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-19T01:29:01.675 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:29:02.050 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:01 smithi103 ceph-mon[18436]: from='osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:29:02.050 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:01 smithi103 ceph-mon[18436]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:29:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:01 smithi190 ceph-mon[22032]: from='osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:29:02.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:01 smithi190 ceph-mon[22032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:29:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:01 smithi146 ceph-mon[22098]: from='osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:29:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:01 smithi146 ceph-mon[22098]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-19T01:29:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:02 smithi190 ceph-mon[22032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-19T01:29:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:02 smithi190 ceph-mon[22032]: from='osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:29:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:02 smithi190 ceph-mon[22032]: osdmap e16: 3 total, 2 up, 3 in 2024-08-19T01:29:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:02 smithi190 ceph-mon[22032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:29:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:02 smithi190 ceph-mon[22032]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:03.158 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:02 smithi103 ceph-mon[18436]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-19T01:29:03.158 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:02 smithi103 ceph-mon[18436]: from='osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:29:03.158 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:02 smithi103 ceph-mon[18436]: osdmap e16: 3 total, 2 up, 3 in 2024-08-19T01:29:03.159 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:03.159 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:02 smithi103 ceph-mon[18436]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:29:03.159 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:02 smithi103 ceph-mon[18436]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:02 smithi146 ceph-mon[22098]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-19T01:29:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:02 smithi146 ceph-mon[22098]: from='osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:29:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:02 smithi146 ceph-mon[22098]: osdmap e16: 3 total, 2 up, 3 in 2024-08-19T01:29:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:02 smithi146 ceph-mon[22098]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:29:03.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:02 smithi146 ceph-mon[22098]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:03.631 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:03 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2[27008]: 2024-08-19T01:29:03.352+0000 7f3079138640 -1 osd.2 0 waiting for initial osdmap 2024-08-19T01:29:03.631 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:03 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2[27008]: 2024-08-19T01:29:03.356+0000 7f3070f58640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:29:04.195 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:03 smithi103 ceph-mon[18436]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:29:04.196 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:03 smithi103 ceph-mon[18436]: osdmap e17: 3 total, 2 up, 3 in 2024-08-19T01:29:04.196 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:03 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:04.196 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:03 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:04.196 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:03 smithi103 ceph-mon[18436]: from='osd.2 ' entity='osd.2' 2024-08-19T01:29:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:29:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:03 smithi146 ceph-mon[22098]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:29:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:03 smithi146 ceph-mon[22098]: osdmap e17: 3 total, 2 up, 3 in 2024-08-19T01:29:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:03 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:03 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:03 smithi146 ceph-mon[22098]: from='osd.2 ' entity='osd.2' 2024-08-19T01:29:04.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-08-19T01:29:04.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:03 smithi190 ceph-mon[22032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:29:04.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:03 smithi190 ceph-mon[22032]: osdmap e17: 3 total, 2 up, 3 in 2024-08-19T01:29:04.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:03 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:04.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:03 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:04.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:03 smithi190 ceph-mon[22032]: from='osd.2 ' entity='osd.2' 2024-08-19T01:29:04.491 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:29:04.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 -- 172.21.15.103:0/1259649706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 msgr2=0x7f325c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:04.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 --2- 172.21.15.103:0/1259649706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 0x7f325c100c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3250009900 tx=0x7f325002a410 comp rx=0 tx=0).stop 2024-08-19T01:29:04.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 -- 172.21.15.103:0/1259649706 shutdown_connections 2024-08-19T01:29:04.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 --2- 172.21.15.103:0/1259649706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c1023c0 0x7f325c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:04.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 --2- 172.21.15.103:0/1259649706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f325c101a00 0x7f325c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:04.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 --2- 172.21.15.103:0/1259649706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 0x7f325c100c00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:04.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 -- 172.21.15.103:0/1259649706 >> 172.21.15.103:0/1259649706 conn(0x7f325c0fbfb0 msgr2=0x7f325c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 -- 172.21.15.103:0/1259649706 shutdown_connections 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.949+0000 7f3262f70640 1 -- 172.21.15.103:0/1259649706 wait complete. 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 Processor -- start 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 -- start start 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 0x7f325c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c101a00 0x7f325c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f325c1023c0 0x7f325c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f325c10c310 con 0x7f325c101a00 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f325c10c190 con 0x7f325c1023c0 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3262f70640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f325c10c490 con 0x7f325c100800 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f325bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c101a00 0x7f325c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:04.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f325bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c101a00 0x7f325c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53126/0 (socket says 172.21.15.103:53126) 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f325bfff640 1 -- 172.21.15.103:0/2845062919 learned_addr learned my addr 172.21.15.103:0/2845062919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f326150d640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f325c1023c0 0x7f325c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.950+0000 7f3260d0c640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 0x7f325c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f326150d640 1 -- 172.21.15.103:0/2845062919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 msgr2=0x7f325c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f326150d640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 0x7f325c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f326150d640 1 -- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c101a00 msgr2=0x7f325c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f326150d640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c101a00 0x7f325c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f326150d640 1 -- 172.21.15.103:0/2845062919 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325c19f780 con 0x7f325c1023c0 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f3260d0c640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 0x7f325c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f325bfff640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c101a00 0x7f325c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f326150d640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f325c1023c0 0x7f325c19f110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f325400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325400ce20 con 0x7f325c1023c0 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325c19fa70 con 0x7f325c1023c0 2024-08-19T01:29:04.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.951+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3254005020 con 0x7f325c1023c0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.952+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254010690 con 0x7f325c1023c0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.952+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f325c19ffb0 con 0x7f325c1023c0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.953+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325c100c00 con 0x7f325c1023c0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.954+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f3254004a00 con 0x7f325c1023c0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.955+0000 7f3259ffb640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f324c0a29d0 0x7f324c0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.955+0000 7f3260d0c640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f324c0a29d0 0x7f324c0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.955+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f32540cdcd0 con 0x7f325c1023c0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.955+0000 7f3260d0c640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f324c0a29d0 0x7f324c0a4e90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3250004770 tx=0x7f3250000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:04.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:04.959+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f325409c1c0 con 0x7f325c1023c0 2024-08-19T01:29:05.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:05.144+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f325c1a0240 con 0x7f324c0a29d0 2024-08-19T01:29:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:29:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:29:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] boot 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: osdmap e18: 3 total, 3 up, 3 in 2024-08-19T01:29:05.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:05.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:05.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] boot 2024-08-19T01:29:05.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: osdmap e18: 3 total, 3 up, 3 in 2024-08-19T01:29:05.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:05.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: osd.2 [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] boot 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: osdmap e18: 3 total, 3 up, 3 in 2024-08-19T01:29:05.454 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-19T01:29:06.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi190 2024-08-19T01:29:06.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi190 to 17088M 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: osdmap e19: 3 total, 3 up, 3 in 2024-08-19T01:29:06.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi190 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi190 to 17088M 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: osdmap e19: 3 total, 3 up, 3 in 2024-08-19T01:29:06.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:29:06.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi190 2024-08-19T01:29:06.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi190 to 17088M 2024-08-19T01:29:06.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-19T01:29:06.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:29:06.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:06.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:06.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:06.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:06.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-19T01:29:06.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: osdmap e19: 3 total, 3 up, 3 in 2024-08-19T01:29:06.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:29:06.944 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:29:06.945 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:06.945 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36591]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:06.945 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36591]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:07.304 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36600]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:29:07.304 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36600]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:07.304 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36600]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:07.304 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:29:06 smithi103 sudo[36600]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:07.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:07.375+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3254098130 con 0x7f325c1023c0 2024-08-19T01:29:07.508 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30530]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:29:07.508 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:07.508 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30530]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:07.508 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30530]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:07.508 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:29:07.508 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:07 smithi146 ceph-mon[22098]: osdmap e20: 3 total, 3 up, 3 in 2024-08-19T01:29:07.509 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:07 smithi146 ceph-mon[22098]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:07.509 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:07.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:29:07.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:07 smithi190 ceph-mon[22032]: osdmap e20: 3 total, 3 up, 3 in 2024-08-19T01:29:07.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:07 smithi190 ceph-mon[22032]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:07.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:07.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:29:07.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:07 smithi103 ceph-mon[18436]: osdmap e20: 3 total, 3 up, 3 in 2024-08-19T01:29:07.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:07 smithi103 ceph-mon[18436]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:07.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:07.951 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30533]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:29:07.951 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30533]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:07.951 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30533]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:07.951 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:29:07 smithi146 sudo[30533]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:08.032 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:07 smithi190 sudo[30631]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:29:08.032 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:07 smithi190 sudo[30631]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:08.032 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:07 smithi190 sudo[30631]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:08.033 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:07 smithi190 sudo[30631]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:08.381 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30634]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:29:08.381 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30634]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:08.381 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30634]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:08.381 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30634]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:08.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36679]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:29:08.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36679]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:08.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36679]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:08.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36679]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: osdmap e21: 3 total, 3 up, 3 in 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/318653712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b"}]: dispatch 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/318653712' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b"}]': finished 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: osdmap e22: 4 total, 3 up, 4 in 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:08.838 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 ceph-mon[22032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:08.839 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30637]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:29:08.839 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30637]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:08.839 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30637]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:08.839 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:08 smithi190 sudo[30637]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:08.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: osdmap e21: 3 total, 3 up, 3 in 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/318653712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b"}]: dispatch 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/318653712' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b"}]': finished 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: osdmap e22: 4 total, 3 up, 4 in 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:08.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:08 smithi146 ceph-mon[22098]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:09.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36682]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:29:09.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36682]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:09.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36682]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:09.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 sudo[36682]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:09.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: osdmap e21: 3 total, 3 up, 3 in 2024-08-19T01:29:09.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-19T01:29:09.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:09.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/318653712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b"}]: dispatch 2024-08-19T01:29:09.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/318653712' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b"}]': finished 2024-08-19T01:29:09.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: osdmap e22: 4 total, 3 up, 4 in 2024-08-19T01:29:09.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:09.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:09.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:08 smithi103 ceph-mon[18436]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 sudo[30640]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:29:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 sudo[30640]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 sudo[30640]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 sudo[30640]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:09.571 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30536]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-19T01:29:09.572 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30536]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:09.572 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30536]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:09.572 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30536]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:09.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:09.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:09.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/299930211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:09.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:09.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:09 smithi190 ceph-mon[22032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:09.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30539]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-19T01:29:09.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30539]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-19T01:29:09.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30539]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-19T01:29:09.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 sudo[30539]: pam_unix(sudo:session): session closed for user root 2024-08-19T01:29:09.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/299930211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:09.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:09.953 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:09 smithi146 ceph-mon[22098]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:10.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:10.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:10.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/299930211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:10.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:10.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:10.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:10.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:10.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:10.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:10.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:10.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:10.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-19T01:29:10.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-19T01:29:10.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-19T01:29:10.056 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-19T01:29:10.057 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:09 smithi103 ceph-mon[18436]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-19T01:29:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:11 smithi103 ceph-mon[18436]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:11 smithi190 ceph-mon[22032]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:11 smithi146 ceph-mon[22098]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:14.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:13 smithi103 ceph-mon[18436]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:14.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:13 smithi190 ceph-mon[22032]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:13 smithi146 ceph-mon[22098]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:16.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:15 smithi103 ceph-mon[18436]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-08-19T01:29:16.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-19T01:29:16.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:15 smithi190 ceph-mon[22032]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-08-19T01:29:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-19T01:29:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:15 smithi146 ceph-mon[22098]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-08-19T01:29:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-19T01:29:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:17.017 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:16 smithi103 ceph-mon[18436]: Deploying daemon osd.3 on smithi103 2024-08-19T01:29:17.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:16 smithi190 ceph-mon[22032]: Deploying daemon osd.3 on smithi103 2024-08-19T01:29:17.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:16 smithi146 ceph-mon[22098]: Deploying daemon osd.3 on smithi103 2024-08-19T01:29:17.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:17 smithi103 ceph-mon[18436]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-19T01:29:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:17 smithi190 ceph-mon[22032]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-19T01:29:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:17 smithi146 ceph-mon[22098]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-19T01:29:19.925 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:19 smithi103 ceph-mon[18436]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-19T01:29:20.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:19 smithi190 ceph-mon[22032]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-19T01:29:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:19 smithi146 ceph-mon[22098]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-19T01:29:21.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:21.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:21.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:21.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:21.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:21.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:21.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:21 smithi103 ceph-mon[18436]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-19T01:29:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:21 smithi190 ceph-mon[22032]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-19T01:29:22.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:21 smithi146 ceph-mon[22098]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-19T01:29:23.719 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 3 on host 'smithi103' 2024-08-19T01:29:23.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.717+0000 7f3259ffb640 1 -- 172.21.15.103:0/2845062919 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f325c1a0240 con 0x7f324c0a29d0 2024-08-19T01:29:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f324c0a29d0 msgr2=0x7f324c0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:23.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f324c0a29d0 0x7f324c0a4e90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3250004770 tx=0x7f3250000bc0 comp rx=0 tx=0).stop 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f325c1023c0 msgr2=0x7f325c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f325c1023c0 0x7f325c19f110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f325400ef10 comp rx=0 tx=0).stop 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 shutdown_connections 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f324c0a29d0 0x7f324c0a4e90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f325c1023c0 0x7f325c19f110 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f325c101a00 0x7f325c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 --2- 172.21.15.103:0/2845062919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f325c100800 0x7f325c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 >> 172.21.15.103:0/2845062919 conn(0x7f325c0fbfb0 msgr2=0x7f325c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 shutdown_connections 2024-08-19T01:29:23.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:29:23.719+0000 7f3262f70640 1 -- 172.21.15.103:0/2845062919 wait complete. 2024-08-19T01:29:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:23 smithi103 ceph-mon[18436]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:29:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:24.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:23 smithi190 ceph-mon[22032]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:29:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:23 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:23 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:23 smithi146 ceph-mon[22098]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:29:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:23 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:23 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:24.417 DEBUG:teuthology.orchestra.run.smithi103:osd.3> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.3.service 2024-08-19T01:29:24.420 INFO:tasks.cephadm:Deploying osd.4 on smithi146 with /dev/vg_nvme/lv_3... 2024-08-19T01:29:24.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-19T01:29:24.679 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:29:25.049 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:24 smithi146 ceph-mon[22098]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-19T01:29:25.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:24 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:25.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:24 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:25.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:24 smithi103 ceph-mon[18436]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-19T01:29:25.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:24 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:25.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:24 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:25.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:24 smithi190 ceph-mon[22032]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-19T01:29:25.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:24 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:25.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:24 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:26.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:25 smithi103 ceph-mon[18436]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:29:26.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:25 smithi103 ceph-mon[18436]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-19T01:29:26.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:25 smithi103 ceph-mon[18436]: osdmap e23: 4 total, 3 up, 4 in 2024-08-19T01:29:26.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:25 smithi103 ceph-mon[18436]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:29:26.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:25 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:26.055 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:29:25 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3[38661]: 2024-08-19T01:29:25.816+0000 7f7f9f1ae640 -1 osd.3 0 waiting for initial osdmap 2024-08-19T01:29:26.055 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:29:25 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3[38661]: 2024-08-19T01:29:25.822+0000 7f7f9afd6640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:29:26.113 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:25 smithi146 ceph-mon[22098]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:29:26.113 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:25 smithi146 ceph-mon[22098]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-19T01:29:26.113 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:25 smithi146 ceph-mon[22098]: osdmap e23: 4 total, 3 up, 4 in 2024-08-19T01:29:26.114 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:25 smithi146 ceph-mon[22098]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:29:26.114 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:25 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:26.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:25 smithi190 ceph-mon[22032]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-19T01:29:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:25 smithi190 ceph-mon[22032]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-19T01:29:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:25 smithi190 ceph-mon[22032]: osdmap e23: 4 total, 3 up, 4 in 2024-08-19T01:29:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:25 smithi190 ceph-mon[22032]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:29:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:25 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:26 smithi103 ceph-mon[18436]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:29:27.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:26 smithi103 ceph-mon[18436]: osdmap e24: 4 total, 3 up, 4 in 2024-08-19T01:29:27.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:26 smithi103 ceph-mon[18436]: osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] boot 2024-08-19T01:29:27.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:26 smithi103 ceph-mon[18436]: osdmap e25: 4 total, 4 up, 4 in 2024-08-19T01:29:27.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:26 smithi146 ceph-mon[22098]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:29:27.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:26 smithi146 ceph-mon[22098]: osdmap e24: 4 total, 3 up, 4 in 2024-08-19T01:29:27.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:26 smithi146 ceph-mon[22098]: osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] boot 2024-08-19T01:29:27.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:26 smithi146 ceph-mon[22098]: osdmap e25: 4 total, 4 up, 4 in 2024-08-19T01:29:27.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:26 smithi190 ceph-mon[22032]: from='osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:29:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:26 smithi190 ceph-mon[22032]: osdmap e24: 4 total, 3 up, 4 in 2024-08-19T01:29:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:26 smithi190 ceph-mon[22032]: osd.3 [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] boot 2024-08-19T01:29:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:26 smithi190 ceph-mon[22032]: osdmap e25: 4 total, 4 up, 4 in 2024-08-19T01:29:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-19T01:29:27.347 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:29:27.373 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_3 2024-08-19T01:29:27.631 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:29:27.948 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:27 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:29:27.948 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:27 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:29:27.948 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:27 smithi146 ceph-mon[22098]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:27.948 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:27 smithi146 ceph-mon[22098]: osdmap e26: 4 total, 4 up, 4 in 2024-08-19T01:29:28.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:27 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:29:28.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:27 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:29:28.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:27 smithi103 ceph-mon[18436]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:28.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:27 smithi103 ceph-mon[18436]: osdmap e26: 4 total, 4 up, 4 in 2024-08-19T01:29:28.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 -- 172.21.15.146:0/3104491435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff660107630 msgr2=0x7ff660109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:28.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 --2- 172.21.15.146:0/3104491435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff660107630 0x7ff660109a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff658009ca0 tx=0x7ff65802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:29:28.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 -- 172.21.15.146:0/3104491435 shutdown_connections 2024-08-19T01:29:28.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 --2- 172.21.15.146:0/3104491435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff660107630 0x7ff660109a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:28.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 --2- 172.21.15.146:0/3104491435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff660104d00 0x7ff6601070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:28.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 --2- 172.21.15.146:0/3104491435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6601023d0 0x7ff6601047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 -- 172.21.15.146:0/3104491435 >> 172.21.15.146:0/3104491435 conn(0x7ff6600fbfd0 msgr2=0x7ff6600fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 -- 172.21.15.146:0/3104491435 shutdown_connections 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.032+0000 7ff667548640 1 -- 172.21.15.146:0/3104491435 wait complete. 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 Processor -- start 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 -- start start 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6601023d0 0x7ff660101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff660104d00 0x7ff660100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff660107630 0x7ff6601008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff66010d4a0 con 0x7ff660104d00 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff66010d320 con 0x7ff6601023d0 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff667548640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff66010d620 con 0x7ff660107630 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff6652e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6601023d0 0x7ff660101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:28.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff6652e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6601023d0 0x7ff660101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36476/0 (socket says 172.21.15.146:36476) 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff6652e4640 1 -- 172.21.15.146:0/1992074961 learned_addr learned my addr 172.21.15.146:0/1992074961 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff665ae5640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff660107630 0x7ff6601008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.033+0000 7ff664ae3640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff660104d00 0x7ff660100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6652e4640 1 -- 172.21.15.146:0/1992074961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff660107630 msgr2=0x7ff6601008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6652e4640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff660107630 0x7ff6601008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6652e4640 1 -- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff660104d00 msgr2=0x7ff660100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6652e4640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff660104d00 0x7ff660100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6652e4640 1 -- 172.21.15.146:0/1992074961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff660101180 con 0x7ff6601023d0 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6652e4640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6601023d0 0x7ff660101d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9f0 tx=0x7ff65400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6527fc640 1 -- 172.21.15.146:0/1992074961 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65400cd80 con 0x7ff6601023d0 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6601a1cc0 con 0x7ff6601023d0 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6527fc640 1 -- 172.21.15.146:0/1992074961 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6540108e0 con 0x7ff6601023d0 2024-08-19T01:29:28.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff6527fc640 1 -- 172.21.15.146:0/1992074961 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654018920 con 0x7ff6601023d0 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.034+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6601a2170 con 0x7ff6601023d0 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.035+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6600fdac0 con 0x7ff6601023d0 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.041+0000 7ff6527fc640 1 -- 172.21.15.146:0/1992074961 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff654010a90 con 0x7ff6601023d0 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.042+0000 7ff6527fc640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff64c0a2b80 0x7ff64c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.042+0000 7ff6527fc640 1 -- 172.21.15.146:0/1992074961 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7ff6540ce660 con 0x7ff6601023d0 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.042+0000 7ff664ae3640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff64c0a2b80 0x7ff64c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.042+0000 7ff664ae3640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff64c0a2b80 0x7ff64c0a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff660101ab0 tx=0x7ff65c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:28.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.045+0000 7ff6527fc640 1 -- 172.21.15.146:0/1992074961 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff65409c0d0 con 0x7ff6601023d0 2024-08-19T01:29:28.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:27 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:29:28.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:27 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:29:28.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:27 smithi190 ceph-mon[22032]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-19T01:29:28.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:27 smithi190 ceph-mon[22032]: osdmap e26: 4 total, 4 up, 4 in 2024-08-19T01:29:28.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:28.230+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff660062360 con 0x7ff64c0a2b80 2024-08-19T01:29:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: osdmap e27: 4 total, 4 up, 4 in 2024-08-19T01:29:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi103 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi103 to 8544M 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:29.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.203 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:28 smithi146 ceph-mon[22098]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: osdmap e27: 4 total, 4 up, 4 in 2024-08-19T01:29:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi103 2024-08-19T01:29:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi103 to 8544M 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:28 smithi103 ceph-mon[18436]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: osdmap e27: 4 total, 4 up, 4 in 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi103 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi103 to 8544M 2024-08-19T01:29:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:29.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:28 smithi190 ceph-mon[22032]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:30 smithi146 ceph-mon[22098]: osdmap e28: 4 total, 4 up, 4 in 2024-08-19T01:29:30.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:30 smithi103 ceph-mon[18436]: osdmap e28: 4 total, 4 up, 4 in 2024-08-19T01:29:30.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:30 smithi190 ceph-mon[22032]: osdmap e28: 4 total, 4 up, 4 in 2024-08-19T01:29:31.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:31 smithi146 ceph-mon[22098]: osdmap e29: 4 total, 4 up, 4 in 2024-08-19T01:29:31.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:31 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/2795191676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]: dispatch 2024-08-19T01:29:31.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:31 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]: dispatch 2024-08-19T01:29:31.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:31 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]': finished 2024-08-19T01:29:31.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:31 smithi146 ceph-mon[22098]: osdmap e30: 5 total, 4 up, 5 in 2024-08-19T01:29:31.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:31.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:31 smithi146 ceph-mon[22098]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-08-19T01:29:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:31 smithi103 ceph-mon[18436]: osdmap e29: 4 total, 4 up, 4 in 2024-08-19T01:29:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:31 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/2795191676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]: dispatch 2024-08-19T01:29:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:31 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]: dispatch 2024-08-19T01:29:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:31 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]': finished 2024-08-19T01:29:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:31 smithi103 ceph-mon[18436]: osdmap e30: 5 total, 4 up, 5 in 2024-08-19T01:29:31.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:31.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:31 smithi103 ceph-mon[18436]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-08-19T01:29:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:31 smithi190 ceph-mon[22032]: osdmap e29: 4 total, 4 up, 4 in 2024-08-19T01:29:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:31 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/2795191676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]: dispatch 2024-08-19T01:29:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:31 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]: dispatch 2024-08-19T01:29:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:31 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d80a218-c821-4a4c-a133-ec98bf694692"}]': finished 2024-08-19T01:29:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:31 smithi190 ceph-mon[22032]: osdmap e30: 5 total, 4 up, 5 in 2024-08-19T01:29:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:31 smithi190 ceph-mon[22032]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-08-19T01:29:32.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:32 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/3298475747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:32.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:32 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/3298475747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:32.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:32 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/3298475747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:33.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:33 smithi146 ceph-mon[22098]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:29:33.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:33 smithi103 ceph-mon[18436]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:29:33.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:33 smithi190 ceph-mon[22032]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:29:35.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:35 smithi146 ceph-mon[22098]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:29:36.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:35 smithi103 ceph-mon[18436]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:29:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:35 smithi190 ceph-mon[22032]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:29:38.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:37 smithi103 ceph-mon[18436]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-19T01:29:38.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-19T01:29:38.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:38.088 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:37 smithi146 ceph-mon[22098]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-19T01:29:38.088 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-19T01:29:38.088 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:38.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:37 smithi190 ceph-mon[22032]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-19T01:29:38.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-19T01:29:38.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:38.939 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:38 smithi146 ceph-mon[22098]: Deploying daemon osd.4 on smithi146 2024-08-19T01:29:39.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:38 smithi103 ceph-mon[18436]: Deploying daemon osd.4 on smithi146 2024-08-19T01:29:39.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:38 smithi190 ceph-mon[22032]: Deploying daemon osd.4 on smithi146 2024-08-19T01:29:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:39 smithi103 ceph-mon[18436]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:40.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:39 smithi190 ceph-mon[22032]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:40.136 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:39 smithi146 ceph-mon[22098]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:42.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:41 smithi103 ceph-mon[18436]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:42.061 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:41 smithi146 ceph-mon[22098]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:42.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:41 smithi190 ceph-mon[22032]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:42.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:42.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:42.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:43.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:43.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:43.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:43 smithi103 ceph-mon[18436]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:44.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:43 smithi190 ceph-mon[22032]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:44.156 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:43 smithi146 ceph-mon[22098]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:45 smithi103 ceph-mon[18436]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:45 smithi190 ceph-mon[22032]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:45 smithi146 ceph-mon[22098]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.254+0000 7ff6527fc640 1 -- 172.21.15.146:0/1992074961 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff660062360 con 0x7ff64c0a2b80 2024-08-19T01:29:46.257 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 4 on host 'smithi146' 2024-08-19T01:29:46.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.256+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff64c0a2b80 msgr2=0x7ff64c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.256+0000 7ff667548640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff64c0a2b80 0x7ff64c0a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff660101ab0 tx=0x7ff65c00b390 comp rx=0 tx=0).stop 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.256+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6601023d0 msgr2=0x7ff660101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6601023d0 0x7ff660101d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9f0 tx=0x7ff65400eeb0 comp rx=0 tx=0).stop 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 shutdown_connections 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff64c0a2b80 0x7ff64c0a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff660107630 0x7ff6601008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff660104d00 0x7ff660100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 --2- 172.21.15.146:0/1992074961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6601023d0 0x7ff660101d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 >> 172.21.15.146:0/1992074961 conn(0x7ff6600fbfd0 msgr2=0x7ff660102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 shutdown_connections 2024-08-19T01:29:46.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:29:46.257+0000 7ff667548640 1 -- 172.21.15.146:0/1992074961 wait complete. 2024-08-19T01:29:46.844 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:46 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:46.844 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:46 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.844 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:46 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:46.844 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:46 smithi146 ceph-mon[22098]: from='osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:29:46.844 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:46 smithi146 ceph-mon[22098]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:29:46.871 DEBUG:teuthology.orchestra.run.smithi146:osd.4> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.4.service 2024-08-19T01:29:46.874 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2024-08-19T01:29:46.874 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-19T01:29:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:46 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:46 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:46 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:46 smithi103 ceph-mon[18436]: from='osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:29:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:46 smithi103 ceph-mon[18436]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:29:47.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:46 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:29:47.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:46 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:47.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:46 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:47.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:46 smithi190 ceph-mon[22032]: from='osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:29:47.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:46 smithi190 ceph-mon[22032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-19T01:29:47.132 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:29:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:47 smithi190 ceph-mon[22032]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:47 smithi190 ceph-mon[22032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-19T01:29:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:47 smithi190 ceph-mon[22032]: from='osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:29:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:47 smithi190 ceph-mon[22032]: osdmap e31: 5 total, 4 up, 5 in 2024-08-19T01:29:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:47 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:47 smithi190 ceph-mon[22032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:29:48.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:47 smithi146 ceph-mon[22098]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:48.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:47 smithi146 ceph-mon[22098]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-19T01:29:48.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:47 smithi146 ceph-mon[22098]: from='osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:29:48.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:47 smithi146 ceph-mon[22098]: osdmap e31: 5 total, 4 up, 5 in 2024-08-19T01:29:48.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:47 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:48.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:47 smithi146 ceph-mon[22098]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:29:48.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:47 smithi103 ceph-mon[18436]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:48.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:47 smithi103 ceph-mon[18436]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-19T01:29:48.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:47 smithi103 ceph-mon[18436]: from='osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:29:48.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:47 smithi103 ceph-mon[18436]: osdmap e31: 5 total, 4 up, 5 in 2024-08-19T01:29:48.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:47 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:48.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:47 smithi103 ceph-mon[18436]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:29:48.371 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:29:48 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4[33286]: 2024-08-19T01:29:48.268+0000 7fd780fee640 -1 osd.4 0 waiting for initial osdmap 2024-08-19T01:29:48.371 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:29:48 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4[33286]: 2024-08-19T01:29:48.274+0000 7fd77ce16640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:29:49.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:49 smithi146 ceph-mon[22098]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:29:49.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:49 smithi146 ceph-mon[22098]: osdmap e32: 5 total, 4 up, 5 in 2024-08-19T01:29:49.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:49.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:49.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:49 smithi146 ceph-mon[22098]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:49.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:49 smithi103 ceph-mon[18436]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:29:49.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:49 smithi103 ceph-mon[18436]: osdmap e32: 5 total, 4 up, 5 in 2024-08-19T01:29:49.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:49.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:49.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:49 smithi103 ceph-mon[18436]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:49 smithi190 ceph-mon[22032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:29:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:49 smithi190 ceph-mon[22032]: osdmap e32: 5 total, 4 up, 5 in 2024-08-19T01:29:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:49 smithi190 ceph-mon[22032]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-19T01:29:49.648 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:29:49.679 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2024-08-19T01:29:49.936 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:29:50.350 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:29:50.350 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:29:50.350 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] boot 2024-08-19T01:29:50.351 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: osdmap e33: 5 total, 5 up, 5 in 2024-08-19T01:29:50.351 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:50.351 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi146 2024-08-19T01:29:50.351 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.351 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.351 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:50.352 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:50.352 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi146 to 8544M 2024-08-19T01:29:50.352 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.352 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:50.352 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:50.352 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 -- 172.21.15.190:0/1442347862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 msgr2=0x7f8fac1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:50.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/1442347862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac1025d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009900 tx=0x7f8fa802a410 comp rx=0 tx=0).stop 2024-08-19T01:29:50.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 -- 172.21.15.190:0/1442347862 shutdown_connections 2024-08-19T01:29:50.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/1442347862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac1054c0 0x7f8fac105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:50.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/1442347862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac102b10 0x7f8fac104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:50.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/1442347862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac1025d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:50.435 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 -- 172.21.15.190:0/1442347862 >> 172.21.15.190:0/1442347862 conn(0x7f8fac0fbfd0 msgr2=0x7f8fac0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 -- 172.21.15.190:0/1442347862 shutdown_connections 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 -- 172.21.15.190:0/1442347862 wait complete. 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 Processor -- start 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.431+0000 7f8fb3e77640 1 -- start start 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb3e77640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb3e77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102b10 0x7f8fac19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb3e77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac1054c0 0x7f8fac1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb3e77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac113ad0 con 0x7f8fac102b10 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb3e77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac113950 con 0x7f8fac1054c0 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb3e77640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac113c50 con 0x7f8fac1001c0 2024-08-19T01:29:50.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac19e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:46214/0 (socket says 172.21.15.190:46214) 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 -- 172.21.15.190:0/3542805766 learned_addr learned my addr 172.21.15.190:0/3542805766 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1412640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102b10 0x7f8fac19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb2414640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac1054c0 0x7f8fac1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 -- 172.21.15.190:0/3542805766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac1054c0 msgr2=0x7f8fac1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac1054c0 0x7f8fac1a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 -- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102b10 msgr2=0x7f8fac19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102b10 0x7f8fac19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 -- 172.21.15.190:0/3542805766 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac1a5d70 con 0x7f8fac1001c0 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb1c13640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac19e140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80314e0 tx=0x7f8fa8003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fa2ffd640 1 -- 172.21.15.190:0/3542805766 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa803f070 con 0x7f8fac1001c0 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac1a6000 con 0x7f8fac1001c0 2024-08-19T01:29:50.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fa2ffd640 1 -- 172.21.15.190:0/3542805766 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa8031e40 con 0x7f8fac1001c0 2024-08-19T01:29:50.438 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.432+0000 7f8fa2ffd640 1 -- 172.21.15.190:0/3542805766 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8035030 con 0x7f8fac1001c0 2024-08-19T01:29:50.438 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.433+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1a64e0 con 0x7f8fac1001c0 2024-08-19T01:29:50.439 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.433+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fac0fdac0 con 0x7f8fac1001c0 2024-08-19T01:29:50.439 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.435+0000 7f8fa2ffd640 1 -- 172.21.15.190:0/3542805766 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8fa8042070 con 0x7f8fac1001c0 2024-08-19T01:29:50.439 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.435+0000 7f8fa2ffd640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8f980a2bb0 0x7f8f980a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:29:50.439 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.436+0000 7f8fb1412640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8f980a2bb0 0x7f8f980a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:29:50.439 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.436+0000 7f8fa2ffd640 1 -- 172.21.15.190:0/3542805766 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f8fa80ee820 con 0x7f8fac1001c0 2024-08-19T01:29:50.439 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.436+0000 7f8fb1412640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8f980a2bb0 0x7f8f980a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fac19f660 tx=0x7f8f9c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:29:50.441 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.440+0000 7f8fa2ffd640 1 -- 172.21.15.190:0/3542805766 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8fa80bc1a0 con 0x7f8fac1001c0 2024-08-19T01:29:50.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:29:50.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:29:50.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] boot 2024-08-19T01:29:50.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: osdmap e33: 5 total, 5 up, 5 in 2024-08-19T01:29:50.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:50.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi146 2024-08-19T01:29:50.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi146 to 8544M 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:50.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:29:50.612+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8fac062360 con 0x7f8f980a2bb0 2024-08-19T01:29:50.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: osd.4 [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] boot 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: osdmap e33: 5 total, 5 up, 5 in 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi146 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi146 to 8544M 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:50.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:50.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:29:50.703 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:29:51.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:51 smithi103 ceph-mon[18436]: osdmap e34: 5 total, 5 up, 5 in 2024-08-19T01:29:51.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:51 smithi103 ceph-mon[18436]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:51.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:51.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:51.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:51.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:51 smithi103 ceph-mon[18436]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:51.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:51 smithi190 ceph-mon[22032]: osdmap e34: 5 total, 5 up, 5 in 2024-08-19T01:29:51.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:51 smithi190 ceph-mon[22032]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:51.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:51.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:51.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:51.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:51 smithi190 ceph-mon[22032]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:51.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:51 smithi146 ceph-mon[22098]: osdmap e34: 5 total, 5 up, 5 in 2024-08-19T01:29:51.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:51 smithi146 ceph-mon[22098]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:29:51.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:29:51.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:29:51.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:29:51.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:51 smithi146 ceph-mon[22098]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:53 smithi190 ceph-mon[22032]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:53 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/3185732152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]: dispatch 2024-08-19T01:29:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:53 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]: dispatch 2024-08-19T01:29:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:53 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]': finished 2024-08-19T01:29:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:53 smithi190 ceph-mon[22032]: osdmap e35: 6 total, 5 up, 6 in 2024-08-19T01:29:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:53.882 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:53 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/2361508561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:53 smithi103 ceph-mon[18436]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:53 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/3185732152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]: dispatch 2024-08-19T01:29:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:53 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]: dispatch 2024-08-19T01:29:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:53 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]': finished 2024-08-19T01:29:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:53 smithi103 ceph-mon[18436]: osdmap e35: 6 total, 5 up, 6 in 2024-08-19T01:29:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:53 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/2361508561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:53 smithi146 ceph-mon[22098]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:53 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/3185732152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]: dispatch 2024-08-19T01:29:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:53 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]: dispatch 2024-08-19T01:29:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:53 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a36b0adc-8fec-4313-8298-55fc4236a809"}]': finished 2024-08-19T01:29:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:53 smithi146 ceph-mon[22098]: osdmap e35: 6 total, 5 up, 6 in 2024-08-19T01:29:54.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:29:54.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:53 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/2361508561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:29:56.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:55 smithi103 ceph-mon[18436]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:56.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:55 smithi190 ceph-mon[22032]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:55 smithi146 ceph-mon[22098]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:57.942 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:57 smithi190 ceph-mon[22032]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:58.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:57 smithi103 ceph-mon[18436]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:29:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:57 smithi146 ceph-mon[22098]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:29:59 smithi103 ceph-mon[18436]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:00.120 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:29:59 smithi190 ceph-mon[22032]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:29:59 smithi146 ceph-mon[22098]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:00.857 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:00 smithi190 ceph-mon[22032]: overall HEALTH_OK 2024-08-19T01:30:00.857 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-19T01:30:00.857 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:00 smithi103 ceph-mon[18436]: overall HEALTH_OK 2024-08-19T01:30:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-19T01:30:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:00 smithi146 ceph-mon[22098]: overall HEALTH_OK 2024-08-19T01:30:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-19T01:30:01.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:01 smithi190 ceph-mon[22032]: Deploying daemon osd.5 on smithi190 2024-08-19T01:30:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:01 smithi190 ceph-mon[22032]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:02.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:01 smithi103 ceph-mon[18436]: Deploying daemon osd.5 on smithi190 2024-08-19T01:30:02.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:01 smithi103 ceph-mon[18436]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:01 smithi146 ceph-mon[22098]: Deploying daemon osd.5 on smithi190 2024-08-19T01:30:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:01 smithi146 ceph-mon[22098]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:03.839 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:03 smithi190 ceph-mon[22032]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:04.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:03 smithi103 ceph-mon[18436]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:03 smithi146 ceph-mon[22098]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:05.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:05 smithi190 ceph-mon[22032]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:05.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:05.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:05.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:05 smithi146 ceph-mon[22098]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:05 smithi103 ceph-mon[18436]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:08.054 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:07 smithi190 ceph-mon[22032]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:08.106 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:07 smithi103 ceph-mon[18436]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:07 smithi146 ceph-mon[22098]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:08.769 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.766+0000 7f8fa2ffd640 1 -- 172.21.15.190:0/3542805766 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8fac062360 con 0x7f8f980a2bb0 2024-08-19T01:30:08.769 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2024-08-19T01:30:08.770 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8f980a2bb0 msgr2=0x7f8f980a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:08.770 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8f980a2bb0 0x7f8f980a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fac19f660 tx=0x7f8f9c00b390 comp rx=0 tx=0).stop 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 msgr2=0x7f8fac19e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac19e140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80314e0 tx=0x7f8fa8003c80 comp rx=0 tx=0).stop 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 shutdown_connections 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8f980a2bb0 0x7f8f980a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fac1054c0 0x7f8fac1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fac102b10 0x7f8fac19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 --2- 172.21.15.190:0/3542805766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fac1001c0 0x7f8fac19e140 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 >> 172.21.15.190:0/3542805766 conn(0x7f8fac0fbfd0 msgr2=0x7f8fac103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 shutdown_connections 2024-08-19T01:30:08.771 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:08.769+0000 7f8fb3e77640 1 -- 172.21.15.190:0/3542805766 wait complete. 2024-08-19T01:30:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:09.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:09.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.450 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.5.service 2024-08-19T01:30:09.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:09.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:09.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:09.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:09.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:09.453 INFO:tasks.cephadm:Deploying osd.6 on smithi103 with /dev/vg_nvme/lv_2... 2024-08-19T01:30:09.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-19T01:30:09.720 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:30:10.191 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:10 smithi103 ceph-mon[18436]: from='osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:30:10.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:10 smithi103 ceph-mon[18436]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:30:10.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:10 smithi190 ceph-mon[22032]: from='osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:30:10.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:10 smithi190 ceph-mon[22032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:30:10.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:10 smithi146 ceph-mon[22098]: from='osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:30:10.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:10 smithi146 ceph-mon[22098]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-19T01:30:11.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:11 smithi103 ceph-mon[18436]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-19T01:30:11.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:11 smithi103 ceph-mon[18436]: osdmap e36: 6 total, 5 up, 6 in 2024-08-19T01:30:11.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:11 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:11.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:11 smithi103 ceph-mon[18436]: from='osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:30:11.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:11 smithi103 ceph-mon[18436]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:30:11.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:11 smithi103 ceph-mon[18436]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:11.199 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-mon[22032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-19T01:30:11.199 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-mon[22032]: osdmap e36: 6 total, 5 up, 6 in 2024-08-19T01:30:11.199 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:11.199 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-mon[22032]: from='osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:30:11.199 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-mon[22032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:30:11.199 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-mon[22032]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:11.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:11 smithi146 ceph-mon[22098]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-19T01:30:11.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:11 smithi146 ceph-mon[22098]: osdmap e36: 6 total, 5 up, 6 in 2024-08-19T01:30:11.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:11 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:11.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:11 smithi146 ceph-mon[22098]: from='osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:30:11.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:11 smithi146 ceph-mon[22098]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:30:11.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:11 smithi146 ceph-mon[22098]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-19T01:30:11.767 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5[33435]: 2024-08-19T01:30:11.482+0000 7f2b4bf66640 -1 osd.5 0 waiting for initial osdmap 2024-08-19T01:30:11.768 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:30:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5[33435]: 2024-08-19T01:30:11.489+0000 7f2b43d86640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:30:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:12 smithi103 ceph-mon[18436]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:30:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:12 smithi103 ceph-mon[18436]: osdmap e37: 6 total, 5 up, 6 in 2024-08-19T01:30:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:12 smithi103 ceph-mon[18436]: from='osd.5 ' entity='osd.5' 2024-08-19T01:30:12.307 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:30:12.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-08-19T01:30:12.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:12 smithi146 ceph-mon[22098]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:30:12.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:12 smithi146 ceph-mon[22098]: osdmap e37: 6 total, 5 up, 6 in 2024-08-19T01:30:12.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:12 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:12 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:12 smithi146 ceph-mon[22098]: from='osd.5 ' entity='osd.5' 2024-08-19T01:30:12.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:12 smithi190 ceph-mon[22032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:30:12.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:12 smithi190 ceph-mon[22032]: osdmap e37: 6 total, 5 up, 6 in 2024-08-19T01:30:12.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:12 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:12.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:12 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:12.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:12 smithi190 ceph-mon[22032]: from='osd.5 ' entity='osd.5' 2024-08-19T01:30:12.598 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:30:13.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 -- 172.21.15.103:0/2214319836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 msgr2=0x7f8c80100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 --2- 172.21.15.103:0/2214319836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c80100c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009900 tx=0x7f8c7002a410 comp rx=0 tx=0).stop 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 -- 172.21.15.103:0/2214319836 shutdown_connections 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 --2- 172.21.15.103:0/2214319836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80102400 0x7f8c80108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 --2- 172.21.15.103:0/2214319836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c80101a40 0x7f8c80101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 --2- 172.21.15.103:0/2214319836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c80100c40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 -- 172.21.15.103:0/2214319836 >> 172.21.15.103:0/2214319836 conn(0x7f8c800fbfd0 msgr2=0x7f8c800fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 -- 172.21.15.103:0/2214319836 shutdown_connections 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 -- 172.21.15.103:0/2214319836 wait complete. 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 Processor -- start 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.076+0000 7f8c84e08640 1 -- start start 2024-08-19T01:30:13.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c84e08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c8006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c84e08640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c80101a40 0x7f8c8019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c84e08640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80102400 0x7f8c8019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c84e08640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8010c370 con 0x7f8c80100840 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c84e08640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8010c1f0 con 0x7f8c80102400 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c84e08640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8010c4f0 con 0x7f8c80101a40 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c8006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c8006b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38080/0 (socket says 172.21.15.103:38080) 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 -- 172.21.15.103:0/3414501585 learned_addr learned my addr 172.21.15.103:0/3414501585 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7ed9d640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80102400 0x7f8c8019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c77fff640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c80101a40 0x7f8c8019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 -- 172.21.15.103:0/3414501585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c80101a40 msgr2=0x7f8c8019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:13.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c80101a40 0x7f8c8019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 -- 172.21.15.103:0/3414501585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80102400 msgr2=0x7f8c8019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80102400 0x7f8c8019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 -- 172.21.15.103:0/3414501585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8019f840 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.077+0000 7f8c7e59c640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c8006b020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c700314e0 tx=0x7f8c70031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.078+0000 7f8c777fe640 1 -- 172.21.15.103:0/3414501585 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7003f030 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.078+0000 7f8c777fe640 1 -- 172.21.15.103:0/3414501585 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c700044b0 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.078+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8019fa70 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.078+0000 7f8c777fe640 1 -- 172.21.15.103:0/3414501585 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70035070 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.078+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8019ff50 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.079+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c80107d80 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.080+0000 7f8c777fe640 1 -- 172.21.15.103:0/3414501585 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8c70042030 con 0x7f8c80100840 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.080+0000 7f8c777fe640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.080+0000 7f8c77fff640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:13.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.081+0000 7f8c777fe640 1 -- 172.21.15.103:0/3414501585 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4562+0+0 (secure 0 0 0) 0x7f8c700ee710 con 0x7f8c80100840 2024-08-19T01:30:13.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.081+0000 7f8c77fff640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c8006b5a0 tx=0x7f8c68005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:13.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.085+0000 7f8c777fe640 1 -- 172.21.15.103:0/3414501585 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8c700bc510 con 0x7f8c80100840 2024-08-19T01:30:13.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:13.262+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8c80062360 con 0x7f8c6c0a2b80 2024-08-19T01:30:13.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:30:13.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:30:13.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] boot 2024-08-19T01:30:13.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: osdmap e38: 6 total, 6 up, 6 in 2024-08-19T01:30:13.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:13.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi190 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi190 to 8544M 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:13 smithi103 ceph-mon[18436]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] boot 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: osdmap e38: 6 total, 6 up, 6 in 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi190 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:13.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi190 to 8544M 2024-08-19T01:30:13.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:13.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:13.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:13 smithi190 ceph-mon[22032]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:30:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:30:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: osd.5 [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] boot 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: osdmap e38: 6 total, 6 up, 6 in 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi190 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi190 to 8544M 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:13 smithi146 ceph-mon[22098]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:14 smithi103 ceph-mon[18436]: osdmap e39: 6 total, 6 up, 6 in 2024-08-19T01:30:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:14 smithi103 ceph-mon[18436]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:14 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:14 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:14 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:14.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:14 smithi190 ceph-mon[22032]: osdmap e39: 6 total, 6 up, 6 in 2024-08-19T01:30:14.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:14 smithi190 ceph-mon[22032]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:14.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:14 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:14.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:14 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:14.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:14 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:14 smithi146 ceph-mon[22098]: osdmap e39: 6 total, 6 up, 6 in 2024-08-19T01:30:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:14 smithi146 ceph-mon[22098]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:14 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:14 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:14 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:15.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:15 smithi103 ceph-mon[18436]: osdmap e40: 6 total, 6 up, 6 in 2024-08-19T01:30:15.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:15 smithi103 ceph-mon[18436]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:15.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:15 smithi190 ceph-mon[22032]: osdmap e40: 6 total, 6 up, 6 in 2024-08-19T01:30:15.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:15 smithi190 ceph-mon[22032]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:15.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:15 smithi146 ceph-mon[22098]: osdmap e40: 6 total, 6 up, 6 in 2024-08-19T01:30:15.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:15 smithi146 ceph-mon[22098]: pgmap v127: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:16.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:16 smithi103 ceph-mon[18436]: osdmap e41: 6 total, 6 up, 6 in 2024-08-19T01:30:16.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:16 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1512634247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "243118b8-58e2-40a3-833c-008ec861cab9"}]: dispatch 2024-08-19T01:30:16.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:16 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1512634247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "243118b8-58e2-40a3-833c-008ec861cab9"}]': finished 2024-08-19T01:30:16.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:16 smithi103 ceph-mon[18436]: osdmap e42: 7 total, 6 up, 7 in 2024-08-19T01:30:16.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:16.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:16 smithi190 ceph-mon[22032]: osdmap e41: 6 total, 6 up, 6 in 2024-08-19T01:30:16.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:16 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1512634247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "243118b8-58e2-40a3-833c-008ec861cab9"}]: dispatch 2024-08-19T01:30:16.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:16 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1512634247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "243118b8-58e2-40a3-833c-008ec861cab9"}]': finished 2024-08-19T01:30:16.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:16 smithi190 ceph-mon[22032]: osdmap e42: 7 total, 6 up, 7 in 2024-08-19T01:30:16.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:16.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:16 smithi146 ceph-mon[22098]: osdmap e41: 6 total, 6 up, 6 in 2024-08-19T01:30:16.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:16 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1512634247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "243118b8-58e2-40a3-833c-008ec861cab9"}]: dispatch 2024-08-19T01:30:16.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:16 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1512634247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "243118b8-58e2-40a3-833c-008ec861cab9"}]': finished 2024-08-19T01:30:16.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:16 smithi146 ceph-mon[22098]: osdmap e42: 7 total, 6 up, 7 in 2024-08-19T01:30:16.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:17 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2315635511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:17 smithi190 ceph-mon[22032]: pgmap v130: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:17 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2315635511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:17 smithi146 ceph-mon[22098]: pgmap v130: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:17.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:17 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2315635511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:17.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:17 smithi103 ceph-mon[18436]: pgmap v130: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:30:19.953 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:19 smithi103 ceph-mon[18436]: pgmap v131: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-08-19T01:30:20.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:19 smithi190 ceph-mon[22032]: pgmap v131: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-08-19T01:30:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:19 smithi146 ceph-mon[22098]: pgmap v131: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-08-19T01:30:22.035 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:21 smithi103 ceph-mon[18436]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-08-19T01:30:22.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:21 smithi190 ceph-mon[22032]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-08-19T01:30:22.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:21 smithi146 ceph-mon[22098]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-08-19T01:30:23.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-19T01:30:23.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:23.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-19T01:30:23.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:23.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-19T01:30:23.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:23.897 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:23 smithi103 ceph-mon[18436]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:23.897 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:23 smithi103 ceph-mon[18436]: Deploying daemon osd.6 on smithi103 2024-08-19T01:30:23.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:23 smithi146 ceph-mon[22098]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:23.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:23 smithi146 ceph-mon[22098]: Deploying daemon osd.6 on smithi103 2024-08-19T01:30:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:23 smithi190 ceph-mon[22032]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:23 smithi190 ceph-mon[22032]: Deploying daemon osd.6 on smithi103 2024-08-19T01:30:26.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:25 smithi103 ceph-mon[18436]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:26.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:25 smithi190 ceph-mon[22032]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:26.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:25 smithi146 ceph-mon[22098]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:28.068 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:27 smithi103 ceph-mon[18436]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:28.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:27 smithi190 ceph-mon[22032]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:27 smithi146 ceph-mon[22098]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:29.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:29.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:29.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:29.880 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:29 smithi103 ceph-mon[18436]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:30.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:29 smithi190 ceph-mon[22032]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:29 smithi146 ceph-mon[22098]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:31.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.561+0000 7f8c777fe640 1 -- 172.21.15.103:0/3414501585 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c80062360 con 0x7f8c6c0a2b80 2024-08-19T01:30:31.564 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 6 on host 'smithi103' 2024-08-19T01:30:31.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c6c0a2b80 msgr2=0x7f8c6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:31.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c8006b5a0 tx=0x7f8c68005f90 comp rx=0 tx=0).stop 2024-08-19T01:30:31.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 msgr2=0x7f8c8006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:31.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c8006b020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8c700314e0 tx=0x7f8c70031a30 comp rx=0 tx=0).stop 2024-08-19T01:30:31.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 shutdown_connections 2024-08-19T01:30:31.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:31.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c80102400 0x7f8c8019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:31.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c80101a40 0x7f8c8019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:31.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 --2- 172.21.15.103:0/3414501585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c80100840 0x7f8c8006b020 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:31.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 >> 172.21.15.103:0/3414501585 conn(0x7f8c800fbfd0 msgr2=0x7f8c80104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:31.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 shutdown_connections 2024-08-19T01:30:31.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:30:31.564+0000 7f8c84e08640 1 -- 172.21.15.103:0/3414501585 wait complete. 2024-08-19T01:30:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:31 smithi190 ceph-mon[22032]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:32.168 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:31 smithi103 ceph-mon[18436]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:32.168 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:32.168 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:32.196 DEBUG:teuthology.orchestra.run.smithi103:osd.6> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.6.service 2024-08-19T01:30:32.203 INFO:tasks.cephadm:Deploying osd.7 on smithi146 with /dev/vg_nvme/lv_2... 2024-08-19T01:30:32.204 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-19T01:30:32.204 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:31 smithi146 ceph-mon[22098]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:32.205 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:32.205 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:32.463 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:30:32.916 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:32 smithi146 ceph-mon[22098]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-19T01:30:32.917 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:32.917 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:33.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:32 smithi190 ceph-mon[22032]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-19T01:30:33.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:33.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:33.168 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:32 smithi103 ceph-mon[18436]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-19T01:30:33.168 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:33.168 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:34.081 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:33 smithi146 ceph-mon[22098]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:34.081 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:33 smithi146 ceph-mon[22098]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-19T01:30:34.081 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:33 smithi146 ceph-mon[22098]: osdmap e43: 7 total, 6 up, 7 in 2024-08-19T01:30:34.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:33 smithi146 ceph-mon[22098]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:30:34.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:34.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:33 smithi190 ceph-mon[22032]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:34.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:33 smithi190 ceph-mon[22032]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-19T01:30:34.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:33 smithi190 ceph-mon[22032]: osdmap e43: 7 total, 6 up, 7 in 2024-08-19T01:30:34.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:33 smithi190 ceph-mon[22032]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:30:34.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:34.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:33 smithi103 ceph-mon[18436]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:34.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:33 smithi103 ceph-mon[18436]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-19T01:30:34.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:33 smithi103 ceph-mon[18436]: osdmap e43: 7 total, 6 up, 7 in 2024-08-19T01:30:34.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:33 smithi103 ceph-mon[18436]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:30:34.200 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:34.200 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:30:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6[45101]: 2024-08-19T01:30:33.822+0000 7f1607895640 -1 osd.6 0 waiting for initial osdmap 2024-08-19T01:30:34.200 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:30:33 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6[45101]: 2024-08-19T01:30:33.829+0000 7f16036bd640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:30:34.968 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:34 smithi146 ceph-mon[22098]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:30:34.969 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:34 smithi146 ceph-mon[22098]: osdmap e44: 7 total, 6 up, 7 in 2024-08-19T01:30:34.969 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:34.969 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:35.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:34 smithi190 ceph-mon[22032]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:30:35.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:34 smithi190 ceph-mon[22032]: osdmap e44: 7 total, 6 up, 7 in 2024-08-19T01:30:35.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:35.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:35.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:34 smithi103 ceph-mon[18436]: from='osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:30:35.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:34 smithi103 ceph-mon[18436]: osdmap e44: 7 total, 6 up, 7 in 2024-08-19T01:30:35.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:35.146 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:35.288 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:30:35.315 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_2 2024-08-19T01:30:35.575 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:30:35.930 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:35 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:30:35.930 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:35 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:30:35.930 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:35 smithi146 ceph-mon[22098]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:35.930 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:35 smithi146 ceph-mon[22098]: osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] boot 2024-08-19T01:30:35.930 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:35 smithi146 ceph-mon[22098]: osdmap e45: 7 total, 7 up, 7 in 2024-08-19T01:30:35.930 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:36.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 -- 172.21.15.146:0/4247807910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8103910 msgr2=0x7f77f80ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/4247807910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8103910 0x7f77f80ff9c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009900 tx=0x7f77ec02a410 comp rx=0 tx=0).stop 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 -- 172.21.15.146:0/4247807910 shutdown_connections 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/4247807910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f8100060 0x7f77f8109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/4247807910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8103910 0x7f77f80ff9c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/4247807910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f8102f40 0x7f77f8103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 -- 172.21.15.146:0/4247807910 >> 172.21.15.146:0/4247807910 conn(0x7f77f80fb770 msgr2=0x7f77f80fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 -- 172.21.15.146:0/4247807910 shutdown_connections 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.012+0000 7f77fd5f6640 1 -- 172.21.15.146:0/4247807910 wait complete. 2024-08-19T01:30:36.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 Processor -- start 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 -- start start 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f8100060 0x7f77f8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8102f40 0x7f77f81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f8103910 0x7f77f819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77f810d460 con 0x7f77f8100060 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77f810d2e0 con 0x7f77f8102f40 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77fd5f6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f77f810d5e0 con 0x7f77f8103910 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77f67fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8102f40 0x7f77f81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77f67fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8102f40 0x7f77f81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55174/0 (socket says 172.21.15.146:55174) 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77f67fc640 1 -- 172.21.15.146:0/153882079 learned_addr learned my addr 172.21.15.146:0/153882079 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77f6ffd640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f8100060 0x7f77f8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.013+0000 7f77f77fe640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f8103910 0x7f77f819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77f67fc640 1 -- 172.21.15.146:0/153882079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f8103910 msgr2=0x7f77f819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77f67fc640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f8103910 0x7f77f819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77f67fc640 1 -- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f8100060 msgr2=0x7f77f8197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77f67fc640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f8100060 0x7f77f8197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77f67fc640 1 -- 172.21.15.146:0/153882079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f819f7e0 con 0x7f77f8102f40 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77f67fc640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8102f40 0x7f77f81980f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0314e0 tx=0x7f77ec0042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77dffff640 1 -- 172.21.15.146:0/153882079 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec03f030 con 0x7f77f8102f40 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f819fa70 con 0x7f77f8102f40 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77dffff640 1 -- 172.21.15.146:0/153882079 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77ec003cb0 con 0x7f77f8102f40 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77dffff640 1 -- 172.21.15.146:0/153882079 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec035070 con 0x7f77f8102f40 2024-08-19T01:30:36.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.014+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f819ffa0 con 0x7f77f8102f40 2024-08-19T01:30:36.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.015+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f8108ff0 con 0x7f77f8102f40 2024-08-19T01:30:36.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.016+0000 7f77dffff640 1 -- 172.21.15.146:0/153882079 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f77ec042030 con 0x7f77f8102f40 2024-08-19T01:30:36.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.017+0000 7f77dffff640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e00a2bb0 0x7f77e00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:36.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.017+0000 7f77dffff640 1 -- 172.21.15.146:0/153882079 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7f77ec0ee940 con 0x7f77f8102f40 2024-08-19T01:30:36.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.017+0000 7f77f6ffd640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e00a2bb0 0x7f77e00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:36.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.017+0000 7f77f6ffd640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e00a2bb0 0x7f77e00a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f77e80058c0 tx=0x7f77e800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:36.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.021+0000 7f77dffff640 1 -- 172.21.15.146:0/153882079 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f77ec0bc5b0 con 0x7f77f8102f40 2024-08-19T01:30:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:35 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:30:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:35 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:30:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:35 smithi190 ceph-mon[22032]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:35 smithi190 ceph-mon[22032]: osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] boot 2024-08-19T01:30:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:35 smithi190 ceph-mon[22032]: osdmap e45: 7 total, 7 up, 7 in 2024-08-19T01:30:36.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:36.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:36.199+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f77f8062360 con 0x7f77e00a2bb0 2024-08-19T01:30:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:35 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:30:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:35 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:30:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:35 smithi103 ceph-mon[18436]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-19T01:30:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:35 smithi103 ceph-mon[18436]: osd.6 [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] boot 2024-08-19T01:30:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:35 smithi103 ceph-mon[18436]: osdmap e45: 7 total, 7 up, 7 in 2024-08-19T01:30:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-19T01:30:36.942 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi103 2024-08-19T01:30:36.942 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: osdmap e46: 7 total, 7 up, 7 in 2024-08-19T01:30:36.942 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:36.942 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:36.942 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:36.942 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi103 to 5696M 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:36.943 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi103 2024-08-19T01:30:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: osdmap e46: 7 total, 7 up, 7 in 2024-08-19T01:30:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:37.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi103 to 5696M 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:37.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi103 2024-08-19T01:30:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: osdmap e46: 7 total, 7 up, 7 in 2024-08-19T01:30:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi103 to 5696M 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:37.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:37 smithi146 ceph-mon[22098]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:37 smithi146 ceph-mon[22098]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:38.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:37 smithi190 ceph-mon[22032]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:38.134 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:37 smithi190 ceph-mon[22032]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:38.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:37 smithi103 ceph-mon[18436]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:38.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:37 smithi103 ceph-mon[18436]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:39.041 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:38 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/4258832350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]: dispatch 2024-08-19T01:30:39.041 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:38 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]: dispatch 2024-08-19T01:30:39.041 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:38 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]': finished 2024-08-19T01:30:39.041 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:38 smithi146 ceph-mon[22098]: osdmap e47: 8 total, 7 up, 8 in 2024-08-19T01:30:39.041 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:39.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:38 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/4258832350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]: dispatch 2024-08-19T01:30:39.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:38 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]: dispatch 2024-08-19T01:30:39.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:38 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]': finished 2024-08-19T01:30:39.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:38 smithi190 ceph-mon[22032]: osdmap e47: 8 total, 7 up, 8 in 2024-08-19T01:30:39.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:39.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:38 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/4258832350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]: dispatch 2024-08-19T01:30:39.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:38 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]: dispatch 2024-08-19T01:30:39.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:38 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01bec701-16f3-4c16-87f7-59daf836ba88"}]': finished 2024-08-19T01:30:39.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:38 smithi103 ceph-mon[18436]: osdmap e47: 8 total, 7 up, 8 in 2024-08-19T01:30:39.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:40.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:39 smithi190 ceph-mon[22032]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:40.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:39 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/878580709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:39 smithi146 ceph-mon[22098]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:39 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/878580709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:40.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:39 smithi103 ceph-mon[18436]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:40.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:39 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/878580709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:30:42.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:41 smithi190 ceph-mon[22032]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:42.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:41 smithi146 ceph-mon[22098]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:42.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:41 smithi103 ceph-mon[18436]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:44.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:43 smithi190 ceph-mon[22032]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:44.160 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:43 smithi146 ceph-mon[22098]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:44.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:43 smithi103 ceph-mon[18436]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:45 smithi190 ceph-mon[22032]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-19T01:30:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:46.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:45 smithi146 ceph-mon[22098]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:46.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-19T01:30:46.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:46.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:45 smithi103 ceph-mon[18436]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:46.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-19T01:30:46.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:47.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:46 smithi190 ceph-mon[22032]: Deploying daemon osd.7 on smithi146 2024-08-19T01:30:47.158 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:46 smithi146 ceph-mon[22098]: Deploying daemon osd.7 on smithi146 2024-08-19T01:30:47.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:46 smithi103 ceph-mon[18436]: Deploying daemon osd.7 on smithi146 2024-08-19T01:30:48.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:47 smithi190 ceph-mon[22032]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:48.198 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:47 smithi146 ceph-mon[22098]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:48.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:47 smithi103 ceph-mon[18436]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:49.194 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:48 smithi146 ceph-mon[22098]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:49.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:48 smithi103 ceph-mon[18436]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:49.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:48 smithi190 ceph-mon[22032]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:50.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:50.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:50.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:50 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:51.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:51.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:51.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:50 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:51.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:51.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:51.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:50 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:52.094 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:51 smithi146 ceph-mon[22098]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:52.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:51 smithi190 ceph-mon[22032]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:52.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:51 smithi103 ceph-mon[18436]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:53.887 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:53 smithi146 ceph-mon[22098]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:53.887 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:53.887 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:53.888 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:53.888 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:53.888 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:53 smithi190 ceph-mon[22032]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:54.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:54.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:54.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.273+0000 7f77dffff640 1 -- 172.21.15.146:0/153882079 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77f8062360 con 0x7f77e00a2bb0 2024-08-19T01:30:54.276 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 7 on host 'smithi146' 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e00a2bb0 msgr2=0x7f77e00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e00a2bb0 0x7f77e00a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f77e80058c0 tx=0x7f77e800a2d0 comp rx=0 tx=0).stop 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8102f40 msgr2=0x7f77f81980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8102f40 0x7f77f81980f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0314e0 tx=0x7f77ec0042e0 comp rx=0 tx=0).stop 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 shutdown_connections 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e00a2bb0 0x7f77e00a5070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f8103910 0x7f77f819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f8102f40 0x7f77f81980f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 --2- 172.21.15.146:0/153882079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f8100060 0x7f77f8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:54.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 >> 172.21.15.146:0/153882079 conn(0x7f77f80fb770 msgr2=0x7f77f81061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:54.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 shutdown_connections 2024-08-19T01:30:54.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:30:54.276+0000 7f77fd5f6640 1 -- 172.21.15.146:0/153882079 wait complete. 2024-08-19T01:30:54.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:53 smithi103 ceph-mon[18436]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:54.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:54.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:54.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.897 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:54 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:54.897 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:54 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.897 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:54 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:54.898 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:54 smithi146 ceph-mon[22098]: from='osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:54.898 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:54 smithi146 ceph-mon[22098]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:54.925 DEBUG:teuthology.orchestra.run.smithi146:osd.7> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.7.service 2024-08-19T01:30:54.928 INFO:tasks.cephadm:Deploying osd.8 on smithi190 with /dev/vg_nvme/lv_2... 2024-08-19T01:30:54.928 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-19T01:30:55.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:54 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:55.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:54 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:55.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:54 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:55.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:54 smithi190 ceph-mon[22032]: from='osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:55.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:54 smithi190 ceph-mon[22032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:55.188 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:30:55.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:54 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:30:55.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:54 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:55.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:54 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:55.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:54 smithi103 ceph-mon[18436]: from='osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:55.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:54 smithi103 ceph-mon[18436]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-19T01:30:56.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:55 smithi190 ceph-mon[22032]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:56.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:55 smithi190 ceph-mon[22032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-19T01:30:56.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:55 smithi190 ceph-mon[22032]: osdmap e48: 8 total, 7 up, 8 in 2024-08-19T01:30:56.020 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:55 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:56.020 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:55 smithi190 ceph-mon[22032]: from='osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:30:56.020 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:55 smithi190 ceph-mon[22032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:30:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:55 smithi146 ceph-mon[22098]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:55 smithi146 ceph-mon[22098]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-19T01:30:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:55 smithi146 ceph-mon[22098]: osdmap e48: 8 total, 7 up, 8 in 2024-08-19T01:30:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:55 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:55 smithi146 ceph-mon[22098]: from='osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:30:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:55 smithi146 ceph-mon[22098]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:30:56.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:55 smithi103 ceph-mon[18436]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:56.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:55 smithi103 ceph-mon[18436]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-19T01:30:56.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:55 smithi103 ceph-mon[18436]: osdmap e48: 8 total, 7 up, 8 in 2024-08-19T01:30:56.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:55 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:56.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:55 smithi103 ceph-mon[18436]: from='osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:30:56.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:55 smithi103 ceph-mon[18436]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:30:56.575 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:30:56 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7[40169]: 2024-08-19T01:30:56.285+0000 7fae7dccd640 -1 osd.7 0 waiting for initial osdmap 2024-08-19T01:30:56.575 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:30:56 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7[40169]: 2024-08-19T01:30:56.295+0000 7fae79af5640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:30:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:57 smithi190 ceph-mon[22032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:30:57.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:57 smithi190 ceph-mon[22032]: osdmap e49: 8 total, 7 up, 8 in 2024-08-19T01:30:57.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:57.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:57.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:57 smithi190 ceph-mon[22032]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:57 smithi103 ceph-mon[18436]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:30:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:57 smithi103 ceph-mon[18436]: osdmap e49: 8 total, 7 up, 8 in 2024-08-19T01:30:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:57 smithi103 ceph-mon[18436]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:57.587 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:57 smithi146 ceph-mon[22098]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:30:57.587 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:57 smithi146 ceph-mon[22098]: osdmap e49: 8 total, 7 up, 8 in 2024-08-19T01:30:57.587 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:57.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:57.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:57 smithi146 ceph-mon[22098]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-19T01:30:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:30:57.936 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2024-08-19T01:30:58.192 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:30:58.550 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:30:58.550 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:30:58.550 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] boot 2024-08-19T01:30:58.550 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: osdmap e50: 8 total, 8 up, 8 in 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:58.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:58 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:30:58.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:30:58.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] boot 2024-08-19T01:30:58.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: osdmap e50: 8 total, 8 up, 8 in 2024-08-19T01:30:58.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:58.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:58.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:58 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 -- 172.21.15.190:0/3409197716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 msgr2=0x7fbf981047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/3409197716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf981047c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009900 tx=0x7fbf9402a410 comp rx=0 tx=0).stop 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 -- 172.21.15.190:0/3409197716 shutdown_connections 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/3409197716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf98107630 0x7fbf98109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/3409197716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf98104d00 0x7fbf981070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/3409197716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf981047c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 -- 172.21.15.190:0/3409197716 >> 172.21.15.190:0/3409197716 conn(0x7fbf980fbfd0 msgr2=0x7fbf980fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 -- 172.21.15.190:0/3409197716 shutdown_connections 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.630+0000 7fbf9fc40640 1 -- 172.21.15.190:0/3409197716 wait complete. 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 Processor -- start 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 -- start start 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf98197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:58.650 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf98104d00 0x7fbf98198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf98107630 0x7fbf9819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9810d570 con 0x7fbf98104d00 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9810d3f0 con 0x7fbf98107630 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9fc40640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9810d6f0 con 0x7fbf981023d0 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9d9dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf98197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9d9dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf98197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:59764/0 (socket says 172.21.15.190:59764) 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9d9dc640 1 -- 172.21.15.190:0/1185389960 learned_addr learned my addr 172.21.15.190:0/1185389960 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.631+0000 7fbf9d1db640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf98104d00 0x7fbf98198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9e1dd640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf98107630 0x7fbf9819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9d9dc640 1 -- 172.21.15.190:0/1185389960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf98107630 msgr2=0x7fbf9819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9d9dc640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf98107630 0x7fbf9819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9d9dc640 1 -- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf98104d00 msgr2=0x7fbf98198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:30:58.651 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9d9dc640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf98104d00 0x7fbf98198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9d9dc640 1 -- 172.21.15.190:0/1185389960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9819f920 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9d9dc640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf98197d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005940 tx=0x7fbf94004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf8effd640 1 -- 172.21.15.190:0/1185389960 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9403f030 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf8effd640 1 -- 172.21.15.190:0/1185389960 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf94003cb0 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9819fbb0 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf8effd640 1 -- 172.21.15.190:0/1185389960 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94035070 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.632+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf981a0090 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.633+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf980fdac0 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.634+0000 7fbf8effd640 1 -- 172.21.15.190:0/1185389960 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbf94042030 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.635+0000 7fbf8effd640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf840a2b80 0x7fbf840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.635+0000 7fbf8effd640 1 -- 172.21.15.190:0/1185389960 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5426+0+0 (secure 0 0 0) 0x7fbf940ef150 con 0x7fbf981023d0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.635+0000 7fbf9d1db640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf840a2b80 0x7fbf840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.636+0000 7fbf9d1db640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf840a2b80 0x7fbf840a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbf9806b020 tx=0x7fbf88009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:30:58.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.638+0000 7fbf8effd640 1 -- 172.21.15.190:0/1185389960 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbf940bcbf0 con 0x7fbf981023d0 2024-08-19T01:30:58.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:30:58.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:30:58.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: osd.7 [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] boot 2024-08-19T01:30:58.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: osdmap e50: 8 total, 8 up, 8 in 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:30:58.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:58 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:30:58.814 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:30:58.813+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbf98062360 con 0x7fbf840a2b80 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi146 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi146 to 5696M 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: osdmap e51: 8 total, 8 up, 8 in 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:59.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:30:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi146 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi146 to 5696M 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: osdmap e51: 8 total, 8 up, 8 in 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:30:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:30:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi146 2024-08-19T01:30:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi146 to 5696M 2024-08-19T01:30:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: osdmap e51: 8 total, 8 up, 8 in 2024-08-19T01:30:59.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:30:59.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:30:59.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:30:59.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:30:59.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:30:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:00.477 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:00 smithi190 ceph-mon[22032]: osdmap e52: 8 total, 8 up, 8 in 2024-08-19T01:31:00.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:00 smithi103 ceph-mon[18436]: osdmap e52: 8 total, 8 up, 8 in 2024-08-19T01:31:00.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:00 smithi146 ceph-mon[22098]: osdmap e52: 8 total, 8 up, 8 in 2024-08-19T01:31:01.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:01 smithi190 ceph-mon[22032]: osdmap e53: 8 total, 8 up, 8 in 2024-08-19T01:31:01.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:01 smithi190 ceph-mon[22032]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:01.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:01 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/152667505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]: dispatch 2024-08-19T01:31:01.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:01 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]: dispatch 2024-08-19T01:31:01.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:01 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]': finished 2024-08-19T01:31:01.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:01 smithi190 ceph-mon[22032]: osdmap e54: 9 total, 8 up, 9 in 2024-08-19T01:31:01.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:01.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:01 smithi146 ceph-mon[22098]: osdmap e53: 8 total, 8 up, 8 in 2024-08-19T01:31:01.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:01 smithi146 ceph-mon[22098]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:01.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:01 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/152667505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]: dispatch 2024-08-19T01:31:01.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:01 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]: dispatch 2024-08-19T01:31:01.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:01 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]': finished 2024-08-19T01:31:01.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:01 smithi146 ceph-mon[22098]: osdmap e54: 9 total, 8 up, 9 in 2024-08-19T01:31:01.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:01.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:01 smithi103 ceph-mon[18436]: osdmap e53: 8 total, 8 up, 8 in 2024-08-19T01:31:01.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:01 smithi103 ceph-mon[18436]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:01.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:01 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/152667505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]: dispatch 2024-08-19T01:31:01.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:01 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]: dispatch 2024-08-19T01:31:01.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:01 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "581ea912-28e7-4dec-8e01-18301ee980cb"}]': finished 2024-08-19T01:31:01.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:01 smithi103 ceph-mon[18436]: osdmap e54: 9 total, 8 up, 9 in 2024-08-19T01:31:01.805 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:02.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:02 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/3783390463' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:02.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:02 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/3783390463' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:02.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:02 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/3783390463' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:03.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:03 smithi190 ceph-mon[22032]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:03.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:03 smithi146 ceph-mon[22098]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:03.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:03 smithi103 ceph-mon[18436]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:05.963 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:05 smithi190 ceph-mon[22032]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:06.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:05 smithi103 ceph-mon[18436]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:05 smithi146 ceph-mon[22098]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:07.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:07 smithi190 ceph-mon[22032]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:08.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:07 smithi103 ceph-mon[18436]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:07 smithi146 ceph-mon[22098]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:09.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-19T01:31:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-19T01:31:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-19T01:31:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:09.963 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:09 smithi190 ceph-mon[22032]: Deploying daemon osd.8 on smithi190 2024-08-19T01:31:09.964 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:09 smithi190 ceph-mon[22032]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:09 smithi146 ceph-mon[22098]: Deploying daemon osd.8 on smithi190 2024-08-19T01:31:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:09 smithi146 ceph-mon[22098]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:10.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:09 smithi103 ceph-mon[18436]: Deploying daemon osd.8 on smithi190 2024-08-19T01:31:10.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:09 smithi103 ceph-mon[18436]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:11 smithi190 ceph-mon[22032]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:11 smithi146 ceph-mon[22098]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:12.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:11 smithi103 ceph-mon[18436]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:13.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:13 smithi190 ceph-mon[22032]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:13.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:13.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:13.849 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:14.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:13 smithi146 ceph-mon[22098]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:14.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:14.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:14.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:13 smithi103 ceph-mon[18436]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:14.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:15 smithi190 ceph-mon[22032]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:15 smithi146 ceph-mon[22098]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:16.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:15 smithi103 ceph-mon[18436]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:16.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.745+0000 7fbf8effd640 1 -- 172.21.15.190:0/1185389960 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf98062360 con 0x7fbf840a2b80 2024-08-19T01:31:16.748 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 8 on host 'smithi190' 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf840a2b80 msgr2=0x7fbf840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf840a2b80 0x7fbf840a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbf9806b020 tx=0x7fbf88009110 comp rx=0 tx=0).stop 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 msgr2=0x7fbf98197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf98197d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005940 tx=0x7fbf94004430 comp rx=0 tx=0).stop 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 shutdown_connections 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf840a2b80 0x7fbf840a5040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf98107630 0x7fbf9819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf98104d00 0x7fbf98198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 --2- 172.21.15.190:0/1185389960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf981023d0 0x7fbf98197d00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:16.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 >> 172.21.15.190:0/1185389960 conn(0x7fbf980fbfd0 msgr2=0x7fbf98102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:16.751 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 shutdown_connections 2024-08-19T01:31:16.751 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:31:16.748+0000 7fbf9fc40640 1 -- 172.21.15.190:0/1185389960 wait complete. 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:16 smithi103 ceph-mon[18436]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:16 smithi190 ceph-mon[22032]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:17.447 DEBUG:teuthology.orchestra.run.smithi190:osd.8> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.8.service 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:16 smithi146 ceph-mon[22098]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:17.454 INFO:tasks.cephadm:Deploying osd.9 on smithi103 with /dev/vg_nvme/lv_1... 2024-08-19T01:31:17.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-19T01:31:17.722 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:18.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:17 smithi103 ceph-mon[18436]: from='osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:31:18.097 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:17 smithi103 ceph-mon[18436]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:31:18.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:17 smithi190 ceph-mon[22032]: from='osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:31:18.293 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:17 smithi190 ceph-mon[22032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:31:18.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:17 smithi146 ceph-mon[22098]: from='osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:31:18.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:17 smithi146 ceph-mon[22098]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-19T01:31:19.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:18 smithi103 ceph-mon[18436]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-19T01:31:19.152 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:18 smithi103 ceph-mon[18436]: osdmap e55: 9 total, 8 up, 9 in 2024-08-19T01:31:19.152 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:18 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:19.152 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:18 smithi103 ceph-mon[18436]: from='osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:31:19.152 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:18 smithi103 ceph-mon[18436]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:31:19.152 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:18 smithi103 ceph-mon[18436]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:19.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:18 smithi190 ceph-mon[22032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-19T01:31:19.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:18 smithi190 ceph-mon[22032]: osdmap e55: 9 total, 8 up, 9 in 2024-08-19T01:31:19.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:18 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:19.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:18 smithi190 ceph-mon[22032]: from='osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:31:19.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:18 smithi190 ceph-mon[22032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:31:19.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:18 smithi190 ceph-mon[22032]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:18 smithi146 ceph-mon[22098]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-19T01:31:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:18 smithi146 ceph-mon[22098]: osdmap e55: 9 total, 8 up, 9 in 2024-08-19T01:31:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:18 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:18 smithi146 ceph-mon[22098]: from='osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:31:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:18 smithi146 ceph-mon[22098]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:31:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:18 smithi146 ceph-mon[22098]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:19.673 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:31:19 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8[40344]: 2024-08-19T01:31:19.404+0000 7ff7a35a5640 -1 osd.8 0 waiting for initial osdmap 2024-08-19T01:31:19.673 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:31:19 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8[40344]: 2024-08-19T01:31:19.414+0000 7ff79abb1640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:31:20.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:19 smithi103 ceph-mon[18436]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:31:20.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:19 smithi103 ceph-mon[18436]: osdmap e56: 9 total, 8 up, 9 in 2024-08-19T01:31:20.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:19 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:20.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:19 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:20.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:19 smithi103 ceph-mon[18436]: from='osd.8 ' entity='osd.8' 2024-08-19T01:31:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:19 smithi190 ceph-mon[22032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:31:20.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:19 smithi190 ceph-mon[22032]: osdmap e56: 9 total, 8 up, 9 in 2024-08-19T01:31:20.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:19 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:20.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:19 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:20.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:19 smithi190 ceph-mon[22032]: from='osd.8 ' entity='osd.8' 2024-08-19T01:31:20.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:19 smithi146 ceph-mon[22098]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:31:20.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:19 smithi146 ceph-mon[22098]: osdmap e56: 9 total, 8 up, 9 in 2024-08-19T01:31:20.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:19 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:20.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:19 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:20.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:19 smithi146 ceph-mon[22098]: from='osd.8 ' entity='osd.8' 2024-08-19T01:31:20.548 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:31:20.574 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-08-19T01:31:20.840 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:31:21.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:31:21.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] boot 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: osdmap e57: 9 total, 9 up, 9 in 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi190 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi190 to 5696M 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.249 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:21.250 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:21.250 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.250 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:21.250 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:20 smithi103 ceph-mon[18436]: osdmap e58: 9 total, 9 up, 9 in 2024-08-19T01:31:21.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 -- 172.21.15.103:0/2676236520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60100840 msgr2=0x7faa60100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:21.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 --2- 172.21.15.103:0/2676236520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60100840 0x7faa60100c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faa50009900 tx=0x7faa5002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 -- 172.21.15.103:0/2676236520 shutdown_connections 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 --2- 172.21.15.103:0/2676236520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa60102400 0x7faa60108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 --2- 172.21.15.103:0/2676236520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa60101a40 0x7faa60101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 --2- 172.21.15.103:0/2676236520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60100840 0x7faa60100c40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 -- 172.21.15.103:0/2676236520 >> 172.21.15.103:0/2676236520 conn(0x7faa600fbfd0 msgr2=0x7faa600fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 -- 172.21.15.103:0/2676236520 shutdown_connections 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 -- 172.21.15.103:0/2676236520 wait complete. 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.330+0000 7faa6532e640 1 Processor -- start 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa6532e640 1 -- start start 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa6532e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa60100840 0x7faa6006b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa6532e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa60101a40 0x7faa6019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:21.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa6532e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60102400 0x7faa6019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa6532e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa6010c350 con 0x7faa60102400 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa6532e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faa6010c1d0 con 0x7faa60101a40 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa6532e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa6010c4d0 con 0x7faa60100840 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60102400 0x7faa6019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60102400 0x7faa6019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40974/0 (socket says 172.21.15.103:40974) 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa60100840 0x7faa6006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 -- 172.21.15.103:0/3549910603 learned_addr learned my addr 172.21.15.103:0/3549910603 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5e7fc640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa60101a40 0x7faa6019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:21.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 -- 172.21.15.103:0/3549910603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa60100840 msgr2=0x7faa6006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa60100840 0x7faa6006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 -- 172.21.15.103:0/3549910603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa60101a40 msgr2=0x7faa6019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa60101a40 0x7faa6019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.331+0000 7faa5f7fe640 1 -- 172.21.15.103:0/3549910603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6019f6a0 con 0x7faa60102400 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.332+0000 7faa5f7fe640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60102400 0x7faa6019ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa5800cce0 tx=0x7faa5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.332+0000 7faa47fff640 1 -- 172.21.15.103:0/3549910603 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5800f450 con 0x7faa60102400 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.332+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6019f990 con 0x7faa60102400 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.332+0000 7faa47fff640 1 -- 172.21.15.103:0/3549910603 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5800faa0 con 0x7faa60102400 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.332+0000 7faa47fff640 1 -- 172.21.15.103:0/3549910603 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58018b70 con 0x7faa60102400 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.332+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6019fe80 con 0x7faa60102400 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.333+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa60107c40 con 0x7faa60102400 2024-08-19T01:31:21.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.334+0000 7faa47fff640 1 -- 172.21.15.103:0/3549910603 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7faa58018d10 con 0x7faa60102400 2024-08-19T01:31:21.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.335+0000 7faa47fff640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7faa480a2b80 0x7faa480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:21.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.335+0000 7faa5effd640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7faa480a2b80 0x7faa480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:21.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.335+0000 7faa47fff640 1 -- 172.21.15.103:0/3549910603 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5858+0+0 (secure 0 0 0) 0x7faa580ceb00 con 0x7faa60102400 2024-08-19T01:31:21.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.335+0000 7faa5effd640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7faa480a2b80 0x7faa480a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faa6006b5a0 tx=0x7faa50000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:21.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.343+0000 7faa47fff640 1 -- 172.21.15.103:0/3549910603 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7faa58014030 con 0x7faa60102400 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] boot 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: osdmap e57: 9 total, 9 up, 9 in 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi190 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi190 to 5696M 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:21.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:20 smithi190 ceph-mon[22032]: osdmap e58: 9 total, 9 up, 9 in 2024-08-19T01:31:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:31:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:31:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: osd.8 [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] boot 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: osdmap e57: 9 total, 9 up, 9 in 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi190 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi190 to 5696M 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:21.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-19T01:31:21.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:20 smithi146 ceph-mon[22098]: osdmap e58: 9 total, 9 up, 9 in 2024-08-19T01:31:21.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:21.525+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7faa60062360 con 0x7faa480a2b80 2024-08-19T01:31:22.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:22 smithi103 ceph-mon[18436]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:22.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:22.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:22.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:22 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:22.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:22 smithi103 ceph-mon[18436]: osdmap e59: 9 total, 9 up, 9 in 2024-08-19T01:31:22.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:22 smithi190 ceph-mon[22032]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:22.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:22.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:22.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:22 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:22.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:22 smithi190 ceph-mon[22032]: osdmap e59: 9 total, 9 up, 9 in 2024-08-19T01:31:22.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:22 smithi146 ceph-mon[22098]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:22.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:22.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:22.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:22 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:22.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:22 smithi146 ceph-mon[22098]: osdmap e59: 9 total, 9 up, 9 in 2024-08-19T01:31:23.704 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:23 smithi146 ceph-mon[22098]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:31:23.704 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:23 smithi146 ceph-mon[22098]: osdmap e60: 9 total, 9 up, 9 in 2024-08-19T01:31:23.756 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:23 smithi103 ceph-mon[18436]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:31:23.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:23 smithi103 ceph-mon[18436]: osdmap e60: 9 total, 9 up, 9 in 2024-08-19T01:31:23.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:23 smithi190 ceph-mon[22032]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-19T01:31:23.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:23 smithi190 ceph-mon[22032]: osdmap e60: 9 total, 9 up, 9 in 2024-08-19T01:31:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:24 smithi146 ceph-mon[22098]: osdmap e61: 9 total, 9 up, 9 in 2024-08-19T01:31:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:24 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/626380728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ae349f-b007-407a-b7e9-0f9b9ba66aec"}]: dispatch 2024-08-19T01:31:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:24 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/626380728' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8ae349f-b007-407a-b7e9-0f9b9ba66aec"}]': finished 2024-08-19T01:31:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:24 smithi146 ceph-mon[22098]: osdmap e62: 10 total, 9 up, 10 in 2024-08-19T01:31:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:24 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:24 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/89241799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:24 smithi146 ceph-mon[22098]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-08-19T01:31:25.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:24 smithi103 ceph-mon[18436]: osdmap e61: 9 total, 9 up, 9 in 2024-08-19T01:31:25.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:24 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/626380728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ae349f-b007-407a-b7e9-0f9b9ba66aec"}]: dispatch 2024-08-19T01:31:25.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:24 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/626380728' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8ae349f-b007-407a-b7e9-0f9b9ba66aec"}]': finished 2024-08-19T01:31:25.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:24 smithi103 ceph-mon[18436]: osdmap e62: 10 total, 9 up, 10 in 2024-08-19T01:31:25.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:24 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:25.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:24 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/89241799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:25.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:24 smithi103 ceph-mon[18436]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-08-19T01:31:25.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:24 smithi190 ceph-mon[22032]: osdmap e61: 9 total, 9 up, 9 in 2024-08-19T01:31:25.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:24 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/626380728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8ae349f-b007-407a-b7e9-0f9b9ba66aec"}]: dispatch 2024-08-19T01:31:25.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:24 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/626380728' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8ae349f-b007-407a-b7e9-0f9b9ba66aec"}]': finished 2024-08-19T01:31:25.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:24 smithi190 ceph-mon[22032]: osdmap e62: 10 total, 9 up, 10 in 2024-08-19T01:31:25.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:24 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:25.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:24 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/89241799' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:25.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:24 smithi190 ceph-mon[22032]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-08-19T01:31:28.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:27 smithi103 ceph-mon[18436]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:31:28.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:27 smithi190 ceph-mon[22032]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:31:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:27 smithi146 ceph-mon[22098]: pgmap v185: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:31:30.118 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:29 smithi103 ceph-mon[18436]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:30.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:29 smithi190 ceph-mon[22032]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:29 smithi146 ceph-mon[22098]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:32.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:31 smithi103 ceph-mon[18436]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:32.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-19T01:31:32.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:31 smithi190 ceph-mon[22032]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-19T01:31:32.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:32.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:31 smithi146 ceph-mon[22098]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:32.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-19T01:31:32.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:33.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:32 smithi190 ceph-mon[22032]: Deploying daemon osd.9 on smithi103 2024-08-19T01:31:33.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:32 smithi103 ceph-mon[18436]: Deploying daemon osd.9 on smithi103 2024-08-19T01:31:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:32 smithi146 ceph-mon[22098]: Deploying daemon osd.9 on smithi103 2024-08-19T01:31:34.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:33 smithi190 ceph-mon[22032]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:34.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:33 smithi103 ceph-mon[18436]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:34.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:33 smithi146 ceph-mon[22098]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:36.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:35 smithi103 ceph-mon[18436]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:36.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:35 smithi190 ceph-mon[22032]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:35 smithi146 ceph-mon[22098]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:38.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:37 smithi103 ceph-mon[18436]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:38.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:38.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:38.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:38.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:37 smithi190 ceph-mon[22032]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:38.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:38.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:38.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:38.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:37 smithi146 ceph-mon[22098]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:38.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:38.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:38.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:39.944 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:39 smithi103 ceph-mon[18436]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:40.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:39 smithi190 ceph-mon[22032]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:39 smithi146 ceph-mon[22098]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:40.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.777+0000 7faa47fff640 1 -- 172.21.15.103:0/3549910603 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faa60062360 con 0x7faa480a2b80 2024-08-19T01:31:40.779 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 9 on host 'smithi103' 2024-08-19T01:31:40.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7faa480a2b80 msgr2=0x7faa480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:40.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7faa480a2b80 0x7faa480a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faa6006b5a0 tx=0x7faa50000bc0 comp rx=0 tx=0).stop 2024-08-19T01:31:40.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60102400 msgr2=0x7faa6019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:40.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60102400 0x7faa6019ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa5800cce0 tx=0x7faa5800a620 comp rx=0 tx=0).stop 2024-08-19T01:31:40.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 shutdown_connections 2024-08-19T01:31:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7faa480a2b80 0x7faa480a5040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa60102400 0x7faa6019ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa60101a40 0x7faa6019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 --2- 172.21.15.103:0/3549910603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa60100840 0x7faa6006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 >> 172.21.15.103:0/3549910603 conn(0x7faa600fbfd0 msgr2=0x7faa60104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 shutdown_connections 2024-08-19T01:31:40.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:31:40.780+0000 7faa6532e640 1 -- 172.21.15.103:0/3549910603 wait complete. 2024-08-19T01:31:41.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:41.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:41.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:41.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:41.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:40 smithi103 ceph-mon[18436]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-19T01:31:41.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:40 smithi190 ceph-mon[22032]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-19T01:31:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:31:41.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:41.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:40 smithi146 ceph-mon[22098]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-19T01:31:41.581 DEBUG:teuthology.orchestra.run.smithi103:osd.9> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.9.service 2024-08-19T01:31:41.584 INFO:tasks.cephadm:Deploying osd.10 on smithi146 with /dev/vg_nvme/lv_1... 2024-08-19T01:31:41.584 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-19T01:31:41.844 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:31:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:42 smithi103 ceph-mon[18436]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-19T01:31:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:42 smithi103 ceph-mon[18436]: osdmap e63: 10 total, 9 up, 10 in 2024-08-19T01:31:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:42 smithi103 ceph-mon[18436]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:31:43.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:43.096 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:42 smithi146 ceph-mon[22098]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-19T01:31:43.096 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:42 smithi146 ceph-mon[22098]: osdmap e63: 10 total, 9 up, 10 in 2024-08-19T01:31:43.096 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:42 smithi146 ceph-mon[22098]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:31:43.097 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:43.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:42 smithi190 ceph-mon[22032]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-19T01:31:43.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:42 smithi190 ceph-mon[22032]: osdmap e63: 10 total, 9 up, 10 in 2024-08-19T01:31:43.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:42 smithi190 ceph-mon[22032]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-08-19T01:31:43.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:43.426 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:31:43.196+0000 7f588b9d9640 -1 osd.9 0 waiting for initial osdmap 2024-08-19T01:31:43.427 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:31:43.206+0000 7f5882fe5640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:31:43.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:43 smithi146 ceph-mon[22098]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:43.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:43 smithi146 ceph-mon[22098]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:31:43.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:43 smithi146 ceph-mon[22098]: osdmap e64: 10 total, 9 up, 10 in 2024-08-19T01:31:43.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:43.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:43.952 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:43 smithi146 ceph-mon[22098]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' 2024-08-19T01:31:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-mon[18436]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-mon[18436]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:31:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-mon[18436]: osdmap e64: 10 total, 9 up, 10 in 2024-08-19T01:31:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:43 smithi103 ceph-mon[18436]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' 2024-08-19T01:31:44.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:43 smithi190 ceph-mon[22032]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:44.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:43 smithi190 ceph-mon[22032]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-08-19T01:31:44.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:43 smithi190 ceph-mon[22032]: osdmap e64: 10 total, 9 up, 10 in 2024-08-19T01:31:44.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:44.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:44.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:43 smithi190 ceph-mon[22032]: from='osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167]' entity='osd.9' 2024-08-19T01:31:44.394 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-19T01:31:44.419 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_1 2024-08-19T01:31:44.669 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] boot 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: osdmap e65: 10 total, 10 up, 10 in 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:45.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:45.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:31:45.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:31:45.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:45.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] boot 2024-08-19T01:31:45.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: osdmap e65: 10 total, 10 up, 10 in 2024-08-19T01:31:45.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:45.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: osd.9 [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] boot 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: osdmap e65: 10 total, 10 up, 10 in 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:31:45.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:45.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:31:45.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:31:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 -- 172.21.15.146:0/2913618801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038104d00 msgr2=0x7fe0381070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/2913618801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038104d00 0x7fe0381070f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009900 tx=0x7fe02c02a410 comp rx=0 tx=0).stop 2024-08-19T01:31:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 -- 172.21.15.146:0/2913618801 shutdown_connections 2024-08-19T01:31:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/2913618801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe038107630 0x7fe038109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/2913618801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038104d00 0x7fe0381070f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/2913618801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0381023d0 0x7fe0381047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 -- 172.21.15.146:0/2913618801 >> 172.21.15.146:0/2913618801 conn(0x7fe0380fbfd0 msgr2=0x7fe0380fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 -- 172.21.15.146:0/2913618801 shutdown_connections 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.168+0000 7fe03c9ae640 1 -- 172.21.15.146:0/2913618801 wait complete. 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 Processor -- start 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 -- start start 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0381023d0 0x7fe038101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe038104d00 0x7fe038100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038107630 0x7fe0381008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810d4d0 con 0x7fe038104d00 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810d350 con 0x7fe038107630 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03c9ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe03810d650 con 0x7fe0381023d0 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe036d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038107630 0x7fe0381008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe035d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe038104d00 0x7fe038100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe036d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038107630 0x7fe0381008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:40926/0 (socket says 172.21.15.146:40926) 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe036d9d640 1 -- 172.21.15.146:0/3899502891 learned_addr learned my addr 172.21.15.146:0/3899502891 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe03659c640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0381023d0 0x7fe038101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.169+0000 7fe036d9d640 1 -- 172.21.15.146:0/3899502891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0381023d0 msgr2=0x7fe038101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe036d9d640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0381023d0 0x7fe038101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe036d9d640 1 -- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe038104d00 msgr2=0x7fe038100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe036d9d640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe038104d00 0x7fe038100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe036d9d640 1 -- 172.21.15.146:0/3899502891 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe038101180 con 0x7fe038107630 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe03659c640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0381023d0 0x7fe038101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe035d9b640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe038104d00 0x7fe038100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe036d9d640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038107630 0x7fe0381008c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe03000eb10 tx=0x7fe03000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe0277fe640 1 -- 172.21.15.146:0/3899502891 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030004b20 con 0x7fe038107630 2024-08-19T01:31:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0381a1d70 con 0x7fe038107630 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe0277fe640 1 -- 172.21.15.146:0/3899502891 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe030004cc0 con 0x7fe038107630 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe0277fe640 1 -- 172.21.15.146:0/3899502891 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030010690 con 0x7fe038107630 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.170+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0381a21d0 con 0x7fe038107630 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.171+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0380fdac0 con 0x7fe038107630 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.172+0000 7fe0277fe640 1 -- 172.21.15.146:0/3899502891 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe030005020 con 0x7fe038107630 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.172+0000 7fe0277fe640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0200a2b80 0x7fe0200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.173+0000 7fe0277fe640 1 -- 172.21.15.146:0/3899502891 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6261+0+0 (secure 0 0 0) 0x7fe0300cee80 con 0x7fe038107630 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.173+0000 7fe03659c640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0200a2b80 0x7fe0200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.173+0000 7fe03659c640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0200a2b80 0x7fe0200a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe028005fd0 tx=0x7fe028005ea0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:31:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.177+0000 7fe0277fe640 1 -- 172.21.15.146:0/3899502891 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe03009c5e0 con 0x7fe038107630 2024-08-19T01:31:45.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:31:45.350+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe038062360 con 0x7fe0200a2b80 2024-08-19T01:31:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:45 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi103 2024-08-19T01:31:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:45 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi103 to 4272M 2024-08-19T01:31:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:45 smithi103 ceph-mon[18436]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:46.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:45 smithi103 ceph-mon[18436]: osdmap e66: 10 total, 10 up, 10 in 2024-08-19T01:31:46.097 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:45 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi103 2024-08-19T01:31:46.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:45 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi103 to 4272M 2024-08-19T01:31:46.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:45 smithi146 ceph-mon[22098]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:46.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:46.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:46.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:46.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:45 smithi146 ceph-mon[22098]: osdmap e66: 10 total, 10 up, 10 in 2024-08-19T01:31:46.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:45 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi103 2024-08-19T01:31:46.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:45 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi103 to 4272M 2024-08-19T01:31:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:45 smithi190 ceph-mon[22032]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-19T01:31:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:31:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:31:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:45 smithi190 ceph-mon[22032]: osdmap e66: 10 total, 10 up, 10 in 2024-08-19T01:31:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:46 smithi103 ceph-mon[18436]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:47.092 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:46 smithi146 ceph-mon[22098]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:47.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:46 smithi190 ceph-mon[22032]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:31:48.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:47 smithi103 ceph-mon[18436]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:48.068 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:47 smithi146 ceph-mon[22098]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:48.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:47 smithi190 ceph-mon[22032]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:49.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:48 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/648128602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]: dispatch 2024-08-19T01:31:49.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:48 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]: dispatch 2024-08-19T01:31:49.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:48 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]': finished 2024-08-19T01:31:49.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:48 smithi103 ceph-mon[18436]: osdmap e67: 11 total, 10 up, 11 in 2024-08-19T01:31:49.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:48 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:49.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:48 smithi103 ceph-mon[18436]: from='client.? 172.21.15.146:0/3598950931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:49.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:48 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/648128602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]: dispatch 2024-08-19T01:31:49.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:48 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]: dispatch 2024-08-19T01:31:49.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:48 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]': finished 2024-08-19T01:31:49.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:48 smithi190 ceph-mon[22032]: osdmap e67: 11 total, 10 up, 11 in 2024-08-19T01:31:49.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:48 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:49.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:48 smithi190 ceph-mon[22032]: from='client.? 172.21.15.146:0/3598950931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:49.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:48 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/648128602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]: dispatch 2024-08-19T01:31:49.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:48 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]: dispatch 2024-08-19T01:31:49.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:48 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d90c65e-e820-4352-b4bd-58204606e5ba"}]': finished 2024-08-19T01:31:49.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:48 smithi146 ceph-mon[22098]: osdmap e67: 11 total, 10 up, 11 in 2024-08-19T01:31:49.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:48 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:31:49.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:48 smithi146 ceph-mon[22098]: from='client.? 172.21.15.146:0/3598950931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:31:50.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:49 smithi103 ceph-mon[18436]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:50.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:49 smithi190 ceph-mon[22032]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:50.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:49 smithi146 ceph-mon[22098]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:51.987 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:51 smithi146 ceph-mon[22098]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:52.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:51 smithi190 ceph-mon[22032]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:52.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:51 smithi103 ceph-mon[18436]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:53.981 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:53 smithi146 ceph-mon[22098]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:54.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:53 smithi190 ceph-mon[22032]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:54.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:53 smithi103 ceph-mon[18436]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:56.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:55 smithi190 ceph-mon[22032]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:56.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:55 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-19T01:31:56.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:55 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:56.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:55 smithi190 ceph-mon[22032]: Deploying daemon osd.10 on smithi146 2024-08-19T01:31:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:55 smithi146 ceph-mon[22098]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:55 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-19T01:31:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:55 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:55 smithi146 ceph-mon[22098]: Deploying daemon osd.10 on smithi146 2024-08-19T01:31:56.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:55 smithi103 ceph-mon[18436]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:56.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:55 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-19T01:31:56.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:55 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:31:56.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:55 smithi103 ceph-mon[18436]: Deploying daemon osd.10 on smithi146 2024-08-19T01:31:57.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:57 smithi146 ceph-mon[22098]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:58.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:57 smithi190 ceph-mon[22032]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:31:58.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:57 smithi103 ceph-mon[18436]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:00.120 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:31:59 smithi146 ceph-mon[22098]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:00.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:31:59 smithi190 ceph-mon[22032]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:00.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:31:59 smithi103 ceph-mon[18436]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:00.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:00.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:00.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:01.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:01.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:01.981 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:01 smithi146 ceph-mon[22098]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:02.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:01 smithi190 ceph-mon[22032]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:02.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:01 smithi103 ceph-mon[18436]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:03.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.525+0000 7fe0277fe640 1 -- 172.21.15.146:0/3899502891 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe038062360 con 0x7fe0200a2b80 2024-08-19T01:32:03.528 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 10 on host 'smithi146' 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0200a2b80 msgr2=0x7fe0200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0200a2b80 0x7fe0200a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe028005fd0 tx=0x7fe028005ea0 comp rx=0 tx=0).stop 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038107630 msgr2=0x7fe0381008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038107630 0x7fe0381008c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe03000eb10 tx=0x7fe03000c2d0 comp rx=0 tx=0).stop 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 shutdown_connections 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0200a2b80 0x7fe0200a5040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe038107630 0x7fe0381008c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe038104d00 0x7fe038100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 --2- 172.21.15.146:0/3899502891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0381023d0 0x7fe038101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 >> 172.21.15.146:0/3899502891 conn(0x7fe0380fbfd0 msgr2=0x7fe038102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 shutdown_connections 2024-08-19T01:32:03.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-19T01:32:03.528+0000 7fe03c9ae640 1 -- 172.21.15.146:0/3899502891 wait complete. 2024-08-19T01:32:04.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:03 smithi190 ceph-mon[22032]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:04.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:03 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:04.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:03 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:04.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:03 smithi146 ceph-mon[22098]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:04.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:03 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:04.135 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:03 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:04.166 DEBUG:teuthology.orchestra.run.smithi146:osd.10> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.10.service 2024-08-19T01:32:04.168 INFO:tasks.cephadm:Deploying osd.11 on smithi190 with /dev/vg_nvme/lv_1... 2024-08-19T01:32:04.168 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-19T01:32:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:03 smithi103 ceph-mon[18436]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:03 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:04.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:03 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:04.431 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:32:05.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:04 smithi146 ceph-mon[22098]: from='osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:32:05.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:04 smithi146 ceph-mon[22098]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:32:05.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:04 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:05.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:04 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:05.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:04 smithi190 ceph-mon[22032]: from='osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:32:05.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:04 smithi190 ceph-mon[22032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:32:05.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:04 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:05.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:04 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:04 smithi103 ceph-mon[18436]: from='osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:32:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:04 smithi103 ceph-mon[18436]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-19T01:32:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:04 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:05.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:04 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:06.037 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-mon[22098]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:06.037 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-mon[22098]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-19T01:32:06.037 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-mon[22098]: osdmap e68: 11 total, 10 up, 11 in 2024-08-19T01:32:06.037 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:06.037 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-mon[22098]: from='osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:32:06.037 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-mon[22098]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:32:06.038 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:32:05.854+0000 7f4365d00640 -1 osd.10 0 waiting for initial osdmap 2024-08-19T01:32:06.038 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:32:05 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:32:05.865+0000 7f4361b28640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:32:06.121 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:05 smithi190 ceph-mon[22032]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:06.121 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:05 smithi190 ceph-mon[22032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-19T01:32:06.121 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:05 smithi190 ceph-mon[22032]: osdmap e68: 11 total, 10 up, 11 in 2024-08-19T01:32:06.121 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:06.121 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:05 smithi190 ceph-mon[22032]: from='osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:32:06.121 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:05 smithi190 ceph-mon[22032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:32:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:05 smithi103 ceph-mon[18436]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:05 smithi103 ceph-mon[18436]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-19T01:32:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:05 smithi103 ceph-mon[18436]: osdmap e68: 11 total, 10 up, 11 in 2024-08-19T01:32:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:05 smithi103 ceph-mon[18436]: from='osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:32:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:05 smithi103 ceph-mon[18436]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-08-19T01:32:07.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:06 smithi190 ceph-mon[22032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:32:07.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:06 smithi190 ceph-mon[22032]: osdmap e69: 11 total, 10 up, 11 in 2024-08-19T01:32:07.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:06 smithi190 ceph-mon[22032]: osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] boot 2024-08-19T01:32:07.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:06 smithi190 ceph-mon[22032]: osdmap e70: 11 total, 11 up, 11 in 2024-08-19T01:32:07.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.046 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-08-19T01:32:07.083 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2024-08-19T01:32:07.125 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:06 smithi146 ceph-mon[22098]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:32:07.125 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:06 smithi146 ceph-mon[22098]: osdmap e69: 11 total, 10 up, 11 in 2024-08-19T01:32:07.125 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.125 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.125 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:06 smithi146 ceph-mon[22098]: osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] boot 2024-08-19T01:32:07.125 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:06 smithi146 ceph-mon[22098]: osdmap e70: 11 total, 11 up, 11 in 2024-08-19T01:32:07.125 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:06 smithi103 ceph-mon[18436]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-08-19T01:32:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:06 smithi103 ceph-mon[18436]: osdmap e69: 11 total, 10 up, 11 in 2024-08-19T01:32:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:06 smithi103 ceph-mon[18436]: osd.10 [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] boot 2024-08-19T01:32:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:06 smithi103 ceph-mon[18436]: osdmap e70: 11 total, 11 up, 11 in 2024-08-19T01:32:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-19T01:32:07.333 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:32:07.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 -- 172.21.15.190:0/1948943918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 msgr2=0x7f6158101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:07.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1948943918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158101d20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6148009900 tx=0x7f614802a410 comp rx=0 tx=0).stop 2024-08-19T01:32:07.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 -- 172.21.15.190:0/1948943918 shutdown_connections 2024-08-19T01:32:07.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1948943918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61581034e0 0x7f6158109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:07.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1948943918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6158102b20 0x7f6158102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1948943918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158101d20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 -- 172.21.15.190:0/1948943918 >> 172.21.15.190:0/1948943918 conn(0x7f61580fd0f0 msgr2=0x7f61580ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 -- 172.21.15.190:0/1948943918 shutdown_connections 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 -- 172.21.15.190:0/1948943918 wait complete. 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 Processor -- start 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.852+0000 7f615e3fc640 1 -- start start 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f615e3fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f615e3fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6158102b20 0x7f6158198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f615e3fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61581034e0 0x7f615819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f615e3fc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f615810d450 con 0x7f61581034e0 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f615e3fc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f615810d2d0 con 0x7f6158102b20 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f615e3fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f615810d5d0 con 0x7f6158101920 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:07.864 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47984/0 (socket says 172.21.15.190:47984) 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 -- 172.21.15.190:0/1217211959 learned_addr learned my addr 172.21.15.190:0/1217211959 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f615c999640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61581034e0 0x7f615819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f614ffff640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6158102b20 0x7f6158198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 -- 172.21.15.190:0/1217211959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6158102b20 msgr2=0x7f6158198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6158102b20 0x7f6158198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 -- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61581034e0 msgr2=0x7f615819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61581034e0 0x7f615819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 -- 172.21.15.190:0/1217211959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615819f780 con 0x7f6158101920 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.853+0000 7f6157fff640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158197b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61480314e0 tx=0x7f61480042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.856+0000 7f6155ffb640 1 -- 172.21.15.190:0/1217211959 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614803f030 con 0x7f6158101920 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.856+0000 7f6155ffb640 1 -- 172.21.15.190:0/1217211959 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6148003cb0 con 0x7f6158101920 2024-08-19T01:32:07.865 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.856+0000 7f6155ffb640 1 -- 172.21.15.190:0/1217211959 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148035070 con 0x7f6158101920 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.857+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615819fa10 con 0x7f6158101920 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.857+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615819ff40 con 0x7f6158101920 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.858+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6158108db0 con 0x7f6158101920 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.858+0000 7f6155ffb640 1 -- 172.21.15.190:0/1217211959 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6148042030 con 0x7f6158101920 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.859+0000 7f6155ffb640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f61440a2c30 0x7f61440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.859+0000 7f614ffff640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f61440a2c30 0x7f61440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.859+0000 7f6155ffb640 1 -- 172.21.15.190:0/1217211959 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6723+0+0 (secure 0 0 0) 0x7f61480eeef0 con 0x7f6158101920 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.860+0000 7f614ffff640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f61440a2c30 0x7f61440a50f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6158199070 tx=0x7f614000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:07.866 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:07.864+0000 7f6155ffb640 1 -- 172.21.15.190:0/1217211959 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f61480bbe30 con 0x7f6158101920 2024-08-19T01:32:08.045 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:08.043+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6158062360 con 0x7f61440a2c30 2024-08-19T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: osdmap e71: 11 total, 11 up, 11 in 2024-08-19T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.045 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:08.046 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:07 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:32:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:32:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: osdmap e71: 11 total, 11 up, 11 in 2024-08-19T01:32:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:08.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:07 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: osdmap e71: 11 total, 11 up, 11 in 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.205 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:08.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:07 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:09.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:08 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi146 2024-08-19T01:32:09.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:08 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi146 to 4272M 2024-08-19T01:32:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:08 smithi190 ceph-mon[22032]: osdmap e72: 11 total, 11 up, 11 in 2024-08-19T01:32:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:32:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:32:09.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:08 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:08 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi146 2024-08-19T01:32:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:08 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi146 to 4272M 2024-08-19T01:32:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:08 smithi146 ceph-mon[22098]: osdmap e72: 11 total, 11 up, 11 in 2024-08-19T01:32:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:32:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:32:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:08 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:08 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi146 2024-08-19T01:32:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:08 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi146 to 4272M 2024-08-19T01:32:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:08 smithi103 ceph-mon[18436]: osdmap e72: 11 total, 11 up, 11 in 2024-08-19T01:32:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-19T01:32:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-19T01:32:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:08 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:10.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:09 smithi190 ceph-mon[22032]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:09 smithi190 ceph-mon[22032]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:09 smithi190 ceph-mon[22032]: osdmap e73: 11 total, 11 up, 11 in 2024-08-19T01:32:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:09 smithi146 ceph-mon[22098]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:09 smithi146 ceph-mon[22098]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:09 smithi146 ceph-mon[22098]: osdmap e73: 11 total, 11 up, 11 in 2024-08-19T01:32:10.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:09 smithi103 ceph-mon[18436]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:10.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:09 smithi103 ceph-mon[18436]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:10.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:09 smithi103 ceph-mon[18436]: osdmap e73: 11 total, 11 up, 11 in 2024-08-19T01:32:11.065 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:10 smithi190 ceph-mon[22032]: osdmap e74: 11 total, 11 up, 11 in 2024-08-19T01:32:11.065 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:10 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/2492279235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]: dispatch 2024-08-19T01:32:11.065 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:10 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]: dispatch 2024-08-19T01:32:11.066 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:10 smithi190 ceph-mon[22032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]': finished 2024-08-19T01:32:11.066 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:10 smithi190 ceph-mon[22032]: osdmap e75: 12 total, 11 up, 12 in 2024-08-19T01:32:11.066 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:10 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:11.066 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:10 smithi190 ceph-mon[22032]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:10 smithi146 ceph-mon[22098]: osdmap e74: 11 total, 11 up, 11 in 2024-08-19T01:32:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:10 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/2492279235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]: dispatch 2024-08-19T01:32:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:10 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]: dispatch 2024-08-19T01:32:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:10 smithi146 ceph-mon[22098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]': finished 2024-08-19T01:32:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:10 smithi146 ceph-mon[22098]: osdmap e75: 12 total, 11 up, 12 in 2024-08-19T01:32:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:10 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:10 smithi146 ceph-mon[22098]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:11.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:10 smithi103 ceph-mon[18436]: osdmap e74: 11 total, 11 up, 11 in 2024-08-19T01:32:11.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:10 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/2492279235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]: dispatch 2024-08-19T01:32:11.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:10 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]: dispatch 2024-08-19T01:32:11.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:10 smithi103 ceph-mon[18436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97"}]': finished 2024-08-19T01:32:11.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:10 smithi103 ceph-mon[18436]: osdmap e75: 12 total, 11 up, 12 in 2024-08-19T01:32:11.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:10 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:11.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:10 smithi103 ceph-mon[18436]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:11 smithi146 ceph-mon[22098]: from='client.? 172.21.15.190:0/3061796200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:32:12.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:11 smithi103 ceph-mon[18436]: from='client.? 172.21.15.190:0/3061796200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:32:12.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:11 smithi190 ceph-mon[22032]: from='client.? 172.21.15.190:0/3061796200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-19T01:32:13.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:12 smithi146 ceph-mon[22098]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:32:13.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:12 smithi103 ceph-mon[18436]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:32:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:12 smithi190 ceph-mon[22032]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-19T01:32:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:15 smithi190 ceph-mon[22032]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:32:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:15 smithi146 ceph-mon[22098]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:32:16.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:15 smithi103 ceph-mon[18436]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-19T01:32:18.066 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:17 smithi190 ceph-mon[22032]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-19T01:32:18.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-19T01:32:18.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:17 smithi146 ceph-mon[22098]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-19T01:32:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-19T01:32:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:18.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:17 smithi103 ceph-mon[18436]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-19T01:32:18.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-19T01:32:18.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:19.073 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:18 smithi190 ceph-mon[22032]: Deploying daemon osd.11 on smithi190 2024-08-19T01:32:19.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:18 smithi146 ceph-mon[22098]: Deploying daemon osd.11 on smithi190 2024-08-19T01:32:19.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:18 smithi103 ceph-mon[18436]: Deploying daemon osd.11 on smithi190 2024-08-19T01:32:20.023 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:19 smithi190 ceph-mon[22032]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-08-19T01:32:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:19 smithi146 ceph-mon[22098]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-08-19T01:32:20.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:19 smithi103 ceph-mon[18436]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-08-19T01:32:22.129 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:21 smithi190 ceph-mon[22032]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-08-19T01:32:22.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:21 smithi146 ceph-mon[22098]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-08-19T01:32:22.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:21 smithi103 ceph-mon[18436]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-08-19T01:32:24.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:23 smithi190 ceph-mon[22032]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-19T01:32:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:23 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:23 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:24.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:23 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:23 smithi146 ceph-mon[22098]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-19T01:32:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:23 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:23 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:23 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:24.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:23 smithi103 ceph-mon[18436]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-19T01:32:24.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:24.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:24.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:23 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:26.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:25 smithi190 ceph-mon[22032]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:26.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:25 smithi146 ceph-mon[22098]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:25 smithi103 ceph-mon[18436]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:26.933 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 11 on host 'smithi190' 2024-08-19T01:32:26.934 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.931+0000 7f6155ffb640 1 -- 172.21.15.190:0/1217211959 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6158062360 con 0x7f61440a2c30 2024-08-19T01:32:26.935 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f61440a2c30 msgr2=0x7f61440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f61440a2c30 0x7f61440a50f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6158199070 tx=0x7f614000b390 comp rx=0 tx=0).stop 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 msgr2=0x7f6158197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158197b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61480314e0 tx=0x7f61480042e0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 shutdown_connections 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f61440a2c30 0x7f61440a50f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61581034e0 0x7f615819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6158102b20 0x7f6158198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 --2- 172.21.15.190:0/1217211959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6158101920 0x7f6158197b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:26.937 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 >> 172.21.15.190:0/1217211959 conn(0x7f61580fd0f0 msgr2=0x7f6158105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:26.937 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 shutdown_connections 2024-08-19T01:32:26.937 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:26.934+0000 7f615e3fc640 1 -- 172.21.15.190:0/1217211959 wait complete. 2024-08-19T01:32:27.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:26 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:26 smithi190 ceph-mon[22032]: from='osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:32:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:26 smithi190 ceph-mon[22032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:32:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:26 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:26 smithi146 ceph-mon[22098]: from='osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:32:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:26 smithi146 ceph-mon[22098]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:32:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:26 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:26 smithi103 ceph-mon[18436]: from='osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:32:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:26 smithi103 ceph-mon[18436]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-19T01:32:27.650 DEBUG:teuthology.orchestra.run.smithi190:osd.11> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.11.service 2024-08-19T01:32:27.652 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-19T01:32:27.653 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: osdmap e76: 12 total, 11 up, 12 in 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: from='osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:27 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:27.918 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: osdmap e76: 12 total, 11 up, 12 in 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: from='osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:28.041 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:28.042 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:27 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: osdmap e76: 12 total, 11 up, 12 in 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: from='osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:28.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:27 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:28.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 -- 172.21.15.103:0/3796240290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec107690 msgr2=0x7f74ec109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:28.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 --2- 172.21.15.103:0/3796240290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec107690 0x7f74ec109a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009ca0 tx=0x7f74e402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 -- 172.21.15.103:0/3796240290 shutdown_connections 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 --2- 172.21.15.103:0/3796240290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec107690 0x7f74ec109a80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 --2- 172.21.15.103:0/3796240290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74ec104d60 0x7f74ec107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 --2- 172.21.15.103:0/3796240290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ec102430 0x7f74ec104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 -- 172.21.15.103:0/3796240290 >> 172.21.15.103:0/3796240290 conn(0x7f74ec0fbfd0 msgr2=0x7f74ec0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 -- 172.21.15.103:0/3796240290 shutdown_connections 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.473+0000 7f74f1d27640 1 -- 172.21.15.103:0/3796240290 wait complete. 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 Processor -- start 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 -- start start 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74ec102430 0x7f74ec197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec104d60 0x7f74ec198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ec107690 0x7f74ec19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10d520 con 0x7f74ec104d60 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10d3a0 con 0x7f74ec107690 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74f1d27640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec10d6a0 con 0x7f74ec102430 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec104d60 0x7f74ec198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec104d60 0x7f74ec198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33482/0 (socket says 172.21.15.103:33482) 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 -- 172.21.15.103:0/308288109 learned_addr learned my addr 172.21.15.103:0/308288109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74ebfff640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ec107690 0x7f74ec19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74eb7fe640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74ec102430 0x7f74ec197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 -- 172.21.15.103:0/308288109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74ec102430 msgr2=0x7f74ec197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74ec102430 0x7f74ec197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 -- 172.21.15.103:0/308288109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ec107690 msgr2=0x7f74ec19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ec107690 0x7f74ec19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 -- 172.21.15.103:0/308288109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec19f800 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.474+0000 7f74e3fff640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec104d60 0x7f74ec198110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c970 tx=0x7f74d400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.475+0000 7f74e97fa640 1 -- 172.21.15.103:0/308288109 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4007bb0 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.475+0000 7f74e97fa640 1 -- 172.21.15.103:0/308288109 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74d4007d50 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.475+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ec19faf0 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.475+0000 7f74e97fa640 1 -- 172.21.15.103:0/308288109 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4018800 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.475+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ec19ffe0 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.476+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ec0fdac0 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.477+0000 7f74e97fa640 1 -- 172.21.15.103:0/308288109 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f74d400f450 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.478+0000 7f74e97fa640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f74d80a2bb0 0x7f74d80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.478+0000 7f74eb7fe640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f74d80a2bb0 0x7f74d80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.478+0000 7f74e97fa640 1 -- 172.21.15.103:0/308288109 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6847+0+0 (secure 0 0 0) 0x7f74d40cee40 con 0x7f74ec104d60 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.479+0000 7f74eb7fe640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f74d80a2bb0 0x7f74d80a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009990 tx=0x7f74dc009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:28.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.481+0000 7f74e97fa640 1 -- 172.21.15.103:0/308288109 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f74d4014030 con 0x7f74ec104d60 2024-08-19T01:32:28.514 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:32:28.237+0000 7fad4ca98640 -1 osd.11 0 waiting for initial osdmap 2024-08-19T01:32:28.514 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:32:28.248+0000 7fad448b8640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-19T01:32:28.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.651+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f74ec02d830 con 0x7f74ec104d60 2024-08-19T01:32:28.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.651+0000 7f74e97fa640 1 -- 172.21.15.103:0/308288109 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f74d409c360 con 0x7f74ec104d60 2024-08-19T01:32:28.652 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:28.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f74d80a2bb0 msgr2=0x7f74d80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:28.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f74d80a2bb0 0x7f74d80a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009990 tx=0x7f74dc009110 comp rx=0 tx=0).stop 2024-08-19T01:32:28.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec104d60 msgr2=0x7f74ec198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:28.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec104d60 0x7f74ec198110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c970 tx=0x7f74d400cd70 comp rx=0 tx=0).stop 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 shutdown_connections 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f74d80a2bb0 0x7f74d80a5070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74ec107690 0x7f74ec19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec104d60 0x7f74ec198110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.653+0000 7f74f1d27640 1 --2- 172.21.15.103:0/308288109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74ec102430 0x7f74ec197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.654+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 >> 172.21.15.103:0/308288109 conn(0x7f74ec0fbfd0 msgr2=0x7f74ec102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.654+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 shutdown_connections 2024-08-19T01:32:28.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:28.654+0000 7f74f1d27640 1 -- 172.21.15.103:0/308288109 wait complete. 2024-08-19T01:32:28.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:28 smithi103 ceph-mon[18436]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:32:28.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:28 smithi103 ceph-mon[18436]: osdmap e77: 12 total, 11 up, 12 in 2024-08-19T01:32:28.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:28.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:28 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:28.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:28 smithi103 ceph-mon[18436]: from='osd.11 ' entity='osd.11' 2024-08-19T01:32:28.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:28 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/308288109' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:32:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-mon[22032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:32:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-mon[22032]: osdmap e77: 12 total, 11 up, 12 in 2024-08-19T01:32:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-mon[22032]: from='osd.11 ' entity='osd.11' 2024-08-19T01:32:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:28 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/308288109' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:32:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:28 smithi146 ceph-mon[22098]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-08-19T01:32:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:28 smithi146 ceph-mon[22098]: osdmap e77: 12 total, 11 up, 12 in 2024-08-19T01:32:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:28 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:28 smithi146 ceph-mon[22098]: from='osd.11 ' entity='osd.11' 2024-08-19T01:32:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:28 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/308288109' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:32:29.512 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1724031125,"num_in_osds":12,"osd_in_since":1724031130,"num_remapped_pgs":0} 2024-08-19T01:32:30.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:29 smithi190 ceph-mon[22032]: purged_snaps scrub starts 2024-08-19T01:32:30.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:29 smithi190 ceph-mon[22032]: purged_snaps scrub ok 2024-08-19T01:32:30.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:29 smithi190 ceph-mon[22032]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:30.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:29 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:30.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:29 smithi190 ceph-mon[22032]: osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] boot 2024-08-19T01:32:30.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:29 smithi190 ceph-mon[22032]: osdmap e78: 12 total, 12 up, 12 in 2024-08-19T01:32:30.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:29 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:29 smithi146 ceph-mon[22098]: purged_snaps scrub starts 2024-08-19T01:32:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:29 smithi146 ceph-mon[22098]: purged_snaps scrub ok 2024-08-19T01:32:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:29 smithi146 ceph-mon[22098]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:29 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:29 smithi146 ceph-mon[22098]: osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] boot 2024-08-19T01:32:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:29 smithi146 ceph-mon[22098]: osdmap e78: 12 total, 12 up, 12 in 2024-08-19T01:32:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:29 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:29 smithi103 ceph-mon[18436]: purged_snaps scrub starts 2024-08-19T01:32:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:29 smithi103 ceph-mon[18436]: purged_snaps scrub ok 2024-08-19T01:32:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:29 smithi103 ceph-mon[18436]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-08-19T01:32:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:29 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:29 smithi103 ceph-mon[18436]: osd.11 [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] boot 2024-08-19T01:32:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:29 smithi103 ceph-mon[18436]: osdmap e78: 12 total, 12 up, 12 in 2024-08-19T01:32:30.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:29 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-19T01:32:30.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-19T01:32:30.770 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:31.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: osdmap e79: 12 total, 12 up, 12 in 2024-08-19T01:32:31.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:31.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:31.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:31.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 -- 172.21.15.103:0/2047005638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa610101930 msgr2=0x7fa610101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 --2- 172.21.15.103:0/2047005638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa610101930 0x7fa610101d30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa600009900 tx=0x7fa60002a410 comp rx=0 tx=0).stop 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 -- 172.21.15.103:0/2047005638 shutdown_connections 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 --2- 172.21.15.103:0/2047005638 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6101034f0 0x7fa610109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 --2- 172.21.15.103:0/2047005638 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa610102b30 0x7fa610102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 --2- 172.21.15.103:0/2047005638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa610101930 0x7fa610101d30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 -- 172.21.15.103:0/2047005638 >> 172.21.15.103:0/2047005638 conn(0x7fa6100fd0c0 msgr2=0x7fa6100ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.276+0000 7fa615821640 1 -- 172.21.15.103:0/2047005638 shutdown_connections 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 -- 172.21.15.103:0/2047005638 wait complete. 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 Processor -- start 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 -- start start 2024-08-19T01:32:31.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa610101930 0x7fa610197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa610102b30 0x7fa6101980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6101034f0 0x7fa61019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010d4e0 con 0x7fa6101034f0 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010d360 con 0x7fa610101930 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa615821640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa61010d660 con 0x7fa610102b30 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa60f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6101034f0 0x7fa61019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa60f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6101034f0 0x7fa61019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33502/0 (socket says 172.21.15.103:33502) 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa60f7fe640 1 -- 172.21.15.103:0/3721345130 learned_addr learned my addr 172.21.15.103:0/3721345130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.277+0000 7fa607fff640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa610102b30 0x7fa6101980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa60effd640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa610101930 0x7fa610197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa60f7fe640 1 -- 172.21.15.103:0/3721345130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa610102b30 msgr2=0x7fa6101980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa60f7fe640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa610102b30 0x7fa6101980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa60f7fe640 1 -- 172.21.15.103:0/3721345130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa610101930 msgr2=0x7fa610197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa60f7fe640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa610101930 0x7fa610197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa60f7fe640 1 -- 172.21.15.103:0/3721345130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61019f7e0 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa60f7fe640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6101034f0 0x7fa61019f170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa60800b970 tx=0x7fa60800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa61481f640 1 -- 172.21.15.103:0/3721345130 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608019070 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa61481f640 1 -- 172.21.15.103:0/3721345130 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa608014440 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa61481f640 1 -- 172.21.15.103:0/3721345130 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6080075d0 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61019fad0 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.278+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61019ffc0 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.279+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa610108d20 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.281+0000 7fa61481f640 1 -- 172.21.15.103:0/3721345130 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa60801c070 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.281+0000 7fa61481f640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa5fc0a2bb0 0x7fa5fc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.281+0000 7fa60effd640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa5fc0a2bb0 0x7fa5fc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.281+0000 7fa61481f640 1 -- 172.21.15.103:0/3721345130 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa6080ce630 con 0x7fa6101034f0 2024-08-19T01:32:31.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.282+0000 7fa60effd640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa5fc0a2bb0 0x7fa5fc0a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa600000c00 tx=0x7fa6000314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:31.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.284+0000 7fa61481f640 1 -- 172.21.15.103:0/3721345130 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa60809b3e0 con 0x7fa6101034f0 2024-08-19T01:32:31.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.453+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa61002d5e0 con 0x7fa6101034f0 2024-08-19T01:32:31.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.453+0000 7fa61481f640 1 -- 172.21.15.103:0/3721345130 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7fa61002d5e0 con 0x7fa6101034f0 2024-08-19T01:32:31.454 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:31.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa5fc0a2bb0 msgr2=0x7fa5fc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa5fc0a2bb0 0x7fa5fc0a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa600000c00 tx=0x7fa6000314a0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6101034f0 msgr2=0x7fa61019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6101034f0 0x7fa61019f170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa60800b970 tx=0x7fa60800be30 comp rx=0 tx=0).stop 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 shutdown_connections 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa5fc0a2bb0 0x7fa5fc0a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6101034f0 0x7fa61019f170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa610102b30 0x7fa6101980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 --2- 172.21.15.103:0/3721345130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa610101930 0x7fa610197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 >> 172.21.15.103:0/3721345130 conn(0x7fa6100fd0c0 msgr2=0x7fa610105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 shutdown_connections 2024-08-19T01:32:31.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:31.455+0000 7fa615821640 1 -- 172.21.15.103:0/3721345130 wait complete. 2024-08-19T01:32:31.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: osdmap e79: 12 total, 12 up, 12 in 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: osdmap e79: 12 total, 12 up, 12 in 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:31.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:31.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:32.114 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1724031149,"num_in_osds":12,"osd_in_since":1724031130,"num_remapped_pgs":0} 2024-08-19T01:32:32.115 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:32:32.363 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:32.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:32 smithi103 ceph-mon[18436]: Detected new or changed devices on smithi190 2024-08-19T01:32:32.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:32 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi190 to 4272M 2024-08-19T01:32:32.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:32 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3721345130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:32:32.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:32 smithi190 ceph-mon[22032]: Detected new or changed devices on smithi190 2024-08-19T01:32:32.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:32 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi190 to 4272M 2024-08-19T01:32:32.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:32 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3721345130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:32:32.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:32 smithi146 ceph-mon[22098]: Detected new or changed devices on smithi190 2024-08-19T01:32:32.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:32 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi190 to 4272M 2024-08-19T01:32:32.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:32 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3721345130' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-19T01:32:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 -- 172.21.15.103:0/3097649477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 msgr2=0x7ff248100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/3097649477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff248100c10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff238009930 tx=0x7ff23802a420 comp rx=0 tx=0).stop 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 -- 172.21.15.103:0/3097649477 shutdown_connections 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/3097649477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff248101150 0x7ff24810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/3097649477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff248100c10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/3097649477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2481067b0 0x7ff248106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 -- 172.21.15.103:0/3097649477 >> 172.21.15.103:0/3097649477 conn(0x7ff2480fc4d0 msgr2=0x7ff2480fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 -- 172.21.15.103:0/3097649477 shutdown_connections 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.844+0000 7ff24f2ea640 1 -- 172.21.15.103:0/3097649477 wait complete. 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 Processor -- start 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 -- start start 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff24810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff248101150 0x7ff24810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2481067b0 0x7ff24810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff248111b90 con 0x7ff2481007b0 2024-08-19T01:32:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff248111a10 con 0x7ff248101150 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24f2ea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff248111d10 con 0x7ff2481067b0 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff24810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff24810df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58658/0 (socket says 172.21.15.103:58658) 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 -- 172.21.15.103:0/104361921 learned_addr learned my addr 172.21.15.103:0/104361921 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24c885640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff248101150 0x7ff24810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d887640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2481067b0 0x7ff24810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 -- 172.21.15.103:0/104361921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2481067b0 msgr2=0x7ff24810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2481067b0 0x7ff24810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 -- 172.21.15.103:0/104361921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff248101150 msgr2=0x7ff24810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff248101150 0x7ff24810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24d086640 1 -- 172.21.15.103:0/104361921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2481a6590 con 0x7ff2481007b0 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.845+0000 7ff24c885640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff248101150 0x7ff24810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:32:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.846+0000 7ff24d086640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff24810df90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff24400cce0 tx=0x7ff24400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.846+0000 7ff23e7fc640 1 -- 172.21.15.103:0/104361921 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400f650 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.846+0000 7ff23e7fc640 1 -- 172.21.15.103:0/104361921 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff24400fca0 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.846+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2481a6880 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.846+0000 7ff23e7fc640 1 -- 172.21.15.103:0/104361921 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244018b90 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.846+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2481a6d90 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.847+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff248100c10 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.848+0000 7ff23e7fc640 1 -- 172.21.15.103:0/104361921 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff24400f7f0 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.849+0000 7ff23e7fc640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2340a2b80 0x7ff2340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.849+0000 7ff24c885640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2340a2b80 0x7ff2340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.849+0000 7ff23e7fc640 1 -- 172.21.15.103:0/104361921 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff2440cf400 con 0x7ff2481007b0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.849+0000 7ff24c885640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2340a2b80 0x7ff2340a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff238000c00 tx=0x7ff238031cf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:32.852+0000 7ff23e7fc640 1 -- 172.21.15.103:0/104361921 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff244014030 con 0x7ff2481007b0 2024-08-19T01:32:33.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.020+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff24802d5e0 con 0x7ff2481007b0 2024-08-19T01:32:33.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.021+0000 7ff23e7fc640 1 -- 172.21.15.103:0/104361921 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18538 (secure 0 0 0) 0x7ff24409c890 con 0x7ff2481007b0 2024-08-19T01:32:33.022 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:33.022 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":79,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:32:30.245929+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:32:33.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2340a2b80 msgr2=0x7ff2340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:33.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2340a2b80 0x7ff2340a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff238000c00 tx=0x7ff238031cf0 comp rx=0 tx=0).stop 2024-08-19T01:32:33.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 msgr2=0x7ff24810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:33.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff24810df90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff24400cce0 tx=0x7ff24400a620 comp rx=0 tx=0).stop 2024-08-19T01:32:33.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 shutdown_connections 2024-08-19T01:32:33.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2340a2b80 0x7ff2340a5040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:33.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2481067b0 0x7ff24810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:33.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff248101150 0x7ff24810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:33.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 --2- 172.21.15.103:0/104361921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2481007b0 0x7ff24810df90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:33.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 >> 172.21.15.103:0/104361921 conn(0x7ff2480fc4d0 msgr2=0x7ff2480fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:33.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 shutdown_connections 2024-08-19T01:32:33.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:33.023+0000 7ff24f2ea640 1 -- 172.21.15.103:0/104361921 wait complete. 2024-08-19T01:32:33.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:33 smithi103 ceph-mon[18436]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:33.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:33 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/104361921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:33.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:33 smithi190 ceph-mon[22032]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:33.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:33 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/104361921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:33.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:33 smithi146 ceph-mon[22098]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:33.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:33 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/104361921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:32:33.748 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-19T01:29:05.010792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-19T01:32:33.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-19T01:32:33.997 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3709921249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 msgr2=0x7fe81810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3709921249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe81810d680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe810009cd0 tx=0x7fe81002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3709921249 shutdown_connections 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3709921249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe81810d680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3709921249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8181007b0 0x7fe818100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3709921249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8181067b0 0x7fe818106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3709921249 >> 172.21.15.103:0/3709921249 conn(0x7fe8180fc4d0 msgr2=0x7fe8180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3709921249 shutdown_connections 2024-08-19T01:32:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.470+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3709921249 wait complete. 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 Processor -- start 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 -- start start 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8181007b0 0x7fe818102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe8181028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8181067b0 0x7fe818102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe818110aa0 con 0x7fe818101150 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe818110920 con 0x7fe8181067b0 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe81d6d9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe818110c20 con 0x7fe8181007b0 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe8181028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe8181028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58676/0 (socket says 172.21.15.103:58676) 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 -- 172.21.15.103:0/3010834626 learned_addr learned my addr 172.21.15.103:0/3010834626 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe816ffd640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8181007b0 0x7fe818102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe8177fe640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8181067b0 0x7fe818102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 -- 172.21.15.103:0/3010834626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8181007b0 msgr2=0x7fe818102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8181007b0 0x7fe818102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 -- 172.21.15.103:0/3010834626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8181067b0 msgr2=0x7fe818102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8181067b0 0x7fe818102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.471+0000 7fe80e7fc640 1 -- 172.21.15.103:0/3010834626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8181a65b0 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.472+0000 7fe80e7fc640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe8181028c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe80000b4a0 tx=0x7fe80000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.472+0000 7fe814ff9640 1 -- 172.21.15.103:0/3010834626 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800014070 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.472+0000 7fe814ff9640 1 -- 172.21.15.103:0/3010834626 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe800010ac0 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.472+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8181a68a0 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.472+0000 7fe814ff9640 1 -- 172.21.15.103:0/3010834626 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800017c30 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.472+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181a6de0 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.473+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe818100c10 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.474+0000 7fe814ff9640 1 -- 172.21.15.103:0/3010834626 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe800017070 con 0x7fe818101150 2024-08-19T01:32:34.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.474+0000 7fe814ff9640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7f80a2b80 0x7fe7f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:34.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.475+0000 7fe816ffd640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7f80a2b80 0x7fe7f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:34.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.475+0000 7fe814ff9640 1 -- 172.21.15.103:0/3010834626 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fe8000ce320 con 0x7fe818101150 2024-08-19T01:32:34.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.475+0000 7fe816ffd640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7f80a2b80 0x7fe7f80a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe808005fd0 tx=0x7fe808005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:34.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.479+0000 7fe814ff9640 1 -- 172.21.15.103:0/3010834626 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe80009b720 con 0x7fe818101150 2024-08-19T01:32:34.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.644+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8181a70a0 con 0x7fe818101150 2024-08-19T01:32:34.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.645+0000 7fe814ff9640 1 -- 172.21.15.103:0/3010834626 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fe80009b020 con 0x7fe818101150 2024-08-19T01:32:34.646 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7f80a2b80 msgr2=0x7fe7f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7f80a2b80 0x7fe7f80a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe808005fd0 tx=0x7fe808005850 comp rx=0 tx=0).stop 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 msgr2=0x7fe8181028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe8181028c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe80000b4a0 tx=0x7fe80000b960 comp rx=0 tx=0).stop 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 shutdown_connections 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7f80a2b80 0x7fe7f80a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8181067b0 0x7fe818102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe818101150 0x7fe8181028c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 --2- 172.21.15.103:0/3010834626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8181007b0 0x7fe818102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:34.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 >> 172.21.15.103:0/3010834626 conn(0x7fe8180fc4d0 msgr2=0x7fe818104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:34.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 shutdown_connections 2024-08-19T01:32:34.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:34.648+0000 7fe81d6d9640 1 -- 172.21.15.103:0/3010834626 wait complete. 2024-08-19T01:32:34.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:34 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3010834626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:32:34.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:34 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3010834626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:32:35.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:34 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3010834626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:32:35.314 INFO:tasks.cephadm:Adding mds.a on smithi103 2024-08-19T01:32:35.314 INFO:tasks.cephadm:Adding mds.d on smithi103 2024-08-19T01:32:35.314 INFO:tasks.cephadm:Adding mds.g on smithi103 2024-08-19T01:32:35.314 INFO:tasks.cephadm:Adding mds.j on smithi103 2024-08-19T01:32:35.314 INFO:tasks.cephadm:Adding mds.b on smithi146 2024-08-19T01:32:35.314 INFO:tasks.cephadm:Adding mds.e on smithi146 2024-08-19T01:32:35.314 INFO:tasks.cephadm:Adding mds.h on smithi146 2024-08-19T01:32:35.315 INFO:tasks.cephadm:Adding mds.k on smithi146 2024-08-19T01:32:35.315 INFO:tasks.cephadm:Adding mds.c on smithi190 2024-08-19T01:32:35.315 INFO:tasks.cephadm:Adding mds.f on smithi190 2024-08-19T01:32:35.315 INFO:tasks.cephadm:Adding mds.i on smithi190 2024-08-19T01:32:35.315 INFO:tasks.cephadm:Adding mds.l on smithi190 2024-08-19T01:32:35.315 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph orch apply mds all '12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l' 2024-08-19T01:32:35.578 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:32:35.971 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:35 smithi190 ceph-mon[22032]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:36.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:35 smithi103 ceph-mon[18436]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 -- 172.21.15.190:0/824127033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0100060 msgr2=0x7fcee0109f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/824127033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0100060 0x7fcee0109f30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fced8009ca0 tx=0x7fced802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 -- 172.21.15.190:0/824127033 shutdown_connections 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/824127033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0100060 0x7fcee0109f30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/824127033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee0103910 0x7fcee00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/824127033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0102f40 0x7fcee0103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 -- 172.21.15.190:0/824127033 >> 172.21.15.190:0/824127033 conn(0x7fcee00fb770 msgr2=0x7fcee00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 -- 172.21.15.190:0/824127033 shutdown_connections 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.051+0000 7fcee6bc4640 1 -- 172.21.15.190:0/824127033 wait complete. 2024-08-19T01:32:36.080 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 Processor -- start 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 -- start start 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0100060 0x7fcee0197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0102f40 0x7fcee0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee0103910 0x7fcee019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010d4a0 con 0x7fcee0100060 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010d320 con 0x7fcee0103910 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee6bc4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010d620 con 0x7fcee0102f40 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0102f40 0x7fcee0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0102f40 0x7fcee0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:41390/0 (socket says 172.21.15.190:41390) 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 -- 172.21.15.190:0/1120074153 learned_addr learned my addr 172.21.15.190:0/1120074153 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee5161640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee0103910 0x7fcee019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcee4960640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0100060 0x7fcee0197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 -- 172.21.15.190:0/1120074153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee0103910 msgr2=0x7fcee019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee0103910 0x7fcee019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 -- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0100060 msgr2=0x7fcee0197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0100060 0x7fcee0197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 -- 172.21.15.190:0/1120074153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee019f670 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fcedffff640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0102f40 0x7fcee0197f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c970 tx=0x7fcecc00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.052+0000 7fceddffb640 1 -- 172.21.15.190:0/1120074153 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc004220 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.053+0000 7fceddffb640 1 -- 172.21.15.190:0/1120074153 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcecc0043c0 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.053+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee019f960 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.053+0000 7fceddffb640 1 -- 172.21.15.190:0/1120074153 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc00f450 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.053+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee019fef0 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.054+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee0108e10 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.055+0000 7fceddffb640 1 -- 172.21.15.190:0/1120074153 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcecc0079b0 con 0x7fcee0102f40 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.055+0000 7fceddffb640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fced00a2c30 0x7fced00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:36.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.055+0000 7fceddffb640 1 -- 172.21.15.190:0/1120074153 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fcecc0ce7e0 con 0x7fcee0102f40 2024-08-19T01:32:36.083 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.055+0000 7fcee4960640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fced00a2c30 0x7fced00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:36.083 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.056+0000 7fcee4960640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fced00a2c30 0x7fced00a50f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcee006a1a0 tx=0x7fced4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:36.083 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.059+0000 7fceddffb640 1 -- 172.21.15.190:0/1120074153 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcecc014030 con 0x7fcee0102f40 2024-08-19T01:32:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:35 smithi146 ceph-mon[22098]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:36.239 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.237+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}) -- 0x7fcee0062460 con 0x7fced00a2c30 2024-08-19T01:32:36.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.245+0000 7fceddffb640 1 -- 172.21.15.190:0/1120074153 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fcee0062460 con 0x7fced00a2c30 2024-08-19T01:32:36.247 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mds.all update... 2024-08-19T01:32:36.249 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fced00a2c30 msgr2=0x7fced00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fced00a2c30 0x7fced00a50f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcee006a1a0 tx=0x7fced4005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0102f40 msgr2=0x7fcee0197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0102f40 0x7fcee0197f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c970 tx=0x7fcecc00ce30 comp rx=0 tx=0).stop 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 shutdown_connections 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fced00a2c30 0x7fced00a50f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcee0103910 0x7fcee019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcee0102f40 0x7fcee0197f80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 --2- 172.21.15.190:0/1120074153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0100060 0x7fcee0197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 >> 172.21.15.190:0/1120074153 conn(0x7fcee00fb770 msgr2=0x7fcee0105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 shutdown_connections 2024-08-19T01:32:36.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-08-19T01:32:36.247+0000 7fcee6bc4640 1 -- 172.21.15.190:0/1120074153 wait complete. 2024-08-19T01:32:37.197 DEBUG:teuthology.orchestra.run.smithi103:mds.a> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.a.service 2024-08-19T01:32:37.200 DEBUG:teuthology.orchestra.run.smithi103:mds.d> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.d.service 2024-08-19T01:32:37.243 DEBUG:teuthology.orchestra.run.smithi103:mds.g> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.g.service 2024-08-19T01:32:37.286 DEBUG:teuthology.orchestra.run.smithi103:mds.j> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.j.service 2024-08-19T01:32:37.328 DEBUG:teuthology.orchestra.run.smithi146:mds.b> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.b.service 2024-08-19T01:32:37.331 DEBUG:teuthology.orchestra.run.smithi146:mds.e> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.e.service 2024-08-19T01:32:37.375 DEBUG:teuthology.orchestra.run.smithi146:mds.h> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.h.service 2024-08-19T01:32:37.418 DEBUG:teuthology.orchestra.run.smithi146:mds.k> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.k.service 2024-08-19T01:32:37.461 DEBUG:teuthology.orchestra.run.smithi190:mds.c> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.c.service 2024-08-19T01:32:37.464 DEBUG:teuthology.orchestra.run.smithi190:mds.f> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.f.service 2024-08-19T01:32:37.483 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:37.483 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-19T01:32:37.483 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:37.483 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:37.483 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:37.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:37.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:37.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:37.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: Deploying daemon mds.l on smithi190 2024-08-19T01:32:37.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:37 smithi190 ceph-mon[22032]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:37.485 DEBUG:teuthology.orchestra.run.smithi190:mds.i> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.i.service 2024-08-19T01:32:37.527 DEBUG:teuthology.orchestra.run.smithi190:mds.l> sudo journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.l.service 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: Deploying daemon mds.l on smithi190 2024-08-19T01:32:37.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:37 smithi103 ceph-mon[18436]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:37.570 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-19T01:32:37.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:37.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:32:37.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-19T01:32:37.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:37.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:32:37.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:32:37.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:37.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:37.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:37.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:37.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: Deploying daemon mds.l on smithi190 2024-08-19T01:32:37.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:37 smithi146 ceph-mon[22098]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:37.821 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:38.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.313+0000 7fe61cf42640 1 -- 172.21.15.103:0/1813027705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 msgr2=0x7fe61810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:38.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.313+0000 7fe61cf42640 1 --2- 172.21.15.103:0/1813027705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61810e340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe610009ca0 tx=0x7fe61002a950 comp rx=0 tx=0).stop 2024-08-19T01:32:38.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 -- 172.21.15.103:0/1813027705 shutdown_connections 2024-08-19T01:32:38.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 --2- 172.21.15.103:0/1813027705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61810e340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 --2- 172.21.15.103:0/1813027705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618103390 0x7fe61810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 --2- 172.21.15.103:0/1813027705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe618102a70 0x7fe618102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 -- 172.21.15.103:0/1813027705 >> 172.21.15.103:0/1813027705 conn(0x7fe6180fc4d0 msgr2=0x7fe6180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 -- 172.21.15.103:0/1813027705 shutdown_connections 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 -- 172.21.15.103:0/1813027705 wait complete. 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 Processor -- start 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.314+0000 7fe61cf42640 1 -- start start 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102a70 0x7fe618198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe618103390 0x7fe618198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111b50 con 0x7fe61810bea0 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181119d0 con 0x7fe618102a70 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111cd0 con 0x7fe618103390 2024-08-19T01:32:38.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58692/0 (socket says 172.21.15.103:58692) 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 -- 172.21.15.103:0/3594535463 learned_addr learned my addr 172.21.15.103:0/3594535463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe615d9b640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe618103390 0x7fe618198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61659c640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102a70 0x7fe618198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 -- 172.21.15.103:0/3594535463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe618103390 msgr2=0x7fe618198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe618103390 0x7fe618198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 -- 172.21.15.103:0/3594535463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102a70 msgr2=0x7fe618198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102a70 0x7fe618198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 -- 172.21.15.103:0/3594535463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61819d0f0 con 0x7fe61810bea0 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe616d9d640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61819ca10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe610004af0 tx=0x7fe610031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe6077fe640 1 -- 172.21.15.103:0/3594535463 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610034040 con 0x7fe61810bea0 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe6077fe640 1 -- 172.21.15.103:0/3594535463 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6100308f0 con 0x7fe61810bea0 2024-08-19T01:32:38.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61819d380 con 0x7fe61810bea0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe6077fe640 1 -- 172.21.15.103:0/3594535463 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610004600 con 0x7fe61810bea0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.315+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6181a4cc0 con 0x7fe61810bea0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.317+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61806ada0 con 0x7fe61810bea0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.317+0000 7fe6077fe640 1 -- 172.21.15.103:0/3594535463 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe610037020 con 0x7fe61810bea0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.318+0000 7fe6077fe640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6000a2b80 0x7fe6000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.318+0000 7fe61659c640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6000a2b80 0x7fe6000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.318+0000 7fe6077fe640 1 -- 172.21.15.103:0/3594535463 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fe6100ef820 con 0x7fe61810bea0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.318+0000 7fe61659c640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6000a2b80 0x7fe6000a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe608009bd0 tx=0x7fe608005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.322+0000 7fe6077fe640 1 -- 172.21.15.103:0/3594535463 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe6100bcc20 con 0x7fe61810bea0 2024-08-19T01:32:38.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.523+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6181994a0 con 0x7fe61810bea0 2024-08-19T01:32:38.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.523+0000 7fe6077fe640 1 -- 172.21.15.103:0/3594535463 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fe6100bc520 con 0x7fe61810bea0 2024-08-19T01:32:38.524 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:38.525 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"btime":"2024-08-19T01:26:05:629511+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-19T01:32:38.525 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 1 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.525+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6000a2b80 msgr2=0x7fe6000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.525+0000 7fe61cf42640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6000a2b80 0x7fe6000a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe608009bd0 tx=0x7fe608005f90 comp rx=0 tx=0).stop 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.525+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 msgr2=0x7fe61819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.525+0000 7fe61cf42640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61819ca10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe610004af0 tx=0x7fe610031f50 comp rx=0 tx=0).stop 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 shutdown_connections 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6000a2b80 0x7fe6000a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe61810bea0 0x7fe61819ca10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe618103390 0x7fe618198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 --2- 172.21.15.103:0/3594535463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102a70 0x7fe618198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:38.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 >> 172.21.15.103:0/3594535463 conn(0x7fe6180fc4d0 msgr2=0x7fe6180fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:38.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 shutdown_connections 2024-08-19T01:32:38.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:38.526+0000 7fe61cf42640 1 -- 172.21.15.103:0/3594535463 wait complete. 2024-08-19T01:32:38.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:38 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3594535463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:38.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:38 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3594535463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:38.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:38 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3594535463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:39.085 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:38 smithi190 systemd[1]: Starting Ceph mds.l for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:39.352 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 podman[51589]: 2024-08-19 01:32:38.987089609 +0000 UTC m=+0.017342755 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:39.352 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 podman[51589]: 2024-08-19 01:32:39.088138572 +0000 UTC m=+0.118391705 container create 9f525248d217e1ac8cebc14c8caf8447e66149e987555c978fbf95bf50baab3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-19T01:32:39.352 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 podman[51589]: 2024-08-19 01:32:39.324718629 +0000 UTC m=+0.354971762 container init 9f525248d217e1ac8cebc14c8caf8447e66149e987555c978fbf95bf50baab3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-08-19T01:32:39.352 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 podman[51589]: 2024-08-19 01:32:39.327582514 +0000 UTC m=+0.357835655 container start 9f525248d217e1ac8cebc14c8caf8447e66149e987555c978fbf95bf50baab3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-08-19T01:32:39.352 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mds[51608]: -- 172.21.15.190:0/179005684 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee90b7b860 con 0x55ee90b59400 2024-08-19T01:32:39.630 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l[51604]: starting mds.l at 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 bash[51589]: 9f525248d217e1ac8cebc14c8caf8447e66149e987555c978fbf95bf50baab3e 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:32:39 smithi190 systemd[1]: Started Ceph mds.l for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6832/1574351198,v1:172.21.15.190:6833/1574351198] up:boot 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: fsmap 1 up:standby 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:39.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:39.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:39.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:39.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6832/1574351198,v1:172.21.15.190:6833/1574351198] up:boot 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: fsmap 1 up:standby 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:39.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6832/1574351198,v1:172.21.15.190:6833/1574351198] up:boot 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: fsmap 1 up:standby 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:40.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:40.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:40.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:40.762 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:40 smithi190 ceph-mon[22032]: Deploying daemon mds.k on smithi146 2024-08-19T01:32:40.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:40 smithi146 ceph-mon[22098]: Deploying daemon mds.k on smithi146 2024-08-19T01:32:41.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:40 smithi103 ceph-mon[18436]: Deploying daemon mds.k on smithi146 2024-08-19T01:32:41.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:41.514 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:41.589 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:41 smithi146 systemd[1]: Starting Ceph mds.k for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:41.854 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:41 smithi146 ceph-mon[22098]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:41.855 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:41 smithi146 podman[51249]: 2024-08-19 01:32:41.619207827 +0000 UTC m=+0.016957095 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:41.855 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:41 smithi146 podman[51249]: 2024-08-19 01:32:41.725132073 +0000 UTC m=+0.122881336 container create 28351662d1da3c6cb4b6609203e6b3f67ada978448eb81b81407be42ac70ae7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-19T01:32:41.876 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:41 smithi103 ceph-mon[18436]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:41.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:41 smithi190 ceph-mon[22032]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 -- 172.21.15.103:0/68386360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 msgr2=0x7f2440100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 --2- 172.21.15.103:0/68386360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440100f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2430009900 tx=0x7f243002a480 comp rx=0 tx=0).stop 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 -- 172.21.15.103:0/68386360 shutdown_connections 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 --2- 172.21.15.103:0/68386360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2440102c20 0x7f244010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 --2- 172.21.15.103:0/68386360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24401022a0 0x7f24401026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 --2- 172.21.15.103:0/68386360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440100f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 -- 172.21.15.103:0/68386360 >> 172.21.15.103:0/68386360 conn(0x7f24400fc500 msgr2=0x7f24400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 -- 172.21.15.103:0/68386360 shutdown_connections 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.955+0000 7f2444d44640 1 -- 172.21.15.103:0/68386360 wait complete. 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 Processor -- start 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 -- start start 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24401022a0 0x7f2440198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2440102c20 0x7f244019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2440110b60 con 0x7f2440100b50 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f24401109e0 con 0x7f24401022a0 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2444d44640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2440110ce0 con 0x7f2440102c20 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39762/0 (socket says 172.21.15.103:39762) 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243e59c640 1 -- 172.21.15.103:0/2298526684 learned_addr learned my addr 172.21.15.103:0/2298526684 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243ed9d640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2440102c20 0x7f244019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f2437fff640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24401022a0 0x7f2440198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243ed9d640 1 -- 172.21.15.103:0/2298526684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24401022a0 msgr2=0x7f2440198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243ed9d640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24401022a0 0x7f2440198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243ed9d640 1 -- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 msgr2=0x7f2440198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243ed9d640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243ed9d640 1 -- 172.21.15.103:0/2298526684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244019d1b0 con 0x7f2440102c20 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.956+0000 7f243e59c640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.957+0000 7f243ed9d640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2440102c20 0x7f244019cad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f243800da10 tx=0x7f243800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.957+0000 7f24377fe640 1 -- 172.21.15.103:0/2298526684 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243800bd70 con 0x7f2440102c20 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.957+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244019d4a0 con 0x7f2440102c20 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.957+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24401a4d80 con 0x7f2440102c20 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.957+0000 7f24377fe640 1 -- 172.21.15.103:0/2298526684 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2438003e20 con 0x7f2440102c20 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.957+0000 7f24377fe640 1 -- 172.21.15.103:0/2298526684 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380106c0 con 0x7f2440102c20 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.958+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2420005290 con 0x7f2440102c20 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.959+0000 7f24377fe640 1 -- 172.21.15.103:0/2298526684 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2438003970 con 0x7f2440102c20 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.960+0000 7f24377fe640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f242c0a2b80 0x7f242c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.960+0000 7f243e59c640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f242c0a2b80 0x7f242c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:41.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.960+0000 7f24377fe640 1 -- 172.21.15.103:0/2298526684 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f24380ceab0 con 0x7f2440102c20 2024-08-19T01:32:41.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.960+0000 7f243e59c640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f242c0a2b80 0x7f242c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24300096c0 tx=0x7f2430005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:41.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:41.963+0000 7f24377fe640 1 -- 172.21.15.103:0/2298526684 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2438021070 con 0x7f2440102c20 2024-08-19T01:32:42.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.169+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2420005580 con 0x7f2440102c20 2024-08-19T01:32:42.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.169+0000 7f24377fe640 1 -- 172.21.15.103:0/2298526684 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f243809be10 con 0x7f2440102c20 2024-08-19T01:32:42.171 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:42.171 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":2,"btime":"2024-08-19T01:32:39:355657+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-19T01:32:42.171 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 2 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f242c0a2b80 msgr2=0x7f242c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f242c0a2b80 0x7f242c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24300096c0 tx=0x7f2430005a20 comp rx=0 tx=0).stop 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2440102c20 msgr2=0x7f244019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2440102c20 0x7f244019cad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f243800da10 tx=0x7f243800ded0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 shutdown_connections 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f242c0a2b80 0x7f242c0a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2440102c20 0x7f244019cad0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24401022a0 0x7f2440198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 --2- 172.21.15.103:0/2298526684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2440100b50 0x7f2440198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 >> 172.21.15.103:0/2298526684 conn(0x7f24400fc500 msgr2=0x7f244006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:42.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 shutdown_connections 2024-08-19T01:32:42.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:42.172+0000 7f2444d44640 1 -- 172.21.15.103:0/2298526684 wait complete. 2024-08-19T01:32:42.201 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:41 smithi146 podman[51249]: 2024-08-19 01:32:41.977733841 +0000 UTC m=+0.375483099 container init 28351662d1da3c6cb4b6609203e6b3f67ada978448eb81b81407be42ac70ae7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0) 2024-08-19T01:32:42.201 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:41 smithi146 podman[51249]: 2024-08-19 01:32:41.980547308 +0000 UTC m=+0.378296570 container start 28351662d1da3c6cb4b6609203e6b3f67ada978448eb81b81407be42ac70ae7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-19T01:32:42.202 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:42 smithi146 ceph-mds[51268]: -- 172.21.15.146:0/2065608299 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5619307ab860 con 0x561930788800 2024-08-19T01:32:42.202 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:42 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k[51264]: starting mds.k at 2024-08-19T01:32:42.202 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:42 smithi146 bash[51249]: 28351662d1da3c6cb4b6609203e6b3f67ada978448eb81b81407be42ac70ae7d 2024-08-19T01:32:42.202 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:32:42 smithi146 systemd[1]: Started Ceph mds.k for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: Deploying daemon mds.j on smithi103 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2298526684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6832/1682379250,v1:172.21.15.146:6833/1682379250] up:boot 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: fsmap 2 up:standby 2024-08-19T01:32:43.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:32:43.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:43 smithi190 ceph-mon[22032]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: Deploying daemon mds.j on smithi103 2024-08-19T01:32:43.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2298526684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:43.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6832/1682379250,v1:172.21.15.146:6833/1682379250] up:boot 2024-08-19T01:32:43.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: fsmap 2 up:standby 2024-08-19T01:32:43.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:32:43.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:43 smithi146 ceph-mon[22098]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:43.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:43.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:43.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:43.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:43.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: Deploying daemon mds.j on smithi103 2024-08-19T01:32:43.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2298526684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:43.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6832/1682379250,v1:172.21.15.146:6833/1682379250] up:boot 2024-08-19T01:32:43.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: fsmap 2 up:standby 2024-08-19T01:32:43.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:32:43.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:43 smithi103 ceph-mon[18436]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:44.708 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:44 smithi103 systemd[1]: Starting Ceph mds.j for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:44.895 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:44.972 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:44 smithi103 podman[57056]: 2024-08-19 01:32:44.610196756 +0000 UTC m=+0.017026170 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:44.972 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:44 smithi103 podman[57056]: 2024-08-19 01:32:44.726567606 +0000 UTC m=+0.133397007 container create 72e1d99557ae287c8e81c8af9897cb26085e245303ad1f41d9091d4a07b21fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:32:44.973 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:44 smithi103 podman[57056]: 2024-08-19 01:32:44.971632311 +0000 UTC m=+0.378461726 container init 72e1d99557ae287c8e81c8af9897cb26085e245303ad1f41d9091d4a07b21fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-19T01:32:45.179 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:45.302 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:44 smithi103 podman[57056]: 2024-08-19 01:32:44.974699267 +0000 UTC m=+0.381528667 container start 72e1d99557ae287c8e81c8af9897cb26085e245303ad1f41d9091d4a07b21fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-19T01:32:45.302 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mds[57093]: -- 172.21.15.103:0/3885996938 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561eb11b3860 con 0x561eb1190800 2024-08-19T01:32:45.302 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j[57071]: starting mds.j at 2024-08-19T01:32:45.302 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:45 smithi103 bash[57056]: 72e1d99557ae287c8e81c8af9897cb26085e245303ad1f41d9091d4a07b21fe5 2024-08-19T01:32:45.303 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:32:45 smithi103 systemd[1]: Started Ceph mds.j for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 -- 172.21.15.103:0/2024453420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff514101150 msgr2=0x7ff51410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 --2- 172.21.15.103:0/2024453420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff514101150 0x7ff51410d680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009cd0 tx=0x7ff50c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 -- 172.21.15.103:0/2024453420 shutdown_connections 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 --2- 172.21.15.103:0/2024453420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff514101150 0x7ff51410d680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 --2- 172.21.15.103:0/2024453420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5141007b0 0x7ff514100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 --2- 172.21.15.103:0/2024453420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5141067b0 0x7ff514106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 -- 172.21.15.103:0/2024453420 >> 172.21.15.103:0/2024453420 conn(0x7ff5140fc4d0 msgr2=0x7ff5140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:45.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 -- 172.21.15.103:0/2024453420 shutdown_connections 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.697+0000 7ff51a048640 1 -- 172.21.15.103:0/2024453420 wait complete. 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 Processor -- start 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 -- start start 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5141007b0 0x7ff51419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff514101150 0x7ff51419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5141067b0 0x7ff51419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff514113d80 con 0x7ff5141067b0 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff514113c00 con 0x7ff514101150 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff51a048640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff514113f00 con 0x7ff5141007b0 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.698+0000 7ff513fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5141067b0 0x7ff51419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5141067b0 0x7ff51419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39804/0 (socket says 172.21.15.103:39804) 2024-08-19T01:32:45.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 -- 172.21.15.103:0/244055647 learned_addr learned my addr 172.21.15.103:0/244055647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff512ffd640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff514101150 0x7ff51419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff5137fe640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5141007b0 0x7ff51419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 -- 172.21.15.103:0/244055647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5141007b0 msgr2=0x7ff51419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5141007b0 0x7ff51419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 -- 172.21.15.103:0/244055647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff514101150 msgr2=0x7ff51419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff514101150 0x7ff51419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 -- 172.21.15.103:0/244055647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51419f1f0 con 0x7ff5141067b0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff513fff640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5141067b0 0x7ff51419eb10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff50c005cc0 tx=0x7ff50c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff510ff9640 1 -- 172.21.15.103:0/244055647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c004240 con 0x7ff5141067b0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff510ff9640 1 -- 172.21.15.103:0/244055647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff50c0305a0 con 0x7ff5141067b0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51419f480 con 0x7ff5141067b0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff510ff9640 1 -- 172.21.15.103:0/244055647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0466c0 con 0x7ff5141067b0 2024-08-19T01:32:45.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.699+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a6dc0 con 0x7ff5141067b0 2024-08-19T01:32:45.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.700+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff514100c10 con 0x7ff5141067b0 2024-08-19T01:32:45.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.701+0000 7ff510ff9640 1 -- 172.21.15.103:0/244055647 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff50c030ac0 con 0x7ff5141067b0 2024-08-19T01:32:45.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.702+0000 7ff510ff9640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff4fc0a2b70 0x7ff4fc0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:45.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.702+0000 7ff5137fe640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff4fc0a2b70 0x7ff4fc0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:45.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.702+0000 7ff510ff9640 1 -- 172.21.15.103:0/244055647 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff50c02e030 con 0x7ff5141067b0 2024-08-19T01:32:45.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.702+0000 7ff5137fe640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff4fc0a2b70 0x7ff4fc0a5030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff504009990 tx=0x7ff504009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:45.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.706+0000 7ff510ff9640 1 -- 172.21.15.103:0/244055647 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff50c04b060 con 0x7ff5141067b0 2024-08-19T01:32:45.879 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mon[18436]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:45.879 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:45.879 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:45.879 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:45.879 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:45.879 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:45.879 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:45 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:45.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.922+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5141a6ff0 con 0x7ff5141067b0 2024-08-19T01:32:45.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.923+0000 7ff510ff9640 1 -- 172.21.15.103:0/244055647 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7ff50c0bd4d0 con 0x7ff5141067b0 2024-08-19T01:32:45.925 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:45.925 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":4,"btime":"2024-08-19T01:32:45:814549+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-19T01:32:45.925 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 4 2024-08-19T01:32:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff4fc0a2b70 msgr2=0x7ff4fc0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff4fc0a2b70 0x7ff4fc0a5030 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff504009990 tx=0x7ff504009110 comp rx=0 tx=0).stop 2024-08-19T01:32:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5141067b0 msgr2=0x7ff51419eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5141067b0 0x7ff51419eb10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff50c005cc0 tx=0x7ff50c003ba0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 shutdown_connections 2024-08-19T01:32:45.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff4fc0a2b70 0x7ff4fc0a5030 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5141067b0 0x7ff51419eb10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff514101150 0x7ff51419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 --2- 172.21.15.103:0/244055647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5141007b0 0x7ff51419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:45.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 >> 172.21.15.103:0/244055647 conn(0x7ff5140fc4d0 msgr2=0x7ff5140fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:45.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 shutdown_connections 2024-08-19T01:32:45.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:45.925+0000 7ff51a048640 1 -- 172.21.15.103:0/244055647 wait complete. 2024-08-19T01:32:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:45 smithi190 ceph-mon[22032]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:45 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:45 smithi146 ceph-mon[22098]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:46.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:46.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:46.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:46.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:46.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:45 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:46 smithi103 ceph-mon[18436]: Deploying daemon mds.d on smithi103 2024-08-19T01:32:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:46 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.103:6834/3930645278,v1:172.21.15.103:6835/3930645278] up:boot 2024-08-19T01:32:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:46 smithi103 ceph-mon[18436]: fsmap 3 up:standby 2024-08-19T01:32:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:46 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-19T01:32:47.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:46 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/244055647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:47.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:46 smithi190 ceph-mon[22032]: Deploying daemon mds.d on smithi103 2024-08-19T01:32:47.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:46 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.103:6834/3930645278,v1:172.21.15.103:6835/3930645278] up:boot 2024-08-19T01:32:47.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:46 smithi190 ceph-mon[22032]: fsmap 3 up:standby 2024-08-19T01:32:47.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:46 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-19T01:32:47.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:46 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/244055647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:47.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:46 smithi146 ceph-mon[22098]: Deploying daemon mds.d on smithi103 2024-08-19T01:32:47.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:46 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.103:6834/3930645278,v1:172.21.15.103:6835/3930645278] up:boot 2024-08-19T01:32:47.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:46 smithi146 ceph-mon[22098]: fsmap 3 up:standby 2024-08-19T01:32:47.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:46 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-19T01:32:47.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:46 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/244055647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:47.804 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:47 smithi103 systemd[1]: Starting Ceph mds.d for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:48.085 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:47 smithi103 ceph-mon[18436]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:48.086 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:47 smithi103 podman[57471]: 2024-08-19 01:32:47.813094934 +0000 UTC m=+0.017887758 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:48.086 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:47 smithi103 podman[57471]: 2024-08-19 01:32:47.943889977 +0000 UTC m=+0.148682844 container create d5bbd1a5cc5e01108766096ccc75780bdeedc4bcfb11306b733b6cc80135ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731) 2024-08-19T01:32:48.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:47 smithi190 ceph-mon[22032]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:47 smithi146 ceph-mon[22098]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:48.554 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:48 smithi103 podman[57471]: 2024-08-19 01:32:48.197108572 +0000 UTC m=+0.401901408 container init d5bbd1a5cc5e01108766096ccc75780bdeedc4bcfb11306b733b6cc80135ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-19T01:32:48.554 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:48 smithi103 podman[57471]: 2024-08-19 01:32:48.199788245 +0000 UTC m=+0.404581070 container start d5bbd1a5cc5e01108766096ccc75780bdeedc4bcfb11306b733b6cc80135ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-19T01:32:48.554 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:48 smithi103 ceph-mds[57491]: -- 172.21.15.103:0/671036770 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586591ef860 con 0x5586591cd400 2024-08-19T01:32:48.554 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:48 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d[57487]: starting mds.d at 2024-08-19T01:32:48.554 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:48 smithi103 bash[57471]: d5bbd1a5cc5e01108766096ccc75780bdeedc4bcfb11306b733b6cc80135ee68 2024-08-19T01:32:48.554 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:32:48 smithi103 systemd[1]: Started Ceph mds.d for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:49.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: Deploying daemon mds.e on smithi146 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.103:6836/3245550347,v1:172.21.15.103:6837/3245550347] up:boot 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: fsmap 4 up:standby 2024-08-19T01:32:49.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:49 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-19T01:32:49.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:49.675 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.675 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: Deploying daemon mds.e on smithi146 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.103:6836/3245550347,v1:172.21.15.103:6837/3245550347] up:boot 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: fsmap 4 up:standby 2024-08-19T01:32:49.676 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:49 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-19T01:32:49.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:49.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:49.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:49.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:49.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: Deploying daemon mds.e on smithi146 2024-08-19T01:32:49.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:49.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.103:6836/3245550347,v1:172.21.15.103:6837/3245550347] up:boot 2024-08-19T01:32:49.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: fsmap 4 up:standby 2024-08-19T01:32:49.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:49 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-19T01:32:49.916 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:50.451 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 systemd[1]: Starting Ceph mds.e for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.431+0000 7f926dfeb640 1 -- 172.21.15.103:0/3358400421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92681022a0 msgr2=0x7f92681026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.431+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3358400421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92681022a0 0x7f92681026e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f925c009900 tx=0x7f925c02a480 comp rx=0 tx=0).stop 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.431+0000 7f926dfeb640 1 -- 172.21.15.103:0/3358400421 shutdown_connections 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.431+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3358400421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9268102c20 0x7f926810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.431+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3358400421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92681022a0 0x7f92681026e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.431+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3358400421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9268100b50 0x7f9268100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.431+0000 7f926dfeb640 1 -- 172.21.15.103:0/3358400421 >> 172.21.15.103:0/3358400421 conn(0x7f92680fc500 msgr2=0x7f92680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.432+0000 7f926dfeb640 1 -- 172.21.15.103:0/3358400421 shutdown_connections 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.432+0000 7f926dfeb640 1 -- 172.21.15.103:0/3358400421 wait complete. 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.432+0000 7f926dfeb640 1 Processor -- start 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.432+0000 7f926dfeb640 1 -- start start 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.432+0000 7f926dfeb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9268100b50 0x7f9268198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.432+0000 7f926dfeb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92681022a0 0x7f9268199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f926dfeb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9268102c20 0x7f92681a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f926dfeb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9268110b20 con 0x7f9268100b50 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f926dfeb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f92681109a0 con 0x7f9268102c20 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f926dfeb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9268110ca0 con 0x7f92681022a0 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9268100b50 0x7f9268198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9268100b50 0x7f9268198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39834/0 (socket says 172.21.15.103:39834) 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 -- 172.21.15.103:0/3989985312 learned_addr learned my addr 172.21.15.103:0/3989985312 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f9267fff640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9268102c20 0x7f92681a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f9266ffd640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92681022a0 0x7f9268199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 -- 172.21.15.103:0/3989985312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92681022a0 msgr2=0x7f9268199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92681022a0 0x7f9268199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 -- 172.21.15.103:0/3989985312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9268102c20 msgr2=0x7f92681a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9268102c20 0x7f92681a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 -- 172.21.15.103:0/3989985312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92681a44f0 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f92677fe640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9268100b50 0x7f9268198f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f925800c920 tx=0x7f925800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f9264ff9640 1 -- 172.21.15.103:0/3989985312 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258007c70 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f9264ff9640 1 -- 172.21.15.103:0/3989985312 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9258007e10 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92681a46c0 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f9264ff9640 1 -- 172.21.15.103:0/3989985312 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258018820 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.433+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92681a4bd0 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.434+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9268100f30 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.436+0000 7f9264ff9640 1 -- 172.21.15.103:0/3989985312 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f92580189c0 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.436+0000 7f9264ff9640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f92500a2b80 0x7f92500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.436+0000 7f9266ffd640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f92500a2b80 0x7f92500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.436+0000 7f9264ff9640 1 -- 172.21.15.103:0/3989985312 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f92580cef10 con 0x7f9268100b50 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.437+0000 7f9266ffd640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f92500a2b80 0x7f92500a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f925c009230 tx=0x7f925c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.439+0000 7f9264ff9640 1 -- 172.21.15.103:0/3989985312 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9258014030 con 0x7f9268100b50 2024-08-19T01:32:50.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.641+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f926806a2d0 con 0x7f9268100b50 2024-08-19T01:32:50.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.641+0000 7f9264ff9640 1 -- 172.21.15.103:0/3989985312 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f926806a2d0 con 0x7f9268100b50 2024-08-19T01:32:50.644 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:50.644 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":5,"btime":"2024-08-19T01:32:48:826160+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-19T01:32:50.644 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 5 2024-08-19T01:32:50.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f92500a2b80 msgr2=0x7f92500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:50.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f92500a2b80 0x7f92500a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f925c009230 tx=0x7f925c0058d0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9268100b50 msgr2=0x7f9268198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9268100b50 0x7f9268198f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f925800c920 tx=0x7f925800cde0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 shutdown_connections 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f92500a2b80 0x7f92500a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9268102c20 0x7f92681a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92681022a0 0x7f9268199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 --2- 172.21.15.103:0/3989985312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9268100b50 0x7f9268198f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 >> 172.21.15.103:0/3989985312 conn(0x7f92680fc500 msgr2=0x7f92680fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:50.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.643+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 shutdown_connections 2024-08-19T01:32:50.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:50.644+0000 7f926dfeb640 1 -- 172.21.15.103:0/3989985312 wait complete. 2024-08-19T01:32:50.817 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:50 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3989985312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:50.817 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 podman[51560]: 2024-08-19 01:32:50.562662204 +0000 UTC m=+0.112617653 container create 20f8936e6c78076affcb7bfc986973d7611782e749ca20cac19960ebeac7cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0) 2024-08-19T01:32:50.817 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 podman[51560]: 2024-08-19 01:32:50.467783544 +0000 UTC m=+0.017738992 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:50.817 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 podman[51560]: 2024-08-19 01:32:50.790223519 +0000 UTC m=+0.340178978 container init 20f8936e6c78076affcb7bfc986973d7611782e749ca20cac19960ebeac7cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:32:50.817 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 podman[51560]: 2024-08-19 01:32:50.793031152 +0000 UTC m=+0.342986602 container start 20f8936e6c78076affcb7bfc986973d7611782e749ca20cac19960ebeac7cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True) 2024-08-19T01:32:51.015 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:50 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3989985312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:51.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:50 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3989985312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:51.201 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 ceph-mds[51579]: -- 172.21.15.146:0/1378256868 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55acec72d860 con 0x55acec70b000 2024-08-19T01:32:51.201 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e[51575]: starting mds.e at 2024-08-19T01:32:51.201 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 bash[51560]: 20f8936e6c78076affcb7bfc986973d7611782e749ca20cac19960ebeac7cffb 2024-08-19T01:32:51.201 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:32:50 smithi146 systemd[1]: Started Ceph mds.e for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:51 smithi103 ceph-mon[18436]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:51 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:52.066 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:51 smithi190 ceph-mon[22032]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:52.066 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:52.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:52.067 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:51 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:51 smithi146 ceph-mon[22098]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:51 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:52.809 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:52 smithi190 ceph-mon[22032]: Deploying daemon mds.f on smithi190 2024-08-19T01:32:52.809 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:52 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6834/3609998196,v1:172.21.15.146:6835/3609998196] up:boot 2024-08-19T01:32:52.809 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:52 smithi190 ceph-mon[22032]: fsmap 5 up:standby 2024-08-19T01:32:52.809 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:52 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:32:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:52 smithi103 ceph-mon[18436]: Deploying daemon mds.f on smithi190 2024-08-19T01:32:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:52 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6834/3609998196,v1:172.21.15.146:6835/3609998196] up:boot 2024-08-19T01:32:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:52 smithi103 ceph-mon[18436]: fsmap 5 up:standby 2024-08-19T01:32:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:52 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:32:53.131 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:52 smithi190 systemd[1]: Starting Ceph mds.f for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:53.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:52 smithi146 ceph-mon[22098]: Deploying daemon mds.f on smithi190 2024-08-19T01:32:53.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:52 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6834/3609998196,v1:172.21.15.146:6835/3609998196] up:boot 2024-08-19T01:32:53.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:52 smithi146 ceph-mon[22098]: fsmap 5 up:standby 2024-08-19T01:32:53.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:52 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:32:53.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:53.477 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 podman[51900]: 2024-08-19 01:32:53.111442067 +0000 UTC m=+0.017307839 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:53.477 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 podman[51900]: 2024-08-19 01:32:53.23084034 +0000 UTC m=+0.136706112 container create 151cd8713b53ba4323a2d8775ecf455a54a3935b092ec0e658fdb8a85d02a5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:32:53.477 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 podman[51900]: 2024-08-19 01:32:53.450223163 +0000 UTC m=+0.356088949 container init 151cd8713b53ba4323a2d8775ecf455a54a3935b092ec0e658fdb8a85d02a5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2) 2024-08-19T01:32:53.477 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 podman[51900]: 2024-08-19 01:32:53.452957521 +0000 UTC m=+0.358823294 container start 151cd8713b53ba4323a2d8775ecf455a54a3935b092ec0e658fdb8a85d02a5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:32:53.477 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mds[51920]: -- 172.21.15.190:0/1357796450 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bb940b860 con 0x563bb93e8800 2024-08-19T01:32:53.699 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:53.880 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f[51916]: starting mds.f at 2024-08-19T01:32:53.880 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 bash[51900]: 151cd8713b53ba4323a2d8775ecf455a54a3935b092ec0e658fdb8a85d02a5de 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:32:53 smithi190 systemd[1]: Started Ceph mds.f for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mon[22032]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:53.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:53 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:53 smithi103 ceph-mon[18436]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:54.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:53 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:53 smithi146 ceph-mon[22098]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:53 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 -- 172.21.15.103:0/1336093724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe750101150 msgr2=0x7fe75010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 --2- 172.21.15.103:0/1336093724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe750101150 0x7fe75010d680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe748009930 tx=0x7fe74802a420 comp rx=0 tx=0).stop 2024-08-19T01:32:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 -- 172.21.15.103:0/1336093724 shutdown_connections 2024-08-19T01:32:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 --2- 172.21.15.103:0/1336093724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe750101150 0x7fe75010d680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 --2- 172.21.15.103:0/1336093724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7501007b0 0x7fe750100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 --2- 172.21.15.103:0/1336093724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7501067b0 0x7fe750106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 -- 172.21.15.103:0/1336093724 >> 172.21.15.103:0/1336093724 conn(0x7fe7500fc4d0 msgr2=0x7fe7500fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 -- 172.21.15.103:0/1336093724 shutdown_connections 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 -- 172.21.15.103:0/1336093724 wait complete. 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 Processor -- start 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.317+0000 7fe755978640 1 -- start start 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe755978640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7501007b0 0x7fe750198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe755978640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe750101150 0x7fe750199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe755978640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7501067b0 0x7fe7501a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe755978640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe750110a60 con 0x7fe7501067b0 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe755978640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501108e0 con 0x7fe7501007b0 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe755978640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe750110be0 con 0x7fe750101150 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7501067b0 0x7fe7501a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7501067b0 0x7fe7501a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58270/0 (socket says 172.21.15.103:58270) 2024-08-19T01:32:54.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 -- 172.21.15.103:0/1012972260 learned_addr learned my addr 172.21.15.103:0/1012972260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74effd640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7501007b0 0x7fe750198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74e7fc640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe750101150 0x7fe750199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 -- 172.21.15.103:0/1012972260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe750101150 msgr2=0x7fe750199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe750101150 0x7fe750199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 -- 172.21.15.103:0/1012972260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7501007b0 msgr2=0x7fe750198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7501007b0 0x7fe750198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 -- 172.21.15.103:0/1012972260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7501a44b0 con 0x7fe7501067b0 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74effd640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7501007b0 0x7fe750198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:32:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74e7fc640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe750101150 0x7fe750199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.318+0000 7fe74f7fe640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7501067b0 0x7fe7501a3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe748005940 tx=0x7fe7480040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.319+0000 7fe754976640 1 -- 172.21.15.103:0/1012972260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74803f030 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.319+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7501a4680 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.319+0000 7fe754976640 1 -- 172.21.15.103:0/1012972260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7480044c0 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.319+0000 7fe754976640 1 -- 172.21.15.103:0/1012972260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748035070 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.319+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501a4b30 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.320+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe750100c10 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.321+0000 7fe754976640 1 -- 172.21.15.103:0/1012972260 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe748042030 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.322+0000 7fe754976640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe73c0a2b80 0x7fe73c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.322+0000 7fe74effd640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe73c0a2b80 0x7fe73c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.322+0000 7fe754976640 1 -- 172.21.15.103:0/1012972260 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fe7480eefa0 con 0x7fe7501067b0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.322+0000 7fe74effd640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe73c0a2b80 0x7fe73c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe740005fd0 tx=0x7fe740009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.326+0000 7fe754976640 1 -- 172.21.15.103:0/1012972260 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe7480bc3a0 con 0x7fe7501067b0 2024-08-19T01:32:54.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.526+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe750108d20 con 0x7fe7501067b0 2024-08-19T01:32:54.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.527+0000 7fe754976640 1 -- 172.21.15.103:0/1012972260 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5575 (secure 0 0 0) 0x7fe7480bbca0 con 0x7fe7501067b0 2024-08-19T01:32:54.529 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:54.530 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":7,"btime":"2024-08-19T01:32:53:702923+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-08-19T01:32:54.530 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 7 2024-08-19T01:32:54.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe73c0a2b80 msgr2=0x7fe73c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:54.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe73c0a2b80 0x7fe73c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe740005fd0 tx=0x7fe740009500 comp rx=0 tx=0).stop 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7501067b0 msgr2=0x7fe7501a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7501067b0 0x7fe7501a3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe748005940 tx=0x7fe7480040c0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 shutdown_connections 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe73c0a2b80 0x7fe73c0a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7501067b0 0x7fe7501a3f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe750101150 0x7fe750199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 --2- 172.21.15.103:0/1012972260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7501007b0 0x7fe750198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 >> 172.21.15.103:0/1012972260 conn(0x7fe7500fc4d0 msgr2=0x7fe7500fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 shutdown_connections 2024-08-19T01:32:54.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:54.529+0000 7fe755978640 1 -- 172.21.15.103:0/1012972260 wait complete. 2024-08-19T01:32:54.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:54 smithi103 ceph-mon[18436]: Deploying daemon mds.c on smithi190 2024-08-19T01:32:54.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:54 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] up:boot 2024-08-19T01:32:54.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:54 smithi103 ceph-mon[18436]: fsmap 6 up:standby 2024-08-19T01:32:54.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:54 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:32:54.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:54 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1012972260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:55.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:54 smithi190 ceph-mon[22032]: Deploying daemon mds.c on smithi190 2024-08-19T01:32:55.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:54 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] up:boot 2024-08-19T01:32:55.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:54 smithi190 ceph-mon[22032]: fsmap 6 up:standby 2024-08-19T01:32:55.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:54 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:32:55.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:54 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1012972260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:55.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:54 smithi146 ceph-mon[22098]: Deploying daemon mds.c on smithi190 2024-08-19T01:32:55.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:54 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] up:boot 2024-08-19T01:32:55.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:54 smithi146 ceph-mon[22098]: fsmap 6 up:standby 2024-08-19T01:32:55.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:54 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:32:55.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:54 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1012972260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:55.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:55 smithi190 ceph-mon[22032]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:56.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:55 smithi103 ceph-mon[18436]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:55 smithi146 ceph-mon[22098]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:56.630 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:32:56 smithi190 systemd[1]: Started Ceph mds.c for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:32:57.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:32:57.477 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: Deploying daemon mds.h on smithi146 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] up:boot 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: fsmap 7 up:standby 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:32:57.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:57 smithi103 ceph-mon[18436]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:57.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: Deploying daemon mds.h on smithi146 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] up:boot 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: fsmap 7 up:standby 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:32:57.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:57 smithi190 ceph-mon[22032]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: Deploying daemon mds.h on smithi146 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] up:boot 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: fsmap 7 up:standby 2024-08-19T01:32:57.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:32:57.702 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:57 smithi146 ceph-mon[22098]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:32:57.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 -- 172.21.15.103:0/400920845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a41022a0 msgr2=0x7f98a41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 --2- 172.21.15.103:0/400920845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a41022a0 0x7f98a41026e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9898009900 tx=0x7f989802a480 comp rx=0 tx=0).stop 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 -- 172.21.15.103:0/400920845 shutdown_connections 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 --2- 172.21.15.103:0/400920845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98a4102c20 0x7f98a410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 --2- 172.21.15.103:0/400920845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a41022a0 0x7f98a41026e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 --2- 172.21.15.103:0/400920845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a4100b50 0x7f98a4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 -- 172.21.15.103:0/400920845 >> 172.21.15.103:0/400920845 conn(0x7f98a40fc500 msgr2=0x7f98a40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 -- 172.21.15.103:0/400920845 shutdown_connections 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 -- 172.21.15.103:0/400920845 wait complete. 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 Processor -- start 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.941+0000 7f98ab989640 1 -- start start 2024-08-19T01:32:57.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a4100b50 0x7f98a4199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98a41022a0 0x7f98a4199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a4102c20 0x7f98a41a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4110b80 con 0x7f98a4100b50 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4110a00 con 0x7f98a4102c20 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4110d00 con 0x7f98a41022a0 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a4100b50 0x7f98a4199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a4100b50 0x7f98a4199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58276/0 (socket says 172.21.15.103:58276) 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 -- 172.21.15.103:0/3661194957 learned_addr learned my addr 172.21.15.103:0/3661194957 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a8f24640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98a41022a0 0x7f98a4199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9f26640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a4102c20 0x7f98a41a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 -- 172.21.15.103:0/3661194957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98a41022a0 msgr2=0x7f98a4199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:57.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98a41022a0 0x7f98a4199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 -- 172.21.15.103:0/3661194957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a4102c20 msgr2=0x7f98a41a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a4102c20 0x7f98a41a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 -- 172.21.15.103:0/3661194957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a41a45d0 con 0x7f98a4100b50 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a8f24640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98a41022a0 0x7f98a4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9f26640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a4102c20 0x7f98a41a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a9725640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a4100b50 0x7f98a4199010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f989c00e9f0 tx=0x7f989c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a27fc640 1 -- 172.21.15.103:0/3661194957 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c00cd80 con 0x7f98a4100b50 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a27fc640 1 -- 172.21.15.103:0/3661194957 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f989c0108e0 con 0x7f98a4100b50 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a41a47a0 con 0x7f98a4100b50 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98a27fc640 1 -- 172.21.15.103:0/3661194957 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c018920 con 0x7f98a4100b50 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.942+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41a4cb0 con 0x7f98a4100b50 2024-08-19T01:32:57.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.943+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4100f30 con 0x7f98a4100b50 2024-08-19T01:32:57.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.944+0000 7f98a27fc640 1 -- 172.21.15.103:0/3661194957 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f989c010a90 con 0x7f98a4100b50 2024-08-19T01:32:57.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.945+0000 7f98a27fc640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f988c0a2b80 0x7f988c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:32:57.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.945+0000 7f98a8f24640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f988c0a2b80 0x7f988c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:32:57.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.945+0000 7f98a27fc640 1 -- 172.21.15.103:0/3661194957 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f989c0cf4b0 con 0x7f98a4100b50 2024-08-19T01:32:57.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.945+0000 7f98a8f24640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f988c0a2b80 0x7f988c0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98a406a7f0 tx=0x7f9898005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:32:57.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:57.948+0000 7f98a27fc640 1 -- 172.21.15.103:0/3661194957 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f989c09c8b0 con 0x7f98a4100b50 2024-08-19T01:32:58.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.148+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98a4108dc0 con 0x7f98a4100b50 2024-08-19T01:32:58.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.149+0000 7f98a27fc640 1 -- 172.21.15.103:0/3661194957 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f989c09c1b0 con 0x7f98a4100b50 2024-08-19T01:32:58.151 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:32:58.151 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":8,"btime":"2024-08-19T01:32:56:714137+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-19T01:32:58.151 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 8 2024-08-19T01:32:58.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f988c0a2b80 msgr2=0x7f988c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:58.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f988c0a2b80 0x7f988c0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98a406a7f0 tx=0x7f9898005a20 comp rx=0 tx=0).stop 2024-08-19T01:32:58.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a4100b50 msgr2=0x7f98a4199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:32:58.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a4100b50 0x7f98a4199010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f989c00e9f0 tx=0x7f989c00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 shutdown_connections 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f988c0a2b80 0x7f988c0a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98a4102c20 0x7f98a41a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98a41022a0 0x7f98a4199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 --2- 172.21.15.103:0/3661194957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98a4100b50 0x7f98a4199010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 >> 172.21.15.103:0/3661194957 conn(0x7f98a40fc500 msgr2=0x7f98a40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 shutdown_connections 2024-08-19T01:32:58.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:32:58.151+0000 7f98ab989640 1 -- 172.21.15.103:0/3661194957 wait complete. 2024-08-19T01:32:58.626 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 systemd[1]: Starting Ceph mds.h for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:32:58.880 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:58 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3661194957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:58.880 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 podman[51865]: 2024-08-19 01:32:58.625477336 +0000 UTC m=+0.113849329 container create 07cc3b08b067427acde2bdfeca23704746cb12ca58dffc21b0d7102f59f68a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-19T01:32:58.880 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 podman[51865]: 2024-08-19 01:32:58.528818142 +0000 UTC m=+0.017190140 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:32:58.880 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 podman[51865]: 2024-08-19 01:32:58.853206616 +0000 UTC m=+0.341578605 container init 07cc3b08b067427acde2bdfeca23704746cb12ca58dffc21b0d7102f59f68a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-19T01:32:58.880 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 podman[51865]: 2024-08-19 01:32:58.856050116 +0000 UTC m=+0.344422105 container start 07cc3b08b067427acde2bdfeca23704746cb12ca58dffc21b0d7102f59f68a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0) 2024-08-19T01:32:59.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:58 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3661194957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:59.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:58 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3661194957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:32:59.201 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 ceph-mds[51884]: -- 172.21.15.146:0/2202248734 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9e134b860 con 0x55b9e1329000 2024-08-19T01:32:59.201 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h[51880]: starting mds.h at 2024-08-19T01:32:59.201 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 bash[51865]: 07cc3b08b067427acde2bdfeca23704746cb12ca58dffc21b0d7102f59f68a1c 2024-08-19T01:32:59.201 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:32:58 smithi146 systemd[1]: Started Ceph mds.h for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:33:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:59 smithi103 ceph-mon[18436]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:00.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:32:59 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:00.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:59 smithi190 ceph-mon[22032]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:00.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:00.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:00.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:32:59 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:59 smithi146 ceph-mon[22098]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:32:59 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:00.857 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:00 smithi190 ceph-mon[22032]: Deploying daemon mds.i on smithi190 2024-08-19T01:33:00.858 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:00 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] up:boot 2024-08-19T01:33:00.858 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:00 smithi190 ceph-mon[22032]: fsmap 8 up:standby 2024-08-19T01:33:00.858 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:33:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:00 smithi103 ceph-mon[18436]: Deploying daemon mds.i on smithi190 2024-08-19T01:33:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:00 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] up:boot 2024-08-19T01:33:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:00 smithi103 ceph-mon[18436]: fsmap 8 up:standby 2024-08-19T01:33:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:33:01.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:01.130 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 systemd[1]: Starting Ceph mds.i for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:33:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:00 smithi146 ceph-mon[22098]: Deploying daemon mds.i on smithi190 2024-08-19T01:33:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:00 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] up:boot 2024-08-19T01:33:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:00 smithi146 ceph-mon[22098]: fsmap 8 up:standby 2024-08-19T01:33:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:33:01.322 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:01.529 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 podman[52503]: 2024-08-19 01:33:01.165805662 +0000 UTC m=+0.017058519 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:33:01.529 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 podman[52503]: 2024-08-19 01:33:01.267100445 +0000 UTC m=+0.118353344 container create 1848559706c10499b7b6ce48c6602088d572baf1d5d87b9ed6d81ab3caed8fe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-19T01:33:01.529 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 podman[52503]: 2024-08-19 01:33:01.528298535 +0000 UTC m=+0.379551402 container init 1848559706c10499b7b6ce48c6602088d572baf1d5d87b9ed6d81ab3caed8fe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 -- 172.21.15.103:0/514123208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c105290 msgr2=0x7f3e9c10ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 --2- 172.21.15.103:0/514123208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c105290 0x7f3e9c10ce10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009ca0 tx=0x7f3e9402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 -- 172.21.15.103:0/514123208 shutdown_connections 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 --2- 172.21.15.103:0/514123208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c105290 0x7f3e9c10ce10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 --2- 172.21.15.103:0/514123208 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e9c100b10 0x7f3e9c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 --2- 172.21.15.103:0/514123208 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e9c100160 0x7f3e9c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 -- 172.21.15.103:0/514123208 >> 172.21.15.103:0/514123208 conn(0x7f3e9c076f60 msgr2=0x7f3e9c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:01.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 -- 172.21.15.103:0/514123208 shutdown_connections 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.816+0000 7f3ea1858640 1 -- 172.21.15.103:0/514123208 wait complete. 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 Processor -- start 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 -- start start 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e9c100160 0x7f3e9c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c100b10 0x7f3e9c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e9c105290 0x7f3e9c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c111b20 con 0x7f3e9c100b10 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c1119a0 con 0x7f3e9c105290 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.817+0000 7f3ea1858640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c111ca0 con 0x7f3e9c100160 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c100b10 0x7f3e9c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c100b10 0x7f3e9c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58300/0 (socket says 172.21.15.103:58300) 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 -- 172.21.15.103:0/3019179258 learned_addr learned my addr 172.21.15.103:0/3019179258 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:01.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9b7fe640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e9c105290 0x7f3e9c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9affd640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e9c100160 0x7f3e9c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 -- 172.21.15.103:0/3019179258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e9c100160 msgr2=0x7f3e9c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e9c100160 0x7f3e9c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 -- 172.21.15.103:0/3019179258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e9c105290 msgr2=0x7f3e9c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e9c105290 0x7f3e9c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 -- 172.21.15.103:0/3019179258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c19cfc0 con 0x7f3e9c100b10 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9b7fe640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e9c105290 0x7f3e9c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3e9a7fc640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c100b10 0x7f3e9c198550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000d950 tx=0x7f3e9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3ea0856640 1 -- 172.21.15.103:0/3019179258 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90017030 con 0x7f3e9c100b10 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9c19d2b0 con 0x7f3e9c100b10 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9c1a4b90 con 0x7f3e9c100b10 2024-08-19T01:33:01.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.818+0000 7f3ea0856640 1 -- 172.21.15.103:0/3019179258 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e90004020 con 0x7f3e9c100b10 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.819+0000 7f3ea0856640 1 -- 172.21.15.103:0/3019179258 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e900047a0 con 0x7f3e9c100b10 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.820+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e9c100540 con 0x7f3e9c100b10 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.821+0000 7f3ea0856640 1 -- 172.21.15.103:0/3019179258 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3e900041d0 con 0x7f3e9c100b10 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.822+0000 7f3ea0856640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3e840a2b80 0x7f3e840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.822+0000 7f3ea0856640 1 -- 172.21.15.103:0/3019179258 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f3e900ced10 con 0x7f3e9c100b10 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.822+0000 7f3e9affd640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3e840a2b80 0x7f3e840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.822+0000 7f3e9affd640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3e840a2b80 0x7f3e840a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009b60 tx=0x7f3e8c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:01.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:01.825+0000 7f3ea0856640 1 -- 172.21.15.103:0/3019179258 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3e9009c110 con 0x7f3e9c100b10 2024-08-19T01:33:01.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mon[22032]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:01.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:01.881 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 podman[52503]: 2024-08-19 01:33:01.531401312 +0000 UTC m=+0.382654176 container start 1848559706c10499b7b6ce48c6602088d572baf1d5d87b9ed6d81ab3caed8fe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:33:01.882 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-mds[52523]: -- 172.21.15.190:0/544660895 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575d6685860 con 0x5575d6663400 2024-08-19T01:33:01.882 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i[52519]: starting mds.i at 2024-08-19T01:33:01.882 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 bash[52503]: 1848559706c10499b7b6ce48c6602088d572baf1d5d87b9ed6d81ab3caed8fe9 2024-08-19T01:33:01.882 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:33:01 smithi190 systemd[1]: Started Ceph mds.i for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:33:02.031 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:01 smithi103 ceph-mon[18436]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:02.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:02.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:02.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:02.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:02.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:02.032 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.058+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e9c107f40 con 0x7f3e9c100b10 2024-08-19T01:33:02.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.058+0000 7f3ea0856640 1 -- 172.21.15.103:0/3019179258 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7f3e9009ba10 con 0x7f3e9c100b10 2024-08-19T01:33:02.061 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:02.061 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":10,"btime":"2024-08-19T01:33:01:724981+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-19T01:33:02.061 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 10 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3e840a2b80 msgr2=0x7f3e840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3e840a2b80 0x7f3e840a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009b60 tx=0x7f3e8c005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c100b10 msgr2=0x7f3e9c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c100b10 0x7f3e9c198550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000d950 tx=0x7f3e9000de10 comp rx=0 tx=0).stop 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 shutdown_connections 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3e840a2b80 0x7f3e840a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e9c105290 0x7f3e9c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e9c100b10 0x7f3e9c198550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 --2- 172.21.15.103:0/3019179258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e9c100160 0x7f3e9c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 >> 172.21.15.103:0/3019179258 conn(0x7f3e9c076f60 msgr2=0x7f3e9c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 shutdown_connections 2024-08-19T01:33:02.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:02.061+0000 7f3ea1858640 1 -- 172.21.15.103:0/3019179258 wait complete. 2024-08-19T01:33:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:01 smithi146 ceph-mon[22098]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:02.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:02 smithi103 ceph-mon[18436]: Deploying daemon mds.g on smithi103 2024-08-19T01:33:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:02 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] up:boot 2024-08-19T01:33:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:02 smithi103 ceph-mon[18436]: fsmap 9 up:standby 2024-08-19T01:33:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:02 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:33:03.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:02 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3019179258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:02 smithi190 ceph-mon[22032]: Deploying daemon mds.g on smithi103 2024-08-19T01:33:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:02 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] up:boot 2024-08-19T01:33:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:02 smithi190 ceph-mon[22032]: fsmap 9 up:standby 2024-08-19T01:33:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:02 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:33:03.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:02 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3019179258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:02 smithi146 ceph-mon[22098]: Deploying daemon mds.g on smithi103 2024-08-19T01:33:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:02 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] up:boot 2024-08-19T01:33:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:02 smithi146 ceph-mon[22098]: fsmap 9 up:standby 2024-08-19T01:33:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:02 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:33:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:02 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3019179258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:03.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:03 smithi103 ceph-mon[18436]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:04.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:03 smithi190 ceph-mon[22032]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:03 smithi146 ceph-mon[22098]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:04.304 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 systemd[1]: Starting Ceph mds.g for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:33:04.634 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 podman[58237]: 2024-08-19 01:33:04.49999487 +0000 UTC m=+0.017179005 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:33:04.905 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 podman[58237]: 2024-08-19 01:33:04.633977948 +0000 UTC m=+0.151162081 container create 6e180b34a10d55bb651d3908704337e6898bc1a869a9f2399ef5e665715adc9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-19T01:33:05.304 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 podman[58237]: 2024-08-19 01:33:04.904241139 +0000 UTC m=+0.421425291 container init 6e180b34a10d55bb651d3908704337e6898bc1a869a9f2399ef5e665715adc9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, ceph=True, org.label-schema.schema-version=1.0) 2024-08-19T01:33:05.304 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 podman[58237]: 2024-08-19 01:33:04.907398848 +0000 UTC m=+0.424582993 container start 6e180b34a10d55bb651d3908704337e6898bc1a869a9f2399ef5e665715adc9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:33:05.304 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 ceph-mds[58256]: -- 172.21.15.103:0/2099828453 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cca53ed860 con 0x55cca53cb400 2024-08-19T01:33:05.304 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g[58252]: starting mds.g at 2024-08-19T01:33:05.304 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 bash[58237]: 6e180b34a10d55bb651d3908704337e6898bc1a869a9f2399ef5e665715adc9a 2024-08-19T01:33:05.304 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:33:04 smithi103 systemd[1]: Started Ceph mds.g for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:33:06.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:05 smithi190 ceph-mon[22032]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:06.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:06.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:06.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:05 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:05 smithi146 ceph-mon[22098]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:05 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:06.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:05 smithi103 ceph-mon[18436]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:06.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:06.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:06.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:06.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:05 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:06.395 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:06.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:06 smithi103 ceph-mon[18436]: Deploying daemon mds.a on smithi103 2024-08-19T01:33:06.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:06 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.103:6838/2158943857,v1:172.21.15.103:6839/2158943857] up:boot 2024-08-19T01:33:06.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:06 smithi103 ceph-mon[18436]: fsmap 10 up:standby 2024-08-19T01:33:06.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:06 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:33:07.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:06 smithi190 ceph-mon[22032]: Deploying daemon mds.a on smithi103 2024-08-19T01:33:07.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:06 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.103:6838/2158943857,v1:172.21.15.103:6839/2158943857] up:boot 2024-08-19T01:33:07.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:06 smithi190 ceph-mon[22032]: fsmap 10 up:standby 2024-08-19T01:33:07.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:06 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:33:07.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:06 smithi146 ceph-mon[22098]: Deploying daemon mds.a on smithi103 2024-08-19T01:33:07.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:06 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.103:6838/2158943857,v1:172.21.15.103:6839/2158943857] up:boot 2024-08-19T01:33:07.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:06 smithi146 ceph-mon[22098]: fsmap 10 up:standby 2024-08-19T01:33:07.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:06 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:33:07.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 -- 172.21.15.103:0/2924391912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01007b0 msgr2=0x7f79b0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/2924391912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01007b0 0x7f79b0100c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009930 tx=0x7f79a402a420 comp rx=0 tx=0).stop 2024-08-19T01:33:07.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 -- 172.21.15.103:0/2924391912 shutdown_connections 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/2924391912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79b0101150 0x7f79b010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/2924391912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01007b0 0x7f79b0100c10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/2924391912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79b01067b0 0x7f79b0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 -- 172.21.15.103:0/2924391912 >> 172.21.15.103:0/2924391912 conn(0x7f79b00fc4d0 msgr2=0x7f79b00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 -- 172.21.15.103:0/2924391912 shutdown_connections 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 -- 172.21.15.103:0/2924391912 wait complete. 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 Processor -- start 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.393+0000 7f79b5c1b640 1 -- start start 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79b01007b0 0x7f79b0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79b0101150 0x7f79b0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01067b0 0x7f79b019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0110a20 con 0x7f79b01067b0 2024-08-19T01:33:07.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f79b01108a0 con 0x7f79b01007b0 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0110ba0 con 0x7f79b0101150 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01067b0 0x7f79b019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01067b0 0x7f79b019c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40994/0 (socket says 172.21.15.103:40994) 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 -- 172.21.15.103:0/3488572785 learned_addr learned my addr 172.21.15.103:0/3488572785 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79af7fe640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79b01007b0 0x7f79b0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79aeffd640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79b0101150 0x7f79b0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 -- 172.21.15.103:0/3488572785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79b0101150 msgr2=0x7f79b0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79b0101150 0x7f79b0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 -- 172.21.15.103:0/3488572785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79b01007b0 msgr2=0x7f79b0198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79b01007b0 0x7f79b0198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 -- 172.21.15.103:0/3488572785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b019cfd0 con 0x7f79b01067b0 2024-08-19T01:33:07.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79affff640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01067b0 0x7f79b019c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f79a800d750 tx=0x7f79a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79acff9640 1 -- 172.21.15.103:0/3488572785 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8017070 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79acff9640 1 -- 172.21.15.103:0/3488572785 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79a80054d0 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b019d2c0 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79acff9640 1 -- 172.21.15.103:0/3488572785 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8004c60 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.394+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b01a4ba0 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.395+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b0100c10 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.396+0000 7f79acff9640 1 -- 172.21.15.103:0/3488572785 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f79a8004e00 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.397+0000 7f79acff9640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f79980a2b80 0x7f79980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.397+0000 7f79af7fe640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f79980a2b80 0x7f79980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.397+0000 7f79acff9640 1 -- 172.21.15.103:0/3488572785 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f79a80ce5d0 con 0x7f79b01067b0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.398+0000 7f79af7fe640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f79980a2b80 0x7f79980a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79a0007e40 tx=0x7f79a00075b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:07.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.401+0000 7f79acff9640 1 -- 172.21.15.103:0/3488572785 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f79a809b9d0 con 0x7f79b01067b0 2024-08-19T01:33:07.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.607+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79b0109d50 con 0x7f79b01067b0 2024-08-19T01:33:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.608+0000 7f79acff9640 1 -- 172.21.15.103:0/3488572785 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7f79a809b2d0 con 0x7f79b01067b0 2024-08-19T01:33:07.611 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:07.611 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":11,"btime":"2024-08-19T01:33:05:819270+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-19T01:33:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 11 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.610+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f79980a2b80 msgr2=0x7f79980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.610+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f79980a2b80 0x7f79980a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f79a0007e40 tx=0x7f79a00075b0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.610+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01067b0 msgr2=0x7f79b019c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.610+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01067b0 0x7f79b019c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f79a800d750 tx=0x7f79a800dc10 comp rx=0 tx=0).stop 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.610+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 shutdown_connections 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.611+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f79980a2b80 0x7f79980a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.611+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79b01067b0 0x7f79b019c8f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.611+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79b0101150 0x7f79b0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.611+0000 7f79b5c1b640 1 --2- 172.21.15.103:0/3488572785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79b01007b0 0x7f79b0198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.611+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 >> 172.21.15.103:0/3488572785 conn(0x7f79b00fc4d0 msgr2=0x7f79b006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.611+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 shutdown_connections 2024-08-19T01:33:07.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:07.611+0000 7f79b5c1b640 1 -- 172.21.15.103:0/3488572785 wait complete. 2024-08-19T01:33:07.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:07 smithi103 ceph-mon[18436]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:07.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:07 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3488572785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:08.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:07 smithi190 ceph-mon[22032]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:08.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:07 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3488572785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:07 smithi146 ceph-mon[22098]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:07 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3488572785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:09.054 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:33:08 smithi103 systemd[1]: Started Ceph mds.a for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:33:10.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.103:6840/1914348964,v1:172.21.15.103:6841/1914348964] up:boot 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: fsmap 11 up:standby 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:10.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:09 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.103:6840/1914348964,v1:172.21.15.103:6841/1914348964] up:boot 2024-08-19T01:33:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: fsmap 11 up:standby 2024-08-19T01:33:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:33:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:10.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:10.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:09 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:10.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.103:6840/1914348964,v1:172.21.15.103:6841/1914348964] up:boot 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: fsmap 11 up:standby 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-19T01:33:10.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:09 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:10.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:10.656 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:10.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:10 smithi146 ceph-mon[22098]: Deploying daemon mds.b on smithi146 2024-08-19T01:33:11.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:10 smithi103 ceph-mon[18436]: Deploying daemon mds.b on smithi146 2024-08-19T01:33:11.117 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:10 smithi190 ceph-mon[22032]: Deploying daemon mds.b on smithi146 2024-08-19T01:33:11.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 -- 172.21.15.103:0/962094785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264100860 msgr2=0x7ff264100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/962094785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264100860 0x7ff264100c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff260009900 tx=0x7ff26002a470 comp rx=0 tx=0).stop 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 -- 172.21.15.103:0/962094785 shutdown_connections 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/962094785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff264105c70 0x7ff264106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/962094785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff264101180 0x7ff264105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/962094785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264100860 0x7ff264100c40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 -- 172.21.15.103:0/962094785 >> 172.21.15.103:0/962094785 conn(0x7ff2640fc4d0 msgr2=0x7ff2640fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 -- 172.21.15.103:0/962094785 shutdown_connections 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.140+0000 7ff26bfa0640 1 -- 172.21.15.103:0/962094785 wait complete. 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 Processor -- start 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 -- start start 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff264100860 0x7ff26419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff264101180 0x7ff26419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264105c70 0x7ff2641a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff264118130 con 0x7ff264105c70 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff264117fb0 con 0x7ff264101180 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26bfa0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641182b0 con 0x7ff264100860 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.141+0000 7ff26a53d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264105c70 0x7ff2641a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264105c70 0x7ff2641a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41026/0 (socket says 172.21.15.103:41026) 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 -- 172.21.15.103:0/1005997931 learned_addr learned my addr 172.21.15.103:0/1005997931 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26953b640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff264101180 0x7ff26419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff269d3c640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff264100860 0x7ff26419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 -- 172.21.15.103:0/1005997931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff264100860 msgr2=0x7ff26419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff264100860 0x7ff26419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 -- 172.21.15.103:0/1005997931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff264101180 msgr2=0x7ff26419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff264101180 0x7ff26419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 -- 172.21.15.103:0/1005997931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2641a3680 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26a53d640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264105c70 0x7ff2641a2fa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00eaa0 tx=0x7ff25c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff25affd640 1 -- 172.21.15.103:0/1005997931 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00cdb0 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff25affd640 1 -- 172.21.15.103:0/1005997931 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff25c005020 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff25affd640 1 -- 172.21.15.103:0/1005997931 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0116d0 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2641a3970 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.142+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641ab250 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.143+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff264105650 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.144+0000 7ff25affd640 1 -- 172.21.15.103:0/1005997931 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff25c0051d0 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.145+0000 7ff25affd640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2500a2b80 0x7ff2500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.146+0000 7ff269d3c640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2500a2b80 0x7ff2500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.146+0000 7ff25affd640 1 -- 172.21.15.103:0/1005997931 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff25c0cf0e0 con 0x7ff264105c70 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.146+0000 7ff269d3c640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2500a2b80 0x7ff2500a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff260031440 tx=0x7ff260005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:11.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.149+0000 7ff25affd640 1 -- 172.21.15.103:0/1005997931 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff25c016030 con 0x7ff264105c70 2024-08-19T01:33:11.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.351+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff26419fa30 con 0x7ff264105c70 2024-08-19T01:33:11.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.351+0000 7ff25affd640 1 -- 172.21.15.103:0/1005997931 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9710 (secure 0 0 0) 0x7ff26419fa30 con 0x7ff264105c70 2024-08-19T01:33:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:11.353 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":12,"btime":"2024-08-19T01:33:08:831621+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-19T01:33:11.353 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 12 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2500a2b80 msgr2=0x7ff2500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2500a2b80 0x7ff2500a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff260031440 tx=0x7ff260005b50 comp rx=0 tx=0).stop 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264105c70 msgr2=0x7ff2641a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264105c70 0x7ff2641a2fa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00eaa0 tx=0x7ff25c00ef60 comp rx=0 tx=0).stop 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 shutdown_connections 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff2500a2b80 0x7ff2500a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff264105c70 0x7ff2641a2fa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff264101180 0x7ff26419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 --2- 172.21.15.103:0/1005997931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff264100860 0x7ff26419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:11.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 >> 172.21.15.103:0/1005997931 conn(0x7ff2640fc4d0 msgr2=0x7ff26410f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:11.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 shutdown_connections 2024-08-19T01:33:11.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:11.354+0000 7ff26bfa0640 1 -- 172.21.15.103:0/1005997931 wait complete. 2024-08-19T01:33:11.701 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:33:11 smithi146 systemd[1]: Started Ceph mds.b for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1005997931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:11 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:12.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1005997931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:11 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:12.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1005997931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:12.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:12.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:11 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:13.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:12 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] up:boot 2024-08-19T01:33:13.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:12 smithi190 ceph-mon[22032]: fsmap 12 up:standby 2024-08-19T01:33:13.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:12 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:33:13.158 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:12 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] up:boot 2024-08-19T01:33:13.158 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:12 smithi146 ceph-mon[22098]: fsmap 12 up:standby 2024-08-19T01:33:13.158 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:12 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:33:13.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:12 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] up:boot 2024-08-19T01:33:13.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:12 smithi103 ceph-mon[18436]: fsmap 12 up:standby 2024-08-19T01:33:13.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:12 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:33:14.120 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:14.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:14.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:14.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:14.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:33:15.115 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:15.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 -- 172.21.15.103:0/612590378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808100b50 msgr2=0x7f8808100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 --2- 172.21.15.103:0/612590378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808100b50 0x7f8808100f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8804009900 tx=0x7f880402a480 comp rx=0 tx=0).stop 2024-08-19T01:33:15.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 -- 172.21.15.103:0/612590378 shutdown_connections 2024-08-19T01:33:15.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 --2- 172.21.15.103:0/612590378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808102c20 0x7f880810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 --2- 172.21.15.103:0/612590378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88081022a0 0x7f88081026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 --2- 172.21.15.103:0/612590378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808100b50 0x7f8808100f30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 -- 172.21.15.103:0/612590378 >> 172.21.15.103:0/612590378 conn(0x7f88080fc500 msgr2=0x7f88080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 -- 172.21.15.103:0/612590378 shutdown_connections 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 -- 172.21.15.103:0/612590378 wait complete. 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 Processor -- start 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.619+0000 7f880f3f6640 1 -- start start 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808100b50 0x7f8808198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88081022a0 0x7f88081994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808102c20 0x7f88081a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110b60 con 0x7f8808102c20 2024-08-19T01:33:15.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f88081109e0 con 0x7f88081022a0 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110ce0 con 0x7f8808100b50 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808102c20 0x7f88081a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808102c20 0x7f88081a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48666/0 (socket says 172.21.15.103:48666) 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 -- 172.21.15.103:0/2976871852 learned_addr learned my addr 172.21.15.103:0/2976871852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880c991640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88081022a0 0x7f88081994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d192640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808100b50 0x7f8808198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 -- 172.21.15.103:0/2976871852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808100b50 msgr2=0x7f8808198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808100b50 0x7f8808198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 -- 172.21.15.103:0/2976871852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88081022a0 msgr2=0x7f88081994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88081022a0 0x7f88081994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 -- 172.21.15.103:0/2976871852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88081a4550 con 0x7f8808102c20 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d192640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808100b50 0x7f8808198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:15.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880d993640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808102c20 0x7f88081a4010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f880000eaa0 tx=0x7f880000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f87fe7fc640 1 -- 172.21.15.103:0/2976871852 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880000cdb0 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f87fe7fc640 1 -- 172.21.15.103:0/2976871852 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8800005020 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f87fe7fc640 1 -- 172.21.15.103:0/2976871852 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000116d0 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88081a4720 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.620+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081a4c30 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.621+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8808100f30 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.623+0000 7f87fe7fc640 1 -- 172.21.15.103:0/2976871852 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f88000051d0 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.623+0000 7f87fe7fc640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87f40a2b80 0x7f87f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.623+0000 7f87fe7fc640 1 -- 172.21.15.103:0/2976871852 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f88000cf0a0 con 0x7f8808102c20 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.623+0000 7f880d192640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87f40a2b80 0x7f87f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.624+0000 7f880d192640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87f40a2b80 0x7f87f40a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8804005ab0 tx=0x7f8804005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:15.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.627+0000 7f87fe7fc640 1 -- 172.21.15.103:0/2976871852 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8800016030 con 0x7f8808102c20 2024-08-19T01:33:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.842+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8808108dc0 con 0x7f8808102c20 2024-08-19T01:33:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.843+0000 7f87fe7fc640 1 -- 172.21.15.103:0/2976871852 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10539 (secure 0 0 0) 0x7f880009c480 con 0x7f8808102c20 2024-08-19T01:33:15.846 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:15.846 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":13,"btime":"2024-08-19T01:33:11:846707+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14580,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-19T01:33:15.847 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 13 2024-08-19T01:33:15.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87f40a2b80 msgr2=0x7f87f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87f40a2b80 0x7f87f40a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8804005ab0 tx=0x7f8804005a20 comp rx=0 tx=0).stop 2024-08-19T01:33:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808102c20 msgr2=0x7f88081a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808102c20 0x7f88081a4010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f880000eaa0 tx=0x7f880000ef60 comp rx=0 tx=0).stop 2024-08-19T01:33:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 shutdown_connections 2024-08-19T01:33:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87f40a2b80 0x7f87f40a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8808102c20 0x7f88081a4010 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88081022a0 0x7f88081994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 --2- 172.21.15.103:0/2976871852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8808100b50 0x7f8808198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 >> 172.21.15.103:0/2976871852 conn(0x7f88080fc500 msgr2=0x7f88080fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 shutdown_connections 2024-08-19T01:33:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:15.846+0000 7f880f3f6640 1 -- 172.21.15.103:0/2976871852 wait complete. 2024-08-19T01:33:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:15 smithi190 ceph-mon[22032]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:15 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:15 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2976871852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:15 smithi146 ceph-mon[22098]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:15 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:15 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2976871852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:16.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:15 smithi103 ceph-mon[18436]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:16.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:15 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:16.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:15 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2976871852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:16.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-19T01:33:17.017 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:17 smithi190 ceph-mon[22032]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:17.435 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.435 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:17.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:17.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:17 smithi103 ceph-mon[18436]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:33:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:33:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:33:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:17 smithi146 ceph-mon[22098]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:17.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 -- 172.21.15.103:0/3884120333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381018b0 msgr2=0x7f1138101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 --2- 172.21.15.103:0/3884120333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381018b0 0x7f1138101d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f112c009930 tx=0x7f112c02a420 comp rx=0 tx=0).stop 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 -- 172.21.15.103:0/3884120333 shutdown_connections 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 --2- 172.21.15.103:0/3884120333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1138102250 0x7f113810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 --2- 172.21.15.103:0/3884120333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381018b0 0x7f1138101d10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 --2- 172.21.15.103:0/3884120333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11381078b0 0x7f1138107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 -- 172.21.15.103:0/3884120333 >> 172.21.15.103:0/3884120333 conn(0x7f11380fd5f0 msgr2=0x7f11380ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 -- 172.21.15.103:0/3884120333 shutdown_connections 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 -- 172.21.15.103:0/3884120333 wait complete. 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 Processor -- start 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.515+0000 7f113f476640 1 -- start start 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113f476640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11381018b0 0x7f1138198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113f476640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1138102250 0x7f1138198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113f476640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381078b0 0x7f113819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113f476640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1138111b20 con 0x7f11381078b0 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113f476640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11381119a0 con 0x7f1138102250 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113f476640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1138111ca0 con 0x7f11381018b0 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381078b0 0x7f113819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381078b0 0x7f113819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48678/0 (socket says 172.21.15.103:48678) 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 -- 172.21.15.103:0/1282769745 learned_addr learned my addr 172.21.15.103:0/1282769745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113ca11640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1138102250 0x7f1138198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113d212640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11381018b0 0x7f1138198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 -- 172.21.15.103:0/1282769745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11381018b0 msgr2=0x7f1138198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11381018b0 0x7f1138198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 -- 172.21.15.103:0/1282769745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1138102250 msgr2=0x7f1138198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1138102250 0x7f1138198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 -- 172.21.15.103:0/1282769745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113819cff0 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113d212640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11381018b0 0x7f1138198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113ca11640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1138102250 0x7f1138198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113da13640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381078b0 0x7f113819c910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f113000cce0 tx=0x7f113000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f11367fc640 1 -- 172.21.15.103:0/1282769745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113000f450 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113819d2e0 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f11367fc640 1 -- 172.21.15.103:0/1282769745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f113000faa0 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.516+0000 7f11367fc640 1 -- 172.21.15.103:0/1282769745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130018b70 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.517+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11381a4bc0 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.517+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1138101d10 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.518+0000 7f11367fc640 1 -- 172.21.15.103:0/1282769745 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1130018d10 con 0x7f11381078b0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.519+0000 7f11367fc640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11200a2b80 0x7f11200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.519+0000 7f113d212640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11200a2b80 0x7f11200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:17.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.520+0000 7f11367fc640 1 -- 172.21.15.103:0/1282769745 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f11300cf0e0 con 0x7f11381078b0 2024-08-19T01:33:17.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.520+0000 7f113d212640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11200a2b80 0x7f11200a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f112800baa0 tx=0x7f1128005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:17.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.524+0000 7f11367fc640 1 -- 172.21.15.103:0/1282769745 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1130014030 con 0x7f11381078b0 2024-08-19T01:33:17.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.692+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f11381098b0 con 0x7f11381078b0 2024-08-19T01:33:17.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.693+0000 7f11367fc640 1 -- 172.21.15.103:0/1282769745 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f113009c4e0 con 0x7f11381078b0 2024-08-19T01:33:17.694 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:17.694 INFO:teuthology.orchestra.run.smithi103.stdout:[ 2024-08-19T01:33:17.694 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:33:17.694 INFO:teuthology.orchestra.run.smithi103.stdout: "poolnum": 1, 2024-08-19T01:33:17.694 INFO:teuthology.orchestra.run.smithi103.stdout: "poolname": ".mgr" 2024-08-19T01:33:17.694 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:33:17.694 INFO:teuthology.orchestra.run.smithi103.stdout:] 2024-08-19T01:33:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11200a2b80 msgr2=0x7f11200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11200a2b80 0x7f11200a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f112800baa0 tx=0x7f1128005d20 comp rx=0 tx=0).stop 2024-08-19T01:33:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381078b0 msgr2=0x7f113819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381078b0 0x7f113819c910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f113000cce0 tx=0x7f113000a620 comp rx=0 tx=0).stop 2024-08-19T01:33:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 shutdown_connections 2024-08-19T01:33:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11200a2b80 0x7f11200a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11381078b0 0x7f113819c910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1138102250 0x7f1138198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 --2- 172.21.15.103:0/1282769745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11381018b0 0x7f1138198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:17.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 >> 172.21.15.103:0/1282769745 conn(0x7f11380fd5f0 msgr2=0x7f11380fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:17.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 shutdown_connections 2024-08-19T01:33:17.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:17.695+0000 7f113f476640 1 -- 172.21.15.103:0/1282769745 wait complete. 2024-08-19T01:33:18.449 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:18 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1282769745' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-19T01:33:18.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:18 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1282769745' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-19T01:33:18.480 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-19T01:33:18.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-19T01:33:18.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:18 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1282769745' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-19T01:33:18.731 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.229+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2468897856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4100850 msgr2=0x7f1dc4100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.229+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2468897856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4100850 0x7f1dc4100c30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009900 tx=0x7f1dbc02a410 comp rx=0 tx=0).stop 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2468897856 shutdown_connections 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2468897856 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc410e230 0x7f1dc4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2468897856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc4101170 0x7f1dc410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2468897856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4100850 0x7f1dc4100c30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2468897856 >> 172.21.15.103:0/2468897856 conn(0x7f1dc40fc500 msgr2=0x7f1dc40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2468897856 shutdown_connections 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2468897856 wait complete. 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 Processor -- start 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 -- start start 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.230+0000 7f1dcc4cc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc4100850 0x7f1dc419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcc4cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4101170 0x7f1dc419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcc4cc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc410e230 0x7f1dc419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcc4cc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4113f50 con 0x7f1dc4101170 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcc4cc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4113dd0 con 0x7f1dc4100850 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcc4cc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41140d0 con 0x7f1dc410e230 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4101170 0x7f1dc419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4101170 0x7f1dc419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48700/0 (socket says 172.21.15.103:48700) 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 -- 172.21.15.103:0/2992418663 learned_addr learned my addr 172.21.15.103:0/2992418663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcaa69640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc410e230 0x7f1dc419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dca268640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc4100850 0x7f1dc419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 -- 172.21.15.103:0/2992418663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc410e230 msgr2=0x7f1dc419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc410e230 0x7f1dc419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 -- 172.21.15.103:0/2992418663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc4100850 msgr2=0x7f1dc419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:19.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc4100850 0x7f1dc419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 -- 172.21.15.103:0/2992418663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc419f4e0 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dca268640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc4100850 0x7f1dc419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcaa69640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc410e230 0x7f1dc419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc9a67640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4101170 0x7f1dc419aa70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1db800b450 tx=0x7f1db800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc37fe640 1 -- 172.21.15.103:0/2992418663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8015020 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc37fe640 1 -- 172.21.15.103:0/2992418663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1db8011d40 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc419f7d0 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dc37fe640 1 -- 172.21.15.103:0/2992418663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db800f340 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.231+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a70b0 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.232+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc406a580 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.234+0000 7f1dc37fe640 1 -- 172.21.15.103:0/2992418663 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1db8018020 con 0x7f1dc4101170 2024-08-19T01:33:19.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.234+0000 7f1dc37fe640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1dac0a2c00 0x7f1dac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.234+0000 7f1dca268640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1dac0a2c00 0x7f1dac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.234+0000 7f1dc37fe640 1 -- 172.21.15.103:0/2992418663 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f1db80ce670 con 0x7f1dc4101170 2024-08-19T01:33:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.235+0000 7f1dca268640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1dac0a2c00 0x7f1dac0a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc031440 tx=0x7f1dbc0095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:19.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.238+0000 7f1dc37fe640 1 -- 172.21.15.103:0/2992418663 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1db801f020 con 0x7f1dc4101170 2024-08-19T01:33:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:19.407+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1dc4109330 con 0x7f1dc4101170 2024-08-19T01:33:19.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:19 smithi103 ceph-mon[18436]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:19 smithi146 ceph-mon[22098]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:19.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:19 smithi190 ceph-mon[22032]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:20.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.149+0000 7f1dc37fe640 1 -- 172.21.15.103:0/2992418663 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f1db800d040 con 0x7f1dc4101170 2024-08-19T01:33:20.150 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' created 2024-08-19T01:33:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1dac0a2c00 msgr2=0x7f1dac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1dac0a2c00 0x7f1dac0a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc031440 tx=0x7f1dbc0095f0 comp rx=0 tx=0).stop 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4101170 msgr2=0x7f1dc419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4101170 0x7f1dc419aa70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1db800b450 tx=0x7f1db800b910 comp rx=0 tx=0).stop 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 shutdown_connections 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1dac0a2c00 0x7f1dac0a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1dc410e230 0x7f1dc419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dc4101170 0x7f1dc419aa70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 --2- 172.21.15.103:0/2992418663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dc4100850 0x7f1dc419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.151+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 >> 172.21.15.103:0/2992418663 conn(0x7f1dc40fc500 msgr2=0x7f1dc40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.152+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 shutdown_connections 2024-08-19T01:33:20.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:20.152+0000 7f1dcc4cc640 1 -- 172.21.15.103:0/2992418663 wait complete. 2024-08-19T01:33:20.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:20 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2992418663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:33:20.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:20 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2992418663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:33:20.628 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:20 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2992418663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-19T01:33:20.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-19T01:33:21.208 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:21.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:21 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2992418663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-19T01:33:21.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:21 smithi103 ceph-mon[18436]: osdmap e80: 12 total, 12 up, 12 in 2024-08-19T01:33:21.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:21 smithi103 ceph-mon[18436]: pgmap v260: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:21 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2992418663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-19T01:33:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:21 smithi146 ceph-mon[22098]: osdmap e80: 12 total, 12 up, 12 in 2024-08-19T01:33:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:21 smithi146 ceph-mon[22098]: pgmap v260: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:21.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:21 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2992418663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-19T01:33:21.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:21 smithi190 ceph-mon[22032]: osdmap e80: 12 total, 12 up, 12 in 2024-08-19T01:33:21.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:21 smithi190 ceph-mon[22032]: pgmap v260: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:21.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 -- 172.21.15.103:0/1334362860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 msgr2=0x7fdc84101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 --2- 172.21.15.103:0/1334362860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc84101d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009930 tx=0x7fdc6c02a420 comp rx=0 tx=0).stop 2024-08-19T01:33:21.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 -- 172.21.15.103:0/1334362860 shutdown_connections 2024-08-19T01:33:21.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 --2- 172.21.15.103:0/1334362860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc84102250 0x7fdc8410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 --2- 172.21.15.103:0/1334362860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc84101d10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 --2- 172.21.15.103:0/1334362860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc841078b0 0x7fdc84107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 -- 172.21.15.103:0/1334362860 >> 172.21.15.103:0/1334362860 conn(0x7fdc840fd5f0 msgr2=0x7fdc840ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 -- 172.21.15.103:0/1334362860 shutdown_connections 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.692+0000 7fdc89695640 1 -- 172.21.15.103:0/1334362860 wait complete. 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 Processor -- start 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 -- start start 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc841980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc84102250 0x7fdc84198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc841078b0 0x7fdc8419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84111b60 con 0x7fdc841018b0 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841119e0 con 0x7fdc841078b0 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc89695640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84111ce0 con 0x7fdc84102250 2024-08-19T01:33:21.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc82ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc841980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc82ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc841980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48718/0 (socket says 172.21.15.103:48718) 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.693+0000 7fdc82ffd640 1 -- 172.21.15.103:0/2098863347 learned_addr learned my addr 172.21.15.103:0/2098863347 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc837fe640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc841078b0 0x7fdc8419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc82ffd640 1 -- 172.21.15.103:0/2098863347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc84102250 msgr2=0x7fdc84198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc82ffd640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc84102250 0x7fdc84198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc82ffd640 1 -- 172.21.15.103:0/2098863347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc841078b0 msgr2=0x7fdc8419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc82ffd640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc841078b0 0x7fdc8419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc82ffd640 1 -- 172.21.15.103:0/2098863347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8419d0a0 con 0x7fdc841018b0 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc837fe640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc841078b0 0x7fdc8419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc82ffd640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc841980f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00ea80 tx=0x7fdc7c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc7bfff640 1 -- 172.21.15.103:0/2098863347 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c00ce40 con 0x7fdc841018b0 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8419d390 con 0x7fdc841018b0 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc7bfff640 1 -- 172.21.15.103:0/2098863347 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc7c005020 con 0x7fdc841018b0 2024-08-19T01:33:21.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc7bfff640 1 -- 172.21.15.103:0/2098863347 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0116d0 con 0x7fdc841018b0 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.694+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc841a4c70 con 0x7fdc841018b0 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.695+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc84101d10 con 0x7fdc841018b0 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.696+0000 7fdc7bfff640 1 -- 172.21.15.103:0/2098863347 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdc7c0051d0 con 0x7fdc841018b0 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.697+0000 7fdc7bfff640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc700a2b80 0x7fdc700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.697+0000 7fdc827fc640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc700a2b80 0x7fdc700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.697+0000 7fdc7bfff640 1 -- 172.21.15.103:0/2098863347 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7fdc7c0cf430 con 0x7fdc841018b0 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.697+0000 7fdc827fc640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc700a2b80 0x7fdc700a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0096c0 tx=0x7fdc6c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:21.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.700+0000 7fdc7bfff640 1 -- 172.21.15.103:0/2098863347 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc7c016030 con 0x7fdc841018b0 2024-08-19T01:33:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:21.870+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdc84199450 con 0x7fdc841018b0 2024-08-19T01:33:22.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.162+0000 7fdc7bfff640 1 -- 172.21.15.103:0/2098863347 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) ==== 170+0+0 (secure 0 0 0) 0x7fdc7c09c6c0 con 0x7fdc841018b0 2024-08-19T01:33:22.163 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' created 2024-08-19T01:33:22.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.164+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc700a2b80 msgr2=0x7fdc700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.164+0000 7fdc89695640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc700a2b80 0x7fdc700a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0096c0 tx=0x7fdc6c031400 comp rx=0 tx=0).stop 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.164+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 msgr2=0x7fdc841980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.164+0000 7fdc89695640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc841980f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00ea80 tx=0x7fdc7c00ef40 comp rx=0 tx=0).stop 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.164+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 shutdown_connections 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.164+0000 7fdc89695640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc700a2b80 0x7fdc700a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.165+0000 7fdc89695640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc841078b0 0x7fdc8419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.165+0000 7fdc89695640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc84102250 0x7fdc84198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.165+0000 7fdc89695640 1 --2- 172.21.15.103:0/2098863347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc841018b0 0x7fdc841980f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.165+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 >> 172.21.15.103:0/2098863347 conn(0x7fdc840fd5f0 msgr2=0x7fdc840fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.165+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 shutdown_connections 2024-08-19T01:33:22.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:22.165+0000 7fdc89695640 1 -- 172.21.15.103:0/2098863347 wait complete. 2024-08-19T01:33:22.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:22 smithi146 ceph-mon[22098]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-19T01:33:22.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:22 smithi146 ceph-mon[22098]: osdmap e81: 12 total, 12 up, 12 in 2024-08-19T01:33:22.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:22 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2098863347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:33:22.543 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:22 smithi103 ceph-mon[18436]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-19T01:33:22.543 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:22 smithi103 ceph-mon[18436]: osdmap e81: 12 total, 12 up, 12 in 2024-08-19T01:33:22.543 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:22 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2098863347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:33:22.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:22 smithi190 ceph-mon[22032]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-19T01:33:22.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:22 smithi190 ceph-mon[22032]: osdmap e81: 12 total, 12 up, 12 in 2024-08-19T01:33:22.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:22 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2098863347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-19T01:33:22.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-19T01:33:23.251 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:23.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:23 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2098863347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-19T01:33:23.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:23 smithi103 ceph-mon[18436]: osdmap e82: 12 total, 12 up, 12 in 2024-08-19T01:33:23.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:23 smithi103 ceph-mon[18436]: pgmap v263: 97 pgs: 17 creating+peering, 66 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:23 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2098863347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-19T01:33:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:23 smithi146 ceph-mon[22098]: osdmap e82: 12 total, 12 up, 12 in 2024-08-19T01:33:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:23 smithi146 ceph-mon[22098]: pgmap v263: 97 pgs: 17 creating+peering, 66 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:23.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:23 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2098863347' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-19T01:33:23.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:23 smithi190 ceph-mon[22032]: osdmap e82: 12 total, 12 up, 12 in 2024-08-19T01:33:23.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:23 smithi190 ceph-mon[22032]: pgmap v263: 97 pgs: 17 creating+peering, 66 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 -- 172.21.15.103:0/158981073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410101160 msgr2=0x7fd41010d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 --2- 172.21.15.103:0/158981073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410101160 0x7fd41010d690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd408009cd0 tx=0x7fd40802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 -- 172.21.15.103:0/158981073 shutdown_connections 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 --2- 172.21.15.103:0/158981073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410101160 0x7fd41010d690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 --2- 172.21.15.103:0/158981073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4101007c0 0x7fd410100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 --2- 172.21.15.103:0/158981073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4101067c0 0x7fd410106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 -- 172.21.15.103:0/158981073 >> 172.21.15.103:0/158981073 conn(0x7fd4100fc500 msgr2=0x7fd4100fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 -- 172.21.15.103:0/158981073 shutdown_connections 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 -- 172.21.15.103:0/158981073 wait complete. 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 Processor -- start 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.780+0000 7fd4160b0640 1 -- start start 2024-08-19T01:33:23.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4101007c0 0x7fd410196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd410101160 0x7fd4101974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4101067c0 0x7fd41019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd410110a90 con 0x7fd4101007c0 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd410110910 con 0x7fd4101067c0 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd410110c10 con 0x7fd410101160 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4101007c0 0x7fd410196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4101007c0 0x7fd410196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59464/0 (socket says 172.21.15.103:59464) 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 -- 172.21.15.103:0/2723282575 learned_addr learned my addr 172.21.15.103:0/2723282575 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40ffff640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4101067c0 0x7fd41019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40effd640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd410101160 0x7fd4101974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 -- 172.21.15.103:0/2723282575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd410101160 msgr2=0x7fd4101974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd410101160 0x7fd4101974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 -- 172.21.15.103:0/2723282575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4101067c0 msgr2=0x7fd41019b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:23.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4101067c0 0x7fd41019b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 -- 172.21.15.103:0/2723282575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41019bf60 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40ffff640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4101067c0 0x7fd41019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40effd640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd410101160 0x7fd4101974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40f7fe640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4101007c0 0x7fd410196fb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d990 tx=0x7fd40000de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40cff9640 1 -- 172.21.15.103:0/2723282575 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40001b070 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40cff9640 1 -- 172.21.15.103:0/2723282575 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd40000edf0 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41019c250 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd40cff9640 1 -- 172.21.15.103:0/2723282575 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400014800 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.781+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41006a4c0 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.783+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd410100c20 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.784+0000 7fd40cff9640 1 -- 172.21.15.103:0/2723282575 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd40001e070 con 0x7fd4101007c0 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.784+0000 7fd40cff9640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd3f80a2c00 0x7fd3f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.784+0000 7fd40effd640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd3f80a2c00 0x7fd3f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:23.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.784+0000 7fd40cff9640 1 -- 172.21.15.103:0/2723282575 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fd4000d05a0 con 0x7fd4101007c0 2024-08-19T01:33:23.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.785+0000 7fd40effd640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd3f80a2c00 0x7fd3f80a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd410105e40 tx=0x7fd404005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:23.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.788+0000 7fd40cff9640 1 -- 172.21.15.103:0/2723282575 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd40009d6b0 con 0x7fd4101007c0 2024-08-19T01:33:23.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:23.990+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd410198310 con 0x7fd4101007c0 2024-08-19T01:33:24.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.171+0000 7fd40cff9640 1 -- 172.21.15.103:0/2723282575 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd40009cfb0 con 0x7fd4101007c0 2024-08-19T01:33:24.173 INFO:teuthology.orchestra.run.smithi103.stderr:new fs with metadata pool 2 and data pool 3 2024-08-19T01:33:24.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd3f80a2c00 msgr2=0x7fd3f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd3f80a2c00 0x7fd3f80a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd410105e40 tx=0x7fd404005e50 comp rx=0 tx=0).stop 2024-08-19T01:33:24.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4101007c0 msgr2=0x7fd410196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:24.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4101007c0 0x7fd410196fb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d990 tx=0x7fd40000de50 comp rx=0 tx=0).stop 2024-08-19T01:33:24.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 shutdown_connections 2024-08-19T01:33:24.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd3f80a2c00 0x7fd3f80a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4101067c0 0x7fd41019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd410101160 0x7fd4101974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 --2- 172.21.15.103:0/2723282575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4101007c0 0x7fd410196fb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:24.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 >> 172.21.15.103:0/2723282575 conn(0x7fd4100fc500 msgr2=0x7fd4100fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:24.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 shutdown_connections 2024-08-19T01:33:24.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:24.174+0000 7fd4160b0640 1 -- 172.21.15.103:0/2723282575 wait complete. 2024-08-19T01:33:24.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:24 smithi103 ceph-mon[18436]: osdmap e83: 12 total, 12 up, 12 in 2024-08-19T01:33:24.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:24 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2723282575' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-19T01:33:24.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:24 smithi103 ceph-mon[18436]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-19T01:33:24.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:24 smithi146 ceph-mon[22098]: osdmap e83: 12 total, 12 up, 12 in 2024-08-19T01:33:24.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:24 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2723282575' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-19T01:33:24.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:24 smithi146 ceph-mon[22098]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-19T01:33:24.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:24 smithi190 ceph-mon[22032]: osdmap e83: 12 total, 12 up, 12 in 2024-08-19T01:33:24.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:24 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2723282575' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-19T01:33:24.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:24 smithi190 ceph-mon[22032]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-19T01:33:24.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:25.221 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:25.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:25 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2723282575' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-19T01:33:25.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:25 smithi146 ceph-mon[22098]: osdmap e84: 12 total, 12 up, 12 in 2024-08-19T01:33:25.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:25 smithi146 ceph-mon[22098]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-19T01:33:25.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:25 smithi146 ceph-mon[22098]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-19T01:33:25.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:25 smithi146 ceph-mon[22098]: pgmap v266: 97 pgs: 25 creating+peering, 46 unknown, 26 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-19T01:33:25.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:25 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2723282575' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-19T01:33:25.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:25 smithi103 ceph-mon[18436]: osdmap e84: 12 total, 12 up, 12 in 2024-08-19T01:33:25.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:25 smithi103 ceph-mon[18436]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-19T01:33:25.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:25 smithi103 ceph-mon[18436]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-19T01:33:25.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:25 smithi103 ceph-mon[18436]: pgmap v266: 97 pgs: 25 creating+peering, 46 unknown, 26 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-19T01:33:25.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:25 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2723282575' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-19T01:33:25.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:25 smithi190 ceph-mon[22032]: osdmap e84: 12 total, 12 up, 12 in 2024-08-19T01:33:25.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:25 smithi190 ceph-mon[22032]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-19T01:33:25.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:25 smithi190 ceph-mon[22032]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-19T01:33:25.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:25 smithi190 ceph-mon[22032]: pgmap v266: 97 pgs: 25 creating+peering, 46 unknown, 26 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-19T01:33:25.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 -- 172.21.15.103:0/1208670681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c106690 msgr2=0x7f000c106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 --2- 172.21.15.103:0/1208670681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c106690 0x7f000c106a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efffc009930 tx=0x7efffc02a420 comp rx=0 tx=0).stop 2024-08-19T01:33:25.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 -- 172.21.15.103:0/1208670681 shutdown_connections 2024-08-19T01:33:25.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 --2- 172.21.15.103:0/1208670681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c100fc0 0x7f000c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 --2- 172.21.15.103:0/1208670681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c100600 0x7f000c100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 --2- 172.21.15.103:0/1208670681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c106690 0x7f000c106a70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 -- 172.21.15.103:0/1208670681 >> 172.21.15.103:0/1208670681 conn(0x7f000c0fc360 msgr2=0x7f000c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:25.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 -- 172.21.15.103:0/1208670681 shutdown_connections 2024-08-19T01:33:25.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.721+0000 7f00117a1640 1 -- 172.21.15.103:0/1208670681 wait complete. 2024-08-19T01:33:25.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.722+0000 7f00117a1640 1 Processor -- start 2024-08-19T01:33:25.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.722+0000 7f00117a1640 1 -- start start 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f00117a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100600 0x7f000c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f00117a1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c100fc0 0x7f000c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f00117a1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c106690 0x7f000c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f00117a1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f000c1108c0 con 0x7f000c100600 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f00117a1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f000c110740 con 0x7f000c100fc0 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f00117a1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f000c110a40 con 0x7f000c106690 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100600 0x7f000c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100600 0x7f000c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59478/0 (socket says 172.21.15.103:59478) 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 -- 172.21.15.103:0/3792930105 learned_addr learned my addr 172.21.15.103:0/3792930105 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000b7fe640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c100fc0 0x7f000c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f0010fa0640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c106690 0x7f000c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 -- 172.21.15.103:0/3792930105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c106690 msgr2=0x7f000c19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c106690 0x7f000c19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 -- 172.21.15.103:0/3792930105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c100fc0 msgr2=0x7f000c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c100fc0 0x7f000c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 -- 172.21.15.103:0/3792930105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c19cf10 con 0x7f000c100600 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f0010fa0640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c106690 0x7f000c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.723+0000 7f000bfff640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100600 0x7f000c197f60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efffc004770 tx=0x7efffc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.724+0000 7f00097fa640 1 -- 172.21.15.103:0/3792930105 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc03f070 con 0x7f000c100600 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.724+0000 7f00097fa640 1 -- 172.21.15.103:0/3792930105 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efffc03b460 con 0x7f000c100600 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.724+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c19d1a0 con 0x7f000c100600 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.724+0000 7f00097fa640 1 -- 172.21.15.103:0/3792930105 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc035070 con 0x7f000c100600 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.724+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c1a4ae0 con 0x7f000c100600 2024-08-19T01:33:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.725+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c101d40 con 0x7f000c100600 2024-08-19T01:33:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.726+0000 7f00097fa640 1 -- 172.21.15.103:0/3792930105 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efffc03b610 con 0x7f000c100600 2024-08-19T01:33:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.726+0000 7f00097fa640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efff40a2b80 0x7efff40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.726+0000 7f000b7fe640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efff40a2b80 0x7efff40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.726+0000 7f00097fa640 1 -- 172.21.15.103:0/3792930105 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7efffc0ef270 con 0x7f000c100600 2024-08-19T01:33:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.727+0000 7f000b7fe640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efff40a2b80 0x7efff40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f000c0fe6d0 tx=0x7f0000009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:25.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.731+0000 7f00097fa640 1 -- 172.21.15.103:0/3792930105 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efffc0bc320 con 0x7f000c100600 2024-08-19T01:33:25.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.900+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f000c1a4cb0 con 0x7f000c100600 2024-08-19T01:33:25.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.901+0000 7f00097fa640 1 -- 172.21.15.103:0/3792930105 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22301 (secure 0 0 0) 0x7efffc0bbc20 con 0x7f000c100600 2024-08-19T01:33:25.903 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:25.903 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:33:23.991024+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:25.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efff40a2b80 msgr2=0x7efff40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efff40a2b80 0x7efff40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f000c0fe6d0 tx=0x7f0000009110 comp rx=0 tx=0).stop 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100600 msgr2=0x7f000c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100600 0x7f000c197f60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efffc004770 tx=0x7efffc0042f0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 shutdown_connections 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efff40a2b80 0x7efff40a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f000c106690 0x7f000c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f000c100fc0 0x7f000c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 --2- 172.21.15.103:0/3792930105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f000c100600 0x7f000c197f60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 >> 172.21.15.103:0/3792930105 conn(0x7f000c0fc360 msgr2=0x7f000c06a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 shutdown_connections 2024-08-19T01:33:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:25.903+0000 7f00117a1640 1 -- 172.21.15.103:0/3792930105 wait complete. 2024-08-19T01:33:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:26 smithi146 ceph-mon[22098]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-19T01:33:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:26 smithi146 ceph-mon[22098]: Cluster is now healthy 2024-08-19T01:33:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:26 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] up:active 2024-08-19T01:33:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:26 smithi146 ceph-mon[22098]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:26 smithi146 ceph-mon[22098]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:26 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3792930105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:26.553 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:26 smithi103 ceph-mon[18436]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-19T01:33:26.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:26 smithi103 ceph-mon[18436]: Cluster is now healthy 2024-08-19T01:33:26.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:26 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] up:active 2024-08-19T01:33:26.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:26 smithi103 ceph-mon[18436]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:26.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:26 smithi103 ceph-mon[18436]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:26.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:26 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3792930105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:26.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:26.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:26 smithi190 ceph-mon[22032]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-19T01:33:26.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:26 smithi190 ceph-mon[22032]: Cluster is now healthy 2024-08-19T01:33:26.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:26 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] up:active 2024-08-19T01:33:26.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:26 smithi190 ceph-mon[22032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:26.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:26 smithi190 ceph-mon[22032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:26.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:26 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3792930105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:26.870 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:27.201 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:33:26 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b[52184]: 2024-08-19T01:33:26.857+0000 7feebf7bc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-19T01:33:27.296 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:27 smithi103 ceph-mon[18436]: pgmap v267: 97 pgs: 16 creating+peering, 17 unknown, 64 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 -- 172.21.15.103:0/1029432472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe81033f0 msgr2=0x7f4fe810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/1029432472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe81033f0 0x7f4fe810b8e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009900 tx=0x7f4fdc02a480 comp rx=0 tx=0).stop 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 -- 172.21.15.103:0/1029432472 shutdown_connections 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/1029432472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe810beb0 0x7f4fe810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/1029432472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe81033f0 0x7f4fe810b8e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/1029432472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fe8102ad0 0x7f4fe8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 -- 172.21.15.103:0/1029432472 >> 172.21.15.103:0/1029432472 conn(0x7f4fe80fc4d0 msgr2=0x7f4fe80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 -- 172.21.15.103:0/1029432472 shutdown_connections 2024-08-19T01:33:27.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.374+0000 7f4fecd4b640 1 -- 172.21.15.103:0/1029432472 wait complete. 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.375+0000 7f4fecd4b640 1 Processor -- start 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.375+0000 7f4fecd4b640 1 -- start start 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.375+0000 7f4fecd4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fe8102ad0 0x7f4fe81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fecd4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe81033f0 0x7f4fe8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fecd4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe810beb0 0x7f4fe819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fecd4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8111b80 con 0x7f4fe810beb0 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fecd4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8111a00 con 0x7f4fe8102ad0 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fecd4b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8111d00 con 0x7f4fe81033f0 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe810beb0 0x7f4fe819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe810beb0 0x7f4fe819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59506/0 (socket says 172.21.15.103:59506) 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 -- 172.21.15.103:0/2469515121 learned_addr learned my addr 172.21.15.103:0/2469515121 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe659c640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fe8102ad0 0x7f4fe81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:27.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 -- 172.21.15.103:0/2469515121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe81033f0 msgr2=0x7f4fe8198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe81033f0 0x7f4fe8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 -- 172.21.15.103:0/2469515121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fe8102ad0 msgr2=0x7f4fe81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fe8102ad0 0x7f4fe81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 -- 172.21.15.103:0/2469515121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe819d070 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe659c640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fe8102ad0 0x7f4fe81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fe6d9d640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe810beb0 0x7f4fe819c990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000b970 tx=0x7f4fe000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fd77fe640 1 -- 172.21.15.103:0/2469515121 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0019070 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe819d360 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fd77fe640 1 -- 172.21.15.103:0/2469515121 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fe0014440 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fd77fe640 1 -- 172.21.15.103:0/2469515121 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00075d0 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.376+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe81a4c40 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.377+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe8102eb0 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.378+0000 7f4fd77fe640 1 -- 172.21.15.103:0/2469515121 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4fe001c070 con 0x7f4fe810beb0 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.379+0000 7f4fd77fe640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4fd00a2b80 0x7f4fd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:27.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.379+0000 7f4fe659c640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4fd00a2b80 0x7f4fd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:27.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.379+0000 7f4fd77fe640 1 -- 172.21.15.103:0/2469515121 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f4fe00ce4f0 con 0x7f4fe810beb0 2024-08-19T01:33:27.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.379+0000 7f4fe659c640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4fd00a2b80 0x7f4fd00a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80058c0 tx=0x7f4fd8005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:27.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.383+0000 7f4fd77fe640 1 -- 172.21.15.103:0/2469515121 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4fe009b5a0 con 0x7f4fe810beb0 2024-08-19T01:33:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.550+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fe8106f50 con 0x7f4fe810beb0 2024-08-19T01:33:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.552+0000 7f4fd77fe640 1 -- 172.21.15.103:0/2469515121 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22301 (secure 0 0 0) 0x7f4fe009aea0 con 0x7f4fe810beb0 2024-08-19T01:33:27.554 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:27.554 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:33:23.991024+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4fd00a2b80 msgr2=0x7f4fd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4fd00a2b80 0x7f4fd00a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80058c0 tx=0x7f4fd8005d30 comp rx=0 tx=0).stop 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe810beb0 msgr2=0x7f4fe819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe810beb0 0x7f4fe819c990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000b970 tx=0x7f4fe000be30 comp rx=0 tx=0).stop 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 shutdown_connections 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4fd00a2b80 0x7f4fd00a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe810beb0 0x7f4fe819c990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fe81033f0 0x7f4fe8198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 --2- 172.21.15.103:0/2469515121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4fe8102ad0 0x7f4fe81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 >> 172.21.15.103:0/2469515121 conn(0x7f4fe80fc4d0 msgr2=0x7f4fe80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 shutdown_connections 2024-08-19T01:33:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:27.554+0000 7f4fecd4b640 1 -- 172.21.15.103:0/2469515121 wait complete. 2024-08-19T01:33:27.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:27 smithi190 ceph-mon[22032]: pgmap v267: 97 pgs: 16 creating+peering, 17 unknown, 64 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-19T01:33:27.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:27 smithi146 ceph-mon[22098]: pgmap v267: 97 pgs: 16 creating+peering, 17 unknown, 64 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-19T01:33:28.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-19T01:33:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:28.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:28 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2469515121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:28.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:28 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2469515121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:28.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:28 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2469515121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:29.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 -- 172.21.15.103:0/568026516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 msgr2=0x7efe90106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 --2- 172.21.15.103:0/568026516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe90106b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009930 tx=0x7efe8c02a420 comp rx=0 tx=0).stop 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 -- 172.21.15.103:0/568026516 shutdown_connections 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 --2- 172.21.15.103:0/568026516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe90101150 0x7efe9010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 --2- 172.21.15.103:0/568026516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe901007b0 0x7efe90100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 --2- 172.21.15.103:0/568026516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe90106b90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 -- 172.21.15.103:0/568026516 >> 172.21.15.103:0/568026516 conn(0x7efe900fc4d0 msgr2=0x7efe900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 -- 172.21.15.103:0/568026516 shutdown_connections 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.970+0000 7efe98525640 1 -- 172.21.15.103:0/568026516 wait complete. 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 Processor -- start 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 -- start start 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe901007b0 0x7efe90198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe90101150 0x7efe901993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe901a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe90110a20 con 0x7efe901067b0 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efe901108a0 con 0x7efe901007b0 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe98525640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efe90110ba0 con 0x7efe90101150 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe901a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe901a3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59526/0 (socket says 172.21.15.103:59526) 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 -- 172.21.15.103:0/271209044 learned_addr learned my addr 172.21.15.103:0/271209044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe95ac0640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe90101150 0x7efe901993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe962c1640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe901007b0 0x7efe90198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 -- 172.21.15.103:0/271209044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe90101150 msgr2=0x7efe901993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe90101150 0x7efe901993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 -- 172.21.15.103:0/271209044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe901007b0 msgr2=0x7efe90198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe901007b0 0x7efe90198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe96ac2640 1 -- 172.21.15.103:0/271209044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe901a4470 con 0x7efe901067b0 2024-08-19T01:33:29.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.971+0000 7efe95ac0640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe90101150 0x7efe901993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.972+0000 7efe96ac2640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe901a3f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe8800eaa0 tx=0x7efe8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.972+0000 7efe877fe640 1 -- 172.21.15.103:0/271209044 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8800cdb0 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.972+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe901a4640 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.972+0000 7efe877fe640 1 -- 172.21.15.103:0/271209044 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe88005020 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.972+0000 7efe877fe640 1 -- 172.21.15.103:0/271209044 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe880116d0 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.972+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe901a4b50 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.973+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe90100c10 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.974+0000 7efe877fe640 1 -- 172.21.15.103:0/271209044 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efe880051d0 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.974+0000 7efe877fe640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efe7c0a2c00 0x7efe7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.975+0000 7efe962c1640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efe7c0a2c00 0x7efe7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.975+0000 7efe877fe640 1 -- 172.21.15.103:0/271209044 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7efe880cf450 con 0x7efe901067b0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.975+0000 7efe962c1640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efe7c0a2c00 0x7efe7c0a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:29.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:28.978+0000 7efe877fe640 1 -- 172.21.15.103:0/271209044 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efe88016030 con 0x7efe901067b0 2024-08-19T01:33:29.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.181+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efe9002d5e0 con 0x7efe901067b0 2024-08-19T01:33:29.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.217+0000 7efe877fe640 1 -- 172.21.15.103:0/271209044 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7efe8809c580 con 0x7efe901067b0 2024-08-19T01:33:29.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efe7c0a2c00 msgr2=0x7efe7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:29.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efe7c0a2c00 0x7efe7c0a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c000bc0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 msgr2=0x7efe901a3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:29.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe901a3f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe8800eaa0 tx=0x7efe8800ef60 comp rx=0 tx=0).stop 2024-08-19T01:33:29.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 shutdown_connections 2024-08-19T01:33:29.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efe7c0a2c00 0x7efe7c0a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe901067b0 0x7efe901a3f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe90101150 0x7efe901993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 --2- 172.21.15.103:0/271209044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe901007b0 0x7efe90198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:29.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 >> 172.21.15.103:0/271209044 conn(0x7efe900fc4d0 msgr2=0x7efe900fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:29.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 shutdown_connections 2024-08-19T01:33:29.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:29.220+0000 7efe98525640 1 -- 172.21.15.103:0/271209044 wait complete. 2024-08-19T01:33:29.543 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:29 smithi103 ceph-mon[18436]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-08-19T01:33:29.543 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:29 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/271209044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:33:29.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:29 smithi190 ceph-mon[22032]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-08-19T01:33:29.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:29 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/271209044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:33:29.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:29 smithi146 ceph-mon[22098]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-08-19T01:33:29.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:29 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/271209044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-19T01:33:29.871 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-08-19T01:33:29.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-08-19T01:33:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:30.515 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:30 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/271209044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-19T01:33:30.516 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:30 smithi103 ceph-mon[18436]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:30.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:30 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/271209044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-19T01:33:30.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:30 smithi190 ceph-mon[22032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:30.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/1848764331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 msgr2=0x7f2ee01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:30.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/1848764331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee01026e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009900 tx=0x7f2ecc02a480 comp rx=0 tx=0).stop 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/1848764331 shutdown_connections 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/1848764331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee0102c20 0x7f2ee010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/1848764331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee01026e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/1848764331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee0100b50 0x7f2ee0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/1848764331 >> 172.21.15.103:0/1848764331 conn(0x7f2ee00fc500 msgr2=0x7f2ee00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/1848764331 shutdown_connections 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.596+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/1848764331 wait complete. 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 Processor -- start 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 -- start start 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee0100b50 0x7f2ee0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:30.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee0102c20 0x7f2ee01a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0110b60 con 0x7f2ee01022a0 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01109e0 con 0x7f2ee0102c20 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee6cf1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0110ce0 con 0x7f2ee0100b50 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2edffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2edffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee0199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59544/0 (socket says 172.21.15.103:59544) 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2edffff640 1 -- 172.21.15.103:0/879320568 learned_addr learned my addr 172.21.15.103:0/879320568 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee4a8d640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee0100b50 0x7f2ee0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.597+0000 7f2ee528e640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee0102c20 0x7f2ee01a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2edffff640 1 -- 172.21.15.103:0/879320568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee0100b50 msgr2=0x7f2ee0198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2edffff640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee0100b50 0x7f2ee0198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2edffff640 1 -- 172.21.15.103:0/879320568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee0102c20 msgr2=0x7f2ee01a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2edffff640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee0102c20 0x7f2ee01a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2edffff640 1 -- 172.21.15.103:0/879320568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee01a44f0 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2ee528e640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee0102c20 0x7f2ee01a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2edffff640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee0199470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc006010 tx=0x7f2ecc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2eddffb640 1 -- 172.21.15.103:0/879320568 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc03f030 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee01a46c0 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2eddffb640 1 -- 172.21.15.103:0/879320568 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ecc003cb0 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2eddffb640 1 -- 172.21.15.103:0/879320568 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc035070 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.598+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a4b10 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.599+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee0100f30 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.600+0000 7f2eddffb640 1 -- 172.21.15.103:0/879320568 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2ecc042030 con 0x7f2ee01022a0 2024-08-19T01:33:30.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.601+0000 7f2eddffb640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2ed00a2b80 0x7f2ed00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:30.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.601+0000 7f2ee4a8d640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2ed00a2b80 0x7f2ed00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:30.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.601+0000 7f2eddffb640 1 -- 172.21.15.103:0/879320568 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f2ecc0ef660 con 0x7f2ee01022a0 2024-08-19T01:33:30.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.601+0000 7f2ee4a8d640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2ed00a2b80 0x7f2ed00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005fd0 tx=0x7f2ed400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:30.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.604+0000 7f2eddffb640 1 -- 172.21.15.103:0/879320568 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2ecc0bc710 con 0x7f2ee01022a0 2024-08-19T01:33:30.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:30 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/271209044' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-19T01:33:30.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:30 smithi146 ceph-mon[22098]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-19T01:33:30.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.806+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f2ee0108dc0 con 0x7f2ee01022a0 2024-08-19T01:33:30.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.926+0000 7f2eddffb640 1 -- 172.21.15.103:0/879320568 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f2ecc0bc010 con 0x7f2ee01022a0 2024-08-19T01:33:30.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2ed00a2b80 msgr2=0x7f2ed00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:30.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2ed00a2b80 0x7f2ed00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005fd0 tx=0x7f2ed400b390 comp rx=0 tx=0).stop 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 msgr2=0x7f2ee0199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee0199470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc006010 tx=0x7f2ecc0042f0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 shutdown_connections 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2ed00a2b80 0x7f2ed00a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee0102c20 0x7f2ee01a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ee01022a0 0x7f2ee0199470 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.928+0000 7f2ee6cf1640 1 --2- 172.21.15.103:0/879320568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ee0100b50 0x7f2ee0198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.929+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 >> 172.21.15.103:0/879320568 conn(0x7f2ee00fc500 msgr2=0x7f2ee00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.929+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 shutdown_connections 2024-08-19T01:33:30.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:30.929+0000 7f2ee6cf1640 1 -- 172.21.15.103:0/879320568 wait complete. 2024-08-19T01:33:31.553 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:31 smithi103 ceph-mon[18436]: pgmap v269: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-19T01:33:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:31 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/879320568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-08-19T01:33:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:31 smithi103 ceph-mon[18436]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-19T01:33:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:31 smithi103 ceph-mon[18436]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-19T01:33:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:31 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/879320568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-19T01:33:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:31 smithi103 ceph-mon[18436]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-19T01:33:31.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:31 smithi103 ceph-mon[18436]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-19T01:33:31.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-19T01:33:31.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:31 smithi190 ceph-mon[22032]: pgmap v269: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-19T01:33:31.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:31 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/879320568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-08-19T01:33:31.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:31 smithi190 ceph-mon[22032]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-19T01:33:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:31 smithi190 ceph-mon[22032]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-19T01:33:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:31 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/879320568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-19T01:33:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:31 smithi190 ceph-mon[22032]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-19T01:33:31.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:31 smithi190 ceph-mon[22032]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-19T01:33:31.700 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:31 smithi146 ceph-mon[22098]: pgmap v269: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-19T01:33:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:31 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/879320568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-08-19T01:33:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:31 smithi146 ceph-mon[22098]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-19T01:33:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:31 smithi146 ceph-mon[22098]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-19T01:33:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:31 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/879320568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-19T01:33:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:31 smithi146 ceph-mon[22098]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-19T01:33:31.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:31 smithi146 ceph-mon[22098]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-19T01:33:31.865 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 -- 172.21.15.103:0/1748445647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be41022a0 msgr2=0x7f5be41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/1748445647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be41022a0 0x7f5be41026e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009900 tx=0x7f5bcc02a480 comp rx=0 tx=0).stop 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 -- 172.21.15.103:0/1748445647 shutdown_connections 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/1748445647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be4102c20 0x7f5be410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/1748445647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be41022a0 0x7f5be41026e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/1748445647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5be4100b50 0x7f5be4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 -- 172.21.15.103:0/1748445647 >> 172.21.15.103:0/1748445647 conn(0x7f5be40fc500 msgr2=0x7f5be40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 -- 172.21.15.103:0/1748445647 shutdown_connections 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 -- 172.21.15.103:0/1748445647 wait complete. 2024-08-19T01:33:32.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 Processor -- start 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.375+0000 7f5be8dc9640 1 -- start start 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be4100b50 0x7f5be4198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5be41022a0 0x7f5be41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be4102c20 0x7f5be41a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4110b60 con 0x7f5be4100b50 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41109e0 con 0x7f5be41022a0 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4110ce0 con 0x7f5be4102c20 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be4100b50 0x7f5be4198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be4100b50 0x7f5be4198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50736/0 (socket says 172.21.15.103:50736) 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 -- 172.21.15.103:0/327923069 learned_addr learned my addr 172.21.15.103:0/327923069 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be2d9d640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be4102c20 0x7f5be41a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be1d9b640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5be41022a0 0x7f5be41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 -- 172.21.15.103:0/327923069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be4102c20 msgr2=0x7f5be41a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be4102c20 0x7f5be41a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 -- 172.21.15.103:0/327923069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5be41022a0 msgr2=0x7f5be41994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5be41022a0 0x7f5be41994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 -- 172.21.15.103:0/327923069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41a4560 con 0x7f5be4100b50 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be1d9b640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5be41022a0 0x7f5be41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be259c640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be4100b50 0x7f5be4198fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400cce0 tx=0x7f5bd400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5bdb7fe640 1 -- 172.21.15.103:0/327923069 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4004b20 con 0x7f5be4100b50 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5bdb7fe640 1 -- 172.21.15.103:0/327923069 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bd4004cc0 con 0x7f5be4100b50 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41a4730 con 0x7f5be4100b50 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5bdb7fe640 1 -- 172.21.15.103:0/327923069 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd400f6b0 con 0x7f5be4100b50 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.376+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a4c40 con 0x7f5be4100b50 2024-08-19T01:33:32.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.378+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be4100f30 con 0x7f5be4100b50 2024-08-19T01:33:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.378+0000 7f5bdb7fe640 1 -- 172.21.15.103:0/327923069 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5bd400f850 con 0x7f5be4100b50 2024-08-19T01:33:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.379+0000 7f5bdb7fe640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5bd00a2b80 0x7f5bd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.379+0000 7f5be1d9b640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5bd00a2b80 0x7f5bd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.379+0000 7f5bdb7fe640 1 -- 172.21.15.103:0/327923069 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f5bd40cf060 con 0x7f5be4100b50 2024-08-19T01:33:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.380+0000 7f5be1d9b640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5bd00a2b80 0x7f5bd00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000c00 tx=0x7f5bcc032c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.383+0000 7f5bdb7fe640 1 -- 172.21.15.103:0/327923069 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5bd4014030 con 0x7f5be4100b50 2024-08-19T01:33:32.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.588+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5be4108dc0 con 0x7f5be4100b50 2024-08-19T01:33:32.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.937+0000 7f5bdb7fe640 1 -- 172.21.15.103:0/327923069 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f5bd409c110 con 0x7f5be4100b50 2024-08-19T01:33:32.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5bd00a2b80 msgr2=0x7f5bd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5bd00a2b80 0x7f5bd00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000c00 tx=0x7f5bcc032c70 comp rx=0 tx=0).stop 2024-08-19T01:33:32.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be4100b50 msgr2=0x7f5be4198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:32.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be4100b50 0x7f5be4198fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400cce0 tx=0x7f5bd400a620 comp rx=0 tx=0).stop 2024-08-19T01:33:32.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 shutdown_connections 2024-08-19T01:33:32.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5bd00a2b80 0x7f5bd00a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5be4102c20 0x7f5be41a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5be41022a0 0x7f5be41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 --2- 172.21.15.103:0/327923069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5be4100b50 0x7f5be4198fa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:32.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 >> 172.21.15.103:0/327923069 conn(0x7f5be40fc500 msgr2=0x7f5be40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:32.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 shutdown_connections 2024-08-19T01:33:32.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:32.939+0000 7f5be8dc9640 1 -- 172.21.15.103:0/327923069 wait complete. 2024-08-19T01:33:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:32 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] up:active 2024-08-19T01:33:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:32 smithi146 ceph-mon[22098]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-19T01:33:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:32 smithi146 ceph-mon[22098]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-19T01:33:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:32 smithi146 ceph-mon[22098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-08-19T01:33:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:32 smithi146 ceph-mon[22098]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-19T01:33:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:32 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/327923069' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:33:33.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:32 smithi146 ceph-mon[22098]: pgmap v270: 97 pgs: 97 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 7 op/s 2024-08-19T01:33:33.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:32 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] up:active 2024-08-19T01:33:33.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:32 smithi103 ceph-mon[18436]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-19T01:33:33.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:32 smithi103 ceph-mon[18436]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-19T01:33:33.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:32 smithi103 ceph-mon[18436]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-08-19T01:33:33.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:32 smithi103 ceph-mon[18436]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-19T01:33:33.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:32 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/327923069' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:33:33.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:32 smithi103 ceph-mon[18436]: pgmap v270: 97 pgs: 97 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 7 op/s 2024-08-19T01:33:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:32 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] up:active 2024-08-19T01:33:33.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:32 smithi190 ceph-mon[22032]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-19T01:33:33.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:32 smithi190 ceph-mon[22032]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-19T01:33:33.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:32 smithi190 ceph-mon[22032]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-08-19T01:33:33.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:32 smithi190 ceph-mon[22032]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-19T01:33:33.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:32 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/327923069' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-19T01:33:33.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:32 smithi190 ceph-mon[22032]: pgmap v270: 97 pgs: 97 active+clean; 580 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 7 op/s 2024-08-19T01:33:33.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-19T01:33:33.920 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:34.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:33 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/327923069' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-19T01:33:34.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:33 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] up:active 2024-08-19T01:33:34.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:33 smithi103 ceph-mon[18436]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-19T01:33:34.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:33 smithi103 ceph-mon[18436]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-08-19T01:33:34.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:33 smithi103 ceph-mon[18436]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-08-19T01:33:34.194 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:33 smithi103 ceph-mon[18436]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-08-19T01:33:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:33 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/327923069' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-19T01:33:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:33 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] up:active 2024-08-19T01:33:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:33 smithi146 ceph-mon[22098]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-19T01:33:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:33 smithi146 ceph-mon[22098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-08-19T01:33:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:33 smithi146 ceph-mon[22098]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-08-19T01:33:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:33 smithi146 ceph-mon[22098]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-08-19T01:33:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:33 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/327923069' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-19T01:33:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:33 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] up:active 2024-08-19T01:33:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:33 smithi190 ceph-mon[22032]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-19T01:33:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:33 smithi190 ceph-mon[22032]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-08-19T01:33:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:33 smithi190 ceph-mon[22032]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-08-19T01:33:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:33 smithi190 ceph-mon[22032]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-08-19T01:33:34.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 -- 172.21.15.103:0/244033622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e81034b0 msgr2=0x7f87e81078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/244033622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e81034b0 0x7f87e81078f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009930 tx=0x7f87d802a420 comp rx=0 tx=0).stop 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 -- 172.21.15.103:0/244033622 shutdown_connections 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/244033622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87e8108030 0x7f87e810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/244033622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e81034b0 0x7f87e81078f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/244033622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e8102b00 0x7f87e8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 -- 172.21.15.103:0/244033622 >> 172.21.15.103:0/244033622 conn(0x7f87e80fc500 msgr2=0x7f87e80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 -- 172.21.15.103:0/244033622 shutdown_connections 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.452+0000 7f87ef2a3640 1 -- 172.21.15.103:0/244033622 wait complete. 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 Processor -- start 2024-08-19T01:33:34.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 -- start start 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e8102b00 0x7f87e8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e81034b0 0x7f87e81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87e8108030 0x7f87e819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111cd0 con 0x7f87e8102b00 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111b50 con 0x7f87e8108030 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ef2a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111e50 con 0x7f87e81034b0 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e8102b00 0x7f87e8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e8102b00 0x7f87e8198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50746/0 (socket says 172.21.15.103:50746) 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 -- 172.21.15.103:0/2145886871 learned_addr learned my addr 172.21.15.103:0/2145886871 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ec83e640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e81034b0 0x7f87e81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 -- 172.21.15.103:0/2145886871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e81034b0 msgr2=0x7f87e81987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e81034b0 0x7f87e81987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 -- 172.21.15.103:0/2145886871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87e8108030 msgr2=0x7f87e819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87e8108030 0x7f87e819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 -- 172.21.15.103:0/2145886871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e819d240 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87ed03f640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e8102b00 0x7f87e8198290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d770 tx=0x7f87e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.453+0000 7f87de7fc640 1 -- 172.21.15.103:0/2145886871 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4017070 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.454+0000 7f87de7fc640 1 -- 172.21.15.103:0/2145886871 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87e40054d0 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.454+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e819d530 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.454+0000 7f87de7fc640 1 -- 172.21.15.103:0/2145886871 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4004cf0 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.454+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81a4e10 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.455+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8102ee0 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.456+0000 7f87de7fc640 1 -- 172.21.15.103:0/2145886871 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f87e4004e90 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.456+0000 7f87de7fc640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87d40a2b80 0x7f87d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.456+0000 7f87ec83e640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87d40a2b80 0x7f87d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.456+0000 7f87de7fc640 1 -- 172.21.15.103:0/2145886871 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f87e40ce960 con 0x7f87e8102b00 2024-08-19T01:33:34.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.457+0000 7f87ec83e640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87d40a2b80 0x7f87d40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f87d8031440 tx=0x7f87d8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:34.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.460+0000 7f87de7fc640 1 -- 172.21.15.103:0/2145886871 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f87e409ba10 con 0x7f87e8102b00 2024-08-19T01:33:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.662+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f87e810a010 con 0x7f87e8102b00 2024-08-19T01:33:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.958+0000 7f87de7fc640 1 -- 172.21.15.103:0/2145886871 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f87e409b310 con 0x7f87e8102b00 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.961+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87d40a2b80 msgr2=0x7f87d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.961+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87d40a2b80 0x7f87d40a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f87d8031440 tx=0x7f87d8000bc0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.961+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e8102b00 msgr2=0x7f87e8198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.961+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e8102b00 0x7f87e8198290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d770 tx=0x7f87e400dc30 comp rx=0 tx=0).stop 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 shutdown_connections 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f87d40a2b80 0x7f87d40a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87e8108030 0x7f87e819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e81034b0 0x7f87e81987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 --2- 172.21.15.103:0/2145886871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87e8102b00 0x7f87e8198290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 >> 172.21.15.103:0/2145886871 conn(0x7f87e80fc500 msgr2=0x7f87e8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 shutdown_connections 2024-08-19T01:33:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:34.962+0000 7f87ef2a3640 1 -- 172.21.15.103:0/2145886871 wait complete. 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] up:active 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: Cluster is now healthy 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2145886871' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-19T01:33:35.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:34 smithi146 ceph-mon[22098]: pgmap v271: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] up:active 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: Cluster is now healthy 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2145886871' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-19T01:33:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:34 smithi103 ceph-mon[18436]: pgmap v271: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] up:active 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: Cluster is now healthy 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2145886871' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-19T01:33:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:34 smithi190 ceph-mon[22032]: pgmap v271: 97 pgs: 97 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-08-19T01:33:35.694 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-08-19T01:33:35.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-19T01:33:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:36.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:35 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2145886871' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-19T01:33:36.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:35 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] up:active 2024-08-19T01:33:36.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:35 smithi103 ceph-mon[18436]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-19T01:33:36.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:35 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:33:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:35 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2145886871' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-19T01:33:36.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:35 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] up:active 2024-08-19T01:33:36.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:35 smithi190 ceph-mon[22032]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-19T01:33:36.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:35 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:33:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:35 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2145886871' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-19T01:33:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:35 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] up:active 2024-08-19T01:33:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:35 smithi146 ceph-mon[22098]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-19T01:33:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:35 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:33:36.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 -- 172.21.15.103:0/1110578285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 msgr2=0x7fdae0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:36.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 --2- 172.21.15.103:0/1110578285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae0100c10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009930 tx=0x7fdad002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 -- 172.21.15.103:0/1110578285 shutdown_connections 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 --2- 172.21.15.103:0/1110578285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdae0101150 0x7fdae010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 --2- 172.21.15.103:0/1110578285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae0100c10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 --2- 172.21.15.103:0/1110578285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdae01067b0 0x7fdae0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 -- 172.21.15.103:0/1110578285 >> 172.21.15.103:0/1110578285 conn(0x7fdae00fc4d0 msgr2=0x7fdae00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 -- 172.21.15.103:0/1110578285 shutdown_connections 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.449+0000 7fdae8368640 1 -- 172.21.15.103:0/1110578285 wait complete. 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 Processor -- start 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 -- start start 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdae0101150 0x7fdae0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:36.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdae01067b0 0x7fdae019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0110a80 con 0x7fdae01007b0 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0110900 con 0x7fdae01067b0 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae8368640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0110c00 con 0x7fdae0101150 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50766/0 (socket says 172.21.15.103:50766) 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 -- 172.21.15.103:0/2372168675 learned_addr learned my addr 172.21.15.103:0/2372168675 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae5903640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdae0101150 0x7fdae0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6905640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdae01067b0 0x7fdae019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 -- 172.21.15.103:0/2372168675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdae0101150 msgr2=0x7fdae0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdae0101150 0x7fdae0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 -- 172.21.15.103:0/2372168675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdae01067b0 msgr2=0x7fdae019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdae01067b0 0x7fdae019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 -- 172.21.15.103:0/2372168675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae019d090 con 0x7fdae01007b0 2024-08-19T01:33:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6905640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdae01067b0 0x7fdae019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.450+0000 7fdae6104640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae01980e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00b4a0 tx=0x7fdadc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.451+0000 7fdad77fe640 1 -- 172.21.15.103:0/2372168675 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc014070 con 0x7fdae01007b0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.451+0000 7fdad77fe640 1 -- 172.21.15.103:0/2372168675 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdadc010ac0 con 0x7fdae01007b0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.451+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae019d380 con 0x7fdae01007b0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.451+0000 7fdad77fe640 1 -- 172.21.15.103:0/2372168675 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc017c30 con 0x7fdae01007b0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.451+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae01a4c60 con 0x7fdae01007b0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.452+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae0100c10 con 0x7fdae01007b0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.453+0000 7fdad77fe640 1 -- 172.21.15.103:0/2372168675 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdadc017070 con 0x7fdae01007b0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.453+0000 7fdad77fe640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdacc0a2b80 0x7fdacc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.454+0000 7fdae5903640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdacc0a2b80 0x7fdacc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.454+0000 7fdad77fe640 1 -- 172.21.15.103:0/2372168675 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fdadc0ce390 con 0x7fdae01007b0 2024-08-19T01:33:36.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.454+0000 7fdae5903640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdacc0a2b80 0x7fdacc0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdad0000c00 tx=0x7fdad0031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:36.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.458+0000 7fdad77fe640 1 -- 172.21.15.103:0/2372168675 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdadc09b440 con 0x7fdae01007b0 2024-08-19T01:33:36.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.648+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdae0109dc0 con 0x7fdacc0a2b80 2024-08-19T01:33:36.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.678+0000 7fdad77fe640 1 -- 172.21.15.103:0/2372168675 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdad00314a0 con 0x7fdacc0a2b80 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdacc0a2b80 msgr2=0x7fdacc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdacc0a2b80 0x7fdacc0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdad0000c00 tx=0x7fdad0031400 comp rx=0 tx=0).stop 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 msgr2=0x7fdae01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae01980e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00b4a0 tx=0x7fdadc00b960 comp rx=0 tx=0).stop 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 shutdown_connections 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdacc0a2b80 0x7fdacc0a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdae01067b0 0x7fdae019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdae0101150 0x7fdae0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 --2- 172.21.15.103:0/2372168675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae01007b0 0x7fdae01980e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:36.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 >> 172.21.15.103:0/2372168675 conn(0x7fdae00fc4d0 msgr2=0x7fdae006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:36.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 shutdown_connections 2024-08-19T01:33:36.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:36.680+0000 7fdae8368640 1 -- 172.21.15.103:0/2372168675 wait complete. 2024-08-19T01:33:36.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:33:36 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:33:36.652+0000 7f4f64543640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:33:36.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:33:36 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:33:36.652+0000 7f4f64543640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:33:36.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:33:36 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:33:36.652+0000 7f4f64543640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:33:36.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:33:36 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:33:36.652+0000 7f4f64543640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:33:36.804 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:33:36 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:33:36.652+0000 7f4f64543640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-19T01:33:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:36 smithi103 ceph-mon[18436]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:36 smithi103 ceph-mon[18436]: pgmap v272: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.3 KiB/s wr, 15 op/s 2024-08-19T01:33:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:36 smithi190 ceph-mon[22032]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:36 smithi190 ceph-mon[22032]: pgmap v272: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.3 KiB/s wr, 15 op/s 2024-08-19T01:33:37.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-19T01:33:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:36 smithi146 ceph-mon[22098]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:36 smithi146 ceph-mon[22098]: pgmap v272: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 4.3 KiB/s wr, 15 op/s 2024-08-19T01:33:37.638 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:38.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:37 smithi103 ceph-mon[18436]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-19T01:33:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 -- 172.21.15.103:0/2918918500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc101150 msgr2=0x7f9afc10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/2918918500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc101150 0x7f9afc10d680 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009cd0 tx=0x7f9af402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 -- 172.21.15.103:0/2918918500 shutdown_connections 2024-08-19T01:33:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/2918918500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc101150 0x7f9afc10d680 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/2918918500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9afc1007b0 0x7f9afc100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/2918918500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9afc1067b0 0x7f9afc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 -- 172.21.15.103:0/2918918500 >> 172.21.15.103:0/2918918500 conn(0x7f9afc0fc4d0 msgr2=0x7f9afc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 -- 172.21.15.103:0/2918918500 shutdown_connections 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.172+0000 7f9b03ea7640 1 -- 172.21.15.103:0/2918918500 wait complete. 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 Processor -- start 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 -- start start 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc1007b0 0x7f9afc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9afc101150 0x7f9afc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9afc1067b0 0x7f9afc19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc111b90 con 0x7f9afc1007b0 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc111a10 con 0x7f9afc1067b0 2024-08-19T01:33:38.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b03ea7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc111d10 con 0x7f9afc101150 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b01c43640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc1007b0 0x7f9afc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b01c43640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc1007b0 0x7f9afc198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50792/0 (socket says 172.21.15.103:50792) 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b01c43640 1 -- 172.21.15.103:0/161208571 learned_addr learned my addr 172.21.15.103:0/161208571 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b02444640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9afc1067b0 0x7f9afc19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b01442640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9afc101150 0x7f9afc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b01c43640 1 -- 172.21.15.103:0/161208571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9afc101150 msgr2=0x7f9afc1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.173+0000 7f9b01c43640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9afc101150 0x7f9afc1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b01c43640 1 -- 172.21.15.103:0/161208571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9afc1067b0 msgr2=0x7f9afc19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b01c43640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9afc1067b0 0x7f9afc19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b01c43640 1 -- 172.21.15.103:0/161208571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc19d020 con 0x7f9afc1007b0 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b01442640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9afc101150 0x7f9afc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b02444640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9afc1067b0 0x7f9afc19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:38.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b01c43640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc1007b0 0x7f9afc198070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9af800b4a0 tx=0x7f9af800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9af2ffd640 1 -- 172.21.15.103:0/161208571 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8014070 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9afc19d310 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9af2ffd640 1 -- 172.21.15.103:0/161208571 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9af8010ac0 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9af2ffd640 1 -- 172.21.15.103:0/161208571 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8017c30 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.174+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9afc1a4bf0 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.175+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9afc100c10 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.176+0000 7f9af2ffd640 1 -- 172.21.15.103:0/161208571 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9af8017070 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.177+0000 7f9af2ffd640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9ae80a2c10 0x7f9ae80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.177+0000 7f9b01442640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9ae80a2c10 0x7f9ae80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.177+0000 7f9af2ffd640 1 -- 172.21.15.103:0/161208571 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f9af80ce770 con 0x7f9afc1007b0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.177+0000 7f9b01442640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9ae80a2c10 0x7f9ae80a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0fea80 tx=0x7f9aec00a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:38.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.180+0000 7f9af2ffd640 1 -- 172.21.15.103:0/161208571 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9af809b830 con 0x7f9afc1007b0 2024-08-19T01:33:38.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.369+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9afc062360 con 0x7f9ae80a2c10 2024-08-19T01:33:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:37 smithi190 ceph-mon[22032]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-19T01:33:38.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.393+0000 7f9af2ffd640 1 -- 172.21.15.103:0/161208571 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9afc062360 con 0x7f9ae80a2c10 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9ae80a2c10 msgr2=0x7f9ae80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9ae80a2c10 0x7f9ae80a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0fea80 tx=0x7f9aec00a5c0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc1007b0 msgr2=0x7f9afc198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc1007b0 0x7f9afc198070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9af800b4a0 tx=0x7f9af800b960 comp rx=0 tx=0).stop 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 shutdown_connections 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9ae80a2c10 0x7f9ae80a50d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9afc1067b0 0x7f9afc19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9afc101150 0x7f9afc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 --2- 172.21.15.103:0/161208571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc1007b0 0x7f9afc198070 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 >> 172.21.15.103:0/161208571 conn(0x7f9afc0fc4d0 msgr2=0x7f9afc0fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 shutdown_connections 2024-08-19T01:33:38.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:38.395+0000 7f9b03ea7640 1 -- 172.21.15.103:0/161208571 wait complete. 2024-08-19T01:33:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:37 smithi146 ceph-mon[22098]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-19T01:33:39.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:38 smithi103 ceph-mon[18436]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:39.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:38 smithi103 ceph-mon[18436]: pgmap v273: 97 pgs: 97 active+clean; 592 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-08-19T01:33:39.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-19T01:33:39.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:38 smithi190 ceph-mon[22032]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:39.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:38 smithi190 ceph-mon[22032]: pgmap v273: 97 pgs: 97 active+clean; 592 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-08-19T01:33:39.450 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:39.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:38 smithi146 ceph-mon[22098]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:39.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:38 smithi146 ceph-mon[22098]: pgmap v273: 97 pgs: 97 active+clean; 592 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.956+0000 7f4695fd2640 1 -- 172.21.15.103:0/2247809482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46901018b0 msgr2=0x7f4690101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.956+0000 7f4695fd2640 1 --2- 172.21.15.103:0/2247809482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46901018b0 0x7f4690101d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4678009930 tx=0x7f467802a420 comp rx=0 tx=0).stop 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.956+0000 7f4695fd2640 1 -- 172.21.15.103:0/2247809482 shutdown_connections 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.956+0000 7f4695fd2640 1 --2- 172.21.15.103:0/2247809482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4690102250 0x7f469010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.956+0000 7f4695fd2640 1 --2- 172.21.15.103:0/2247809482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46901018b0 0x7f4690101d10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.956+0000 7f4695fd2640 1 --2- 172.21.15.103:0/2247809482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46901078b0 0x7f4690107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.956+0000 7f4695fd2640 1 -- 172.21.15.103:0/2247809482 >> 172.21.15.103:0/2247809482 conn(0x7f46900fd5f0 msgr2=0x7f46900ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 -- 172.21.15.103:0/2247809482 shutdown_connections 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 -- 172.21.15.103:0/2247809482 wait complete. 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 Processor -- start 2024-08-19T01:33:39.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 -- start start 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46901018b0 0x7f4690198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4690102250 0x7f46901986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46901078b0 0x7f469019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4690111ba0 con 0x7f4690102250 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4690111a20 con 0x7f46901078b0 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4695fd2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4690111d20 con 0x7f46901018b0 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4690102250 0x7f46901986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4690102250 0x7f46901986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50814/0 (socket says 172.21.15.103:50814) 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.957+0000 7f4687fff640 1 -- 172.21.15.103:0/482428888 learned_addr learned my addr 172.21.15.103:0/482428888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:39.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f468f7fe640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46901018b0 0x7f4690198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f468ffff640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46901078b0 0x7f469019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4687fff640 1 -- 172.21.15.103:0/482428888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46901018b0 msgr2=0x7f4690198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4687fff640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46901018b0 0x7f4690198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4687fff640 1 -- 172.21.15.103:0/482428888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46901078b0 msgr2=0x7f469019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4687fff640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46901078b0 0x7f469019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4687fff640 1 -- 172.21.15.103:0/482428888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019d130 con 0x7f4690102250 2024-08-19T01:33:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4687fff640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4690102250 0x7f46901986c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4678005940 tx=0x7f46780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f468d7fa640 1 -- 172.21.15.103:0/482428888 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467803f030 con 0x7f4690102250 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469019d3c0 con 0x7f4690102250 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f468d7fa640 1 -- 172.21.15.103:0/482428888 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46780318a0 con 0x7f4690102250 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f468d7fa640 1 -- 172.21.15.103:0/482428888 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678035070 con 0x7f4690102250 2024-08-19T01:33:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.958+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a4d00 con 0x7f4690102250 2024-08-19T01:33:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.959+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4690101d10 con 0x7f4690102250 2024-08-19T01:33:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.960+0000 7f468d7fa640 1 -- 172.21.15.103:0/482428888 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4678042030 con 0x7f4690102250 2024-08-19T01:33:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.961+0000 7f468d7fa640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f467c0a2c90 0x7f467c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.961+0000 7f468f7fe640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f467c0a2c90 0x7f467c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.961+0000 7f468d7fa640 1 -- 172.21.15.103:0/482428888 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f46780ef9d0 con 0x7f4690102250 2024-08-19T01:33:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.962+0000 7f468f7fe640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f467c0a2c90 0x7f467c0a5150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4680009bd0 tx=0x7f4680005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:39.964+0000 7f468d7fa640 1 -- 172.21.15.103:0/482428888 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f46780bc980 con 0x7f4690102250 2024-08-19T01:33:40.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.158+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4690062360 con 0x7f467c0a2c90 2024-08-19T01:33:40.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.163+0000 7f468d7fa640 1 -- 172.21.15.103:0/482428888 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4690062360 con 0x7f467c0a2c90 2024-08-19T01:33:40.165 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_0/a1d53bc3-7e67-4c1d-889f-ad0027be0cfc 2024-08-19T01:33:40.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f467c0a2c90 msgr2=0x7f467c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:40.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f467c0a2c90 0x7f467c0a5150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4680009bd0 tx=0x7f4680005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:40.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4690102250 msgr2=0x7f46901986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4690102250 0x7f46901986c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4678005940 tx=0x7f46780042f0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 shutdown_connections 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f467c0a2c90 0x7f467c0a5150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46901078b0 0x7f469019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4690102250 0x7f46901986c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 --2- 172.21.15.103:0/482428888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46901018b0 0x7f4690198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 >> 172.21.15.103:0/482428888 conn(0x7f46900fd5f0 msgr2=0x7f46900fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 shutdown_connections 2024-08-19T01:33:40.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:40.165+0000 7f4695fd2640 1 -- 172.21.15.103:0/482428888 wait complete. 2024-08-19T01:33:40.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-19T01:33:41.091 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 -- 172.21.15.103:0/1853683880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 msgr2=0x7f85c8100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/1853683880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8100f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009900 tx=0x7f85b802a480 comp rx=0 tx=0).stop 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 -- 172.21.15.103:0/1853683880 shutdown_connections 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/1853683880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85c8102c20 0x7f85c810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/1853683880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c81022a0 0x7f85c81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/1853683880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8100f30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 -- 172.21.15.103:0/1853683880 >> 172.21.15.103:0/1853683880 conn(0x7f85c80fc500 msgr2=0x7f85c80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 -- 172.21.15.103:0/1853683880 shutdown_connections 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.598+0000 7f85cd2f6640 1 -- 172.21.15.103:0/1853683880 wait complete. 2024-08-19T01:33:41.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 Processor -- start 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 -- start start 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85c81022a0 0x7f85c81994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8102c20 0x7f85c81a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8110b80 con 0x7f85c8100b50 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8110a00 con 0x7f85c8102c20 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85cd2f6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8110d00 con 0x7f85c81022a0 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50838/0 (socket says 172.21.15.103:50838) 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85c6ffd640 1 -- 172.21.15.103:0/3997556656 learned_addr learned my addr 172.21.15.103:0/3997556656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85c77fe640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8102c20 0x7f85c81a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.599+0000 7f85be7fc640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85c81022a0 0x7f85c81994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c6ffd640 1 -- 172.21.15.103:0/3997556656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85c81022a0 msgr2=0x7f85c81994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c6ffd640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85c81022a0 0x7f85c81994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c6ffd640 1 -- 172.21.15.103:0/3997556656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8102c20 msgr2=0x7f85c81a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c6ffd640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8102c20 0x7f85c81a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c6ffd640 1 -- 172.21.15.103:0/3997556656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c81a4530 con 0x7f85c8100b50 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c77fe640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8102c20 0x7f85c81a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85be7fc640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85c81022a0 0x7f85c81994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c6ffd640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8198f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006010 tx=0x7f85b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c4ff9640 1 -- 172.21.15.103:0/3997556656 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b803f070 con 0x7f85c8100b50 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c81a4700 con 0x7f85c8100b50 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c4ff9640 1 -- 172.21.15.103:0/3997556656 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85b8003aa0 con 0x7f85c8100b50 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85c4ff9640 1 -- 172.21.15.103:0/3997556656 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8035030 con 0x7f85c8100b50 2024-08-19T01:33:41.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.600+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c81a4b50 con 0x7f85c8100b50 2024-08-19T01:33:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.601+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c8100f30 con 0x7f85c8100b50 2024-08-19T01:33:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.602+0000 7f85c4ff9640 1 -- 172.21.15.103:0/3997556656 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f85b8042070 con 0x7f85c8100b50 2024-08-19T01:33:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.603+0000 7f85c4ff9640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f85b40a2c10 0x7f85b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.603+0000 7f85be7fc640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f85b40a2c10 0x7f85b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.603+0000 7f85c4ff9640 1 -- 172.21.15.103:0/3997556656 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f85b80ef680 con 0x7f85c8100b50 2024-08-19T01:33:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.603+0000 7f85be7fc640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f85b40a2c10 0x7f85b40a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85c806a7f0 tx=0x7f85b0005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:41.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.607+0000 7f85c4ff9640 1 -- 172.21.15.103:0/3997556656 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f85b80bc630 con 0x7f85c8100b50 2024-08-19T01:33:41.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.799+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f85c8108e30 con 0x7f85b40a2c10 2024-08-19T01:33:41.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.824+0000 7f85c4ff9640 1 -- 172.21.15.103:0/3997556656 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85b000da80 con 0x7f85b40a2c10 2024-08-19T01:33:41.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f85b40a2c10 msgr2=0x7f85b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f85b40a2c10 0x7f85b40a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85c806a7f0 tx=0x7f85b0005d90 comp rx=0 tx=0).stop 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 msgr2=0x7f85c8198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8198f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006010 tx=0x7f85b80042f0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 shutdown_connections 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f85b40a2c10 0x7f85b40a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8102c20 0x7f85c81a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85c81022a0 0x7f85c81994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 --2- 172.21.15.103:0/3997556656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85c8100b50 0x7f85c8198f70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 >> 172.21.15.103:0/3997556656 conn(0x7f85c80fc500 msgr2=0x7f85c80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 shutdown_connections 2024-08-19T01:33:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:41.826+0000 7f85cd2f6640 1 -- 172.21.15.103:0/3997556656 wait complete. 2024-08-19T01:33:42.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:41 smithi103 ceph-mon[18436]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:42.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:41 smithi103 ceph-mon[18436]: pgmap v274: 97 pgs: 97 active+clean; 592 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.1 KiB/s wr, 19 op/s 2024-08-19T01:33:42.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:41 smithi190 ceph-mon[22032]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:42.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:41 smithi190 ceph-mon[22032]: pgmap v274: 97 pgs: 97 active+clean; 592 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.1 KiB/s wr, 19 op/s 2024-08-19T01:33:42.154 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:41 smithi146 ceph-mon[22098]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:42.154 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:41 smithi146 ceph-mon[22098]: pgmap v274: 97 pgs: 97 active+clean; 592 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.1 KiB/s wr, 19 op/s 2024-08-19T01:33:42.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-19T01:33:42.701 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:43.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:42 smithi103 ceph-mon[18436]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:43.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:42 smithi190 ceph-mon[22032]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:43.167 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:42 smithi146 ceph-mon[22098]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 -- 172.21.15.103:0/2541770963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 msgr2=0x7f2f1c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 --2- 172.21.15.103:0/2541770963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c100c10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009930 tx=0x7f2f1802a420 comp rx=0 tx=0).stop 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 -- 172.21.15.103:0/2541770963 shutdown_connections 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 --2- 172.21.15.103:0/2541770963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1c101150 0x7f2f1c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 --2- 172.21.15.103:0/2541770963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c100c10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 --2- 172.21.15.103:0/2541770963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f1c1067b0 0x7f2f1c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 -- 172.21.15.103:0/2541770963 >> 172.21.15.103:0/2541770963 conn(0x7f2f1c0fc4d0 msgr2=0x7f2f1c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 -- 172.21.15.103:0/2541770963 shutdown_connections 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.191+0000 7f2f2452b640 1 -- 172.21.15.103:0/2541770963 wait complete. 2024-08-19T01:33:43.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 Processor -- start 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 -- start start 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c0fca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f1c101150 0x7f2f1c0fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1c1067b0 0x7f2f1c0fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c110ad0 con 0x7f2f1c1007b0 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c110950 con 0x7f2f1c101150 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f2452b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c110c50 con 0x7f2f1c1067b0 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c0fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c0fca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58876/0 (socket says 172.21.15.103:58876) 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 -- 172.21.15.103:0/856314050 learned_addr learned my addr 172.21.15.103:0/856314050 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f22ac8640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1c1067b0 0x7f2f1c0fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f21ac6640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f1c101150 0x7f2f1c0fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 -- 172.21.15.103:0/856314050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1c1067b0 msgr2=0x7f2f1c0fd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:43.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1c1067b0 0x7f2f1c0fd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 -- 172.21.15.103:0/856314050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f1c101150 msgr2=0x7f2f1c0fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f1c101150 0x7f2f1c0fcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 -- 172.21.15.103:0/856314050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c1a6530 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f222c7640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c0fca20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c920 tx=0x7f2f1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.192+0000 7f2f0f7fe640 1 -- 172.21.15.103:0/856314050 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10007c70 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.193+0000 7f2f0f7fe640 1 -- 172.21.15.103:0/856314050 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f10007e10 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.193+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1c1a6820 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.193+0000 7f2f0f7fe640 1 -- 172.21.15.103:0/856314050 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10018820 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.193+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1c1a6d30 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.194+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1c100c10 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.195+0000 7f2f0f7fe640 1 -- 172.21.15.103:0/856314050 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2f100189c0 con 0x7f2f1c1007b0 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.196+0000 7f2f0f7fe640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f080a2c10 0x7f2f080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:43.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.196+0000 7f2f21ac6640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f080a2c10 0x7f2f080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:43.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.196+0000 7f2f0f7fe640 1 -- 172.21.15.103:0/856314050 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f2f100cf240 con 0x7f2f1c1007b0 2024-08-19T01:33:43.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.196+0000 7f2f21ac6640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f080a2c10 0x7f2f080a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f18006010 tx=0x7f2f18031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:43.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.200+0000 7f2f0f7fe640 1 -- 172.21.15.103:0/856314050 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2f10014030 con 0x7f2f1c1007b0 2024-08-19T01:33:43.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.393+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2f1c062360 con 0x7f2f080a2c10 2024-08-19T01:33:43.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.398+0000 7f2f0f7fe640 1 -- 172.21.15.103:0/856314050 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2f1c062360 con 0x7f2f080a2c10 2024-08-19T01:33:43.400 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_1/997ca262-9d41-4b68-bb77-7f0a16e5700a 2024-08-19T01:33:43.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f080a2c10 msgr2=0x7f2f080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f080a2c10 0x7f2f080a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f18006010 tx=0x7f2f18031400 comp rx=0 tx=0).stop 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 msgr2=0x7f2f1c0fca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c0fca20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c920 tx=0x7f2f1000cde0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 shutdown_connections 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f080a2c10 0x7f2f080a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f1c1067b0 0x7f2f1c0fd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f1c101150 0x7f2f1c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 --2- 172.21.15.103:0/856314050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f1c1007b0 0x7f2f1c0fca20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 >> 172.21.15.103:0/856314050 conn(0x7f2f1c0fc4d0 msgr2=0x7f2f1c06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 shutdown_connections 2024-08-19T01:33:43.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:43.400+0000 7f2f2452b640 1 -- 172.21.15.103:0/856314050 wait complete. 2024-08-19T01:33:44.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:43 smithi103 ceph-mon[18436]: pgmap v275: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.1 KiB/s wr, 23 op/s 2024-08-19T01:33:44.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:43 smithi190 ceph-mon[22032]: pgmap v275: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.1 KiB/s wr, 23 op/s 2024-08-19T01:33:44.134 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:33:44.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:43 smithi146 ceph-mon[22098]: pgmap v275: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.1 KiB/s wr, 23 op/s 2024-08-19T01:33:44.387 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:44.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 -- 172.21.15.103:0/2785780808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e201078b0 msgr2=0x7f7e20107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 --2- 172.21.15.103:0/2785780808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e201078b0 0x7f7e20107c90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009930 tx=0x7f7e1002a420 comp rx=0 tx=0).stop 2024-08-19T01:33:44.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 -- 172.21.15.103:0/2785780808 shutdown_connections 2024-08-19T01:33:44.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 --2- 172.21.15.103:0/2785780808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e20102250 0x7f7e2010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 --2- 172.21.15.103:0/2785780808 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e201018b0 0x7f7e20101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 --2- 172.21.15.103:0/2785780808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e201078b0 0x7f7e20107c90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 -- 172.21.15.103:0/2785780808 >> 172.21.15.103:0/2785780808 conn(0x7f7e200fd5f0 msgr2=0x7f7e200ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 -- 172.21.15.103:0/2785780808 shutdown_connections 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.889+0000 7f7e264fa640 1 -- 172.21.15.103:0/2785780808 wait complete. 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 Processor -- start 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 -- start start 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e201018b0 0x7f7e20198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e20102250 0x7f7e201986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e201078b0 0x7f7e2019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20111b80 con 0x7f7e20102250 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20111a00 con 0x7f7e201078b0 2024-08-19T01:33:44.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e264fa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20111d00 con 0x7f7e201018b0 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.890+0000 7f7e1f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e20102250 0x7f7e201986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e20102250 0x7f7e201986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58896/0 (socket says 172.21.15.103:58896) 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 -- 172.21.15.103:0/1941099405 learned_addr learned my addr 172.21.15.103:0/1941099405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1ffff640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e201018b0 0x7f7e20198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e24a97640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e201078b0 0x7f7e2019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 -- 172.21.15.103:0/1941099405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e201018b0 msgr2=0x7f7e20198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e201018b0 0x7f7e20198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 -- 172.21.15.103:0/1941099405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e201078b0 msgr2=0x7f7e2019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e201078b0 0x7f7e2019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 -- 172.21.15.103:0/1941099405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2019d130 con 0x7f7e20102250 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1f7fe640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e20102250 0x7f7e201986c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b4a0 tx=0x7f7e1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1d7fa640 1 -- 172.21.15.103:0/1941099405 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14014070 con 0x7f7e20102250 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1d7fa640 1 -- 172.21.15.103:0/1941099405 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e14010ac0 con 0x7f7e20102250 2024-08-19T01:33:44.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2019d420 con 0x7f7e20102250 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e1d7fa640 1 -- 172.21.15.103:0/1941099405 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14017c30 con 0x7f7e20102250 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.891+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e201a4d00 con 0x7f7e20102250 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.892+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e20101d10 con 0x7f7e20102250 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.893+0000 7f7e1d7fa640 1 -- 172.21.15.103:0/1941099405 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7e14017070 con 0x7f7e20102250 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.894+0000 7f7e1d7fa640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7e080a2c90 0x7f7e080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.894+0000 7f7e1ffff640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7e080a2c90 0x7f7e080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.894+0000 7f7e1d7fa640 1 -- 172.21.15.103:0/1941099405 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f7e140ce690 con 0x7f7e20102250 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.895+0000 7f7e1ffff640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7e080a2c90 0x7f7e080a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009900 tx=0x7f7e10000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:44.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:44.897+0000 7f7e1d7fa640 1 -- 172.21.15.103:0/1941099405 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7e1409b000 con 0x7f7e20102250 2024-08-19T01:33:45.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.100+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e201994e0 con 0x7f7e20102250 2024-08-19T01:33:45.103 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:44 smithi103 ceph-mon[18436]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:45.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.101+0000 7f7e1d7fa640 1 -- 172.21.15.103:0/1941099405 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11989 (secure 0 0 0) 0x7f7e201994e0 con 0x7f7e20102250 2024-08-19T01:33:45.103 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:45.103 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-08-19T01:33:34:962864+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:33:34.962843+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:33:45.104 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-08-19T01:33:45.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7e080a2c90 msgr2=0x7f7e080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7e080a2c90 0x7f7e080a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009900 tx=0x7f7e10000bc0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e20102250 msgr2=0x7f7e201986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e20102250 0x7f7e201986c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b4a0 tx=0x7f7e1400b960 comp rx=0 tx=0).stop 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 shutdown_connections 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7e080a2c90 0x7f7e080a5150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e201078b0 0x7f7e2019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e20102250 0x7f7e201986c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.103+0000 7f7e264fa640 1 --2- 172.21.15.103:0/1941099405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e201018b0 0x7f7e20198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.104+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 >> 172.21.15.103:0/1941099405 conn(0x7f7e200fd5f0 msgr2=0x7f7e200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.104+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 shutdown_connections 2024-08-19T01:33:45.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:45.104+0000 7f7e264fa640 1 -- 172.21.15.103:0/1941099405 wait complete. 2024-08-19T01:33:45.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:44 smithi190 ceph-mon[22032]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:45.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:44 smithi146 ceph-mon[22098]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:33:45.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:46.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:45 smithi190 ceph-mon[22032]: pgmap v276: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-08-19T01:33:46.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:45 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1941099405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:45 smithi146 ceph-mon[22098]: pgmap v276: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-08-19T01:33:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:45 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1941099405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:46.231 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:46.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:45 smithi103 ceph-mon[18436]: pgmap v276: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.6 KiB/s wr, 21 op/s 2024-08-19T01:33:46.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:45 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1941099405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:33:46.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.730+0000 7f52c4cd7640 1 -- 172.21.15.103:0/65288600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c01033a0 msgr2=0x7f52c010b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.730+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/65288600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c01033a0 0x7f52c010b890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009900 tx=0x7f52b402a480 comp rx=0 tx=0).stop 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.730+0000 7f52c4cd7640 1 -- 172.21.15.103:0/65288600 shutdown_connections 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.730+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/65288600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c010beb0 0x7f52c010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.730+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/65288600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c01033a0 0x7f52c010b890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.730+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/65288600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52c0102a80 0x7f52c0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.730+0000 7f52c4cd7640 1 -- 172.21.15.103:0/65288600 >> 172.21.15.103:0/65288600 conn(0x7f52c00fc500 msgr2=0x7f52c00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.731+0000 7f52c4cd7640 1 -- 172.21.15.103:0/65288600 shutdown_connections 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.731+0000 7f52c4cd7640 1 -- 172.21.15.103:0/65288600 wait complete. 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.731+0000 7f52c4cd7640 1 Processor -- start 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.731+0000 7f52c4cd7640 1 -- start start 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.731+0000 7f52c4cd7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c0102a80 0x7f52c01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.731+0000 7f52c4cd7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c01033a0 0x7f52c0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.731+0000 7f52c4cd7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52c010beb0 0x7f52c019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52c4cd7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0111ba0 con 0x7f52c0102a80 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52c4cd7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0111a20 con 0x7f52c01033a0 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52c4cd7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0111d20 con 0x7f52c010beb0 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c0102a80 0x7f52c01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c0102a80 0x7f52c01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58918/0 (socket says 172.21.15.103:58918) 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 -- 172.21.15.103:0/1848201817 learned_addr learned my addr 172.21.15.103:0/1848201817 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52bed9d640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52c010beb0 0x7f52c019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52bdd9b640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c01033a0 0x7f52c0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 -- 172.21.15.103:0/1848201817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52c010beb0 msgr2=0x7f52c019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52c010beb0 0x7f52c019c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 -- 172.21.15.103:0/1848201817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c01033a0 msgr2=0x7f52c0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c01033a0 0x7f52c0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 -- 172.21.15.103:0/1848201817 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c019d090 con 0x7f52c0102a80 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52be59c640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c0102a80 0x7f52c01980e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52b000b4a0 tx=0x7f52b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52af7fe640 1 -- 172.21.15.103:0/1848201817 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0014070 con 0x7f52c0102a80 2024-08-19T01:33:46.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c019d380 con 0x7f52c0102a80 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52af7fe640 1 -- 172.21.15.103:0/1848201817 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b0010ac0 con 0x7f52c0102a80 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.732+0000 7f52af7fe640 1 -- 172.21.15.103:0/1848201817 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0017c30 con 0x7f52c0102a80 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.733+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c01a4c60 con 0x7f52c0102a80 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.733+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c0102e60 con 0x7f52c0102a80 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.735+0000 7f52af7fe640 1 -- 172.21.15.103:0/1848201817 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f52b0017070 con 0x7f52c0102a80 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.735+0000 7f52af7fe640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52a80a2c10 0x7f52a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.735+0000 7f52bdd9b640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52a80a2c10 0x7f52a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.736+0000 7f52af7fe640 1 -- 172.21.15.103:0/1848201817 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f52b00ce250 con 0x7f52c0102a80 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.736+0000 7f52bdd9b640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52a80a2c10 0x7f52a80a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52b4031440 tx=0x7f52b4005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:46.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.740+0000 7f52af7fe640 1 -- 172.21.15.103:0/1848201817 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f52b009acc0 con 0x7f52c0102a80 2024-08-19T01:33:46.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.909+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52c0106f20 con 0x7f52c0102a80 2024-08-19T01:33:46.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.910+0000 7f52af7fe640 1 -- 172.21.15.103:0/1848201817 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22301 (secure 0 0 0) 0x7f52b009efc0 con 0x7f52c0102a80 2024-08-19T01:33:46.912 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:46.913 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:33:23.991024+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52a80a2c10 msgr2=0x7f52a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52a80a2c10 0x7f52a80a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52b4031440 tx=0x7f52b4005e80 comp rx=0 tx=0).stop 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c0102a80 msgr2=0x7f52c01980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c0102a80 0x7f52c01980e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52b000b4a0 tx=0x7f52b000b960 comp rx=0 tx=0).stop 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 shutdown_connections 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52a80a2c10 0x7f52a80a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52c010beb0 0x7f52c019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f52c01033a0 0x7f52c0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 --2- 172.21.15.103:0/1848201817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52c0102a80 0x7f52c01980e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.912+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 >> 172.21.15.103:0/1848201817 conn(0x7f52c00fc500 msgr2=0x7f52c00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.913+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 shutdown_connections 2024-08-19T01:33:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:46.913+0000 7f52c4cd7640 1 -- 172.21.15.103:0/1848201817 wait complete. 2024-08-19T01:33:47.612 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-19T01:33:47.613 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:33:47.862 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:48.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:47 smithi103 ceph-mon[18436]: pgmap v277: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-08-19T01:33:48.127 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:47 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1848201817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:48.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:47 smithi190 ceph-mon[22032]: pgmap v277: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-08-19T01:33:48.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:47 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1848201817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:47 smithi146 ceph-mon[22098]: pgmap v277: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-08-19T01:33:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:47 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1848201817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:48.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 -- 172.21.15.103:0/162129438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 msgr2=0x7eff4c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 --2- 172.21.15.103:0/162129438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c10d770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7eff44009ca0 tx=0x7eff4402a950 comp rx=0 tx=0).stop 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 -- 172.21.15.103:0/162129438 shutdown_connections 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 --2- 172.21.15.103:0/162129438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c10d770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 --2- 172.21.15.103:0/162129438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c1022a0 0x7eff4c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 --2- 172.21.15.103:0/162129438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff4c100b50 0x7eff4c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 -- 172.21.15.103:0/162129438 >> 172.21.15.103:0/162129438 conn(0x7eff4c0fc500 msgr2=0x7eff4c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 -- 172.21.15.103:0/162129438 shutdown_connections 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 -- 172.21.15.103:0/162129438 wait complete. 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 Processor -- start 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.341+0000 7eff52d45640 1 -- start start 2024-08-19T01:33:48.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff52d45640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c100b50 0x7eff4c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff52d45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff4c1022a0 0x7eff4c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff52d45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff52d45640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c110b60 con 0x7eff4c102c20 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff52d45640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1109e0 con 0x7eff4c100b50 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff52d45640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c110ce0 con 0x7eff4c1022a0 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58938/0 (socket says 172.21.15.103:58938) 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 -- 172.21.15.103:0/986259901 learned_addr learned my addr 172.21.15.103:0/986259901 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff50ae1640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c100b50 0x7eff4c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 -- 172.21.15.103:0/986259901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff4c1022a0 msgr2=0x7eff4c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff4c1022a0 0x7eff4c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 -- 172.21.15.103:0/986259901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c100b50 msgr2=0x7eff4c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:48.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c100b50 0x7eff4c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 -- 172.21.15.103:0/986259901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c19d000 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff50ae1640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c100b50 0x7eff4c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff512e2640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c19c920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7eff44006010 tx=0x7eff4402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff49ffb640 1 -- 172.21.15.103:0/986259901 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44034040 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff49ffb640 1 -- 172.21.15.103:0/986259901 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff44003e00 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c19d290 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.342+0000 7eff49ffb640 1 -- 172.21.15.103:0/986259901 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44045480 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.343+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c1a4bd0 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.344+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4c101ee0 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.345+0000 7eff49ffb640 1 -- 172.21.15.103:0/986259901 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7eff44037020 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.345+0000 7eff49ffb640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7eff3c0a2c10 0x7eff3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.345+0000 7eff50ae1640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7eff3c0a2c10 0x7eff3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.346+0000 7eff49ffb640 1 -- 172.21.15.103:0/986259901 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7eff440f0120 con 0x7eff4c102c20 2024-08-19T01:33:48.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.346+0000 7eff50ae1640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7eff3c0a2c10 0x7eff3c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff400058c0 tx=0x7eff40005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:48.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.349+0000 7eff49ffb640 1 -- 172.21.15.103:0/986259901 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7eff440bd1e0 con 0x7eff4c102c20 2024-08-19T01:33:48.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.510+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff4c109e40 con 0x7eff3c0a2c10 2024-08-19T01:33:48.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.517+0000 7eff49ffb640 1 -- 172.21.15.103:0/986259901 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7eff4000b6b0 con 0x7eff3c0a2c10 2024-08-19T01:33:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:48.523 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7eff3c0a2c10 msgr2=0x7eff3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7eff3c0a2c10 0x7eff3c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff400058c0 tx=0x7eff40005d30 comp rx=0 tx=0).stop 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 msgr2=0x7eff4c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c19c920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7eff44006010 tx=0x7eff4402f710 comp rx=0 tx=0).stop 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 shutdown_connections 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7eff3c0a2c10 0x7eff3c0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff4c102c20 0x7eff4c19c920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff4c1022a0 0x7eff4c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 --2- 172.21.15.103:0/986259901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c100b50 0x7eff4c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:48.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 >> 172.21.15.103:0/986259901 conn(0x7eff4c0fc500 msgr2=0x7eff4c069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:48.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 shutdown_connections 2024-08-19T01:33:48.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:48.521+0000 7eff52d45640 1 -- 172.21.15.103:0/986259901 wait complete. 2024-08-19T01:33:49.239 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-08-19T01:33:46.776375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639256,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":248,"num_read_kb":207,"num_write":412,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360352,"kb_used_data":11200,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581472,"statfs":{"total":1151940427776,"available":1151571427328,"internally_reserved":0,"allocated":11468800,"data_stored":7132296,"data_compressed":107457,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44394,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":122,"num_read_kb":98,"num_write":133,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003064"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.190579+0000","last_change":"2024-08-19T01:33:23.202035+0000","last_active":"2024-08-19T01:33:24.190579+0000","last_peered":"2024-08-19T01:33:24.190579+0000","last_clean":"2024-08-19T01:33:24.190579+0000","last_became_active":"2024-08-19T01:33:23.201418+0000","last_became_peered":"2024-08-19T01:33:23.201418+0000","last_unstale":"2024-08-19T01:33:24.190579+0000","last_undegraded":"2024-08-19T01:33:24.190579+0000","last_fullsized":"2024-08-19T01:33:24.190579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:54:57.927424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184681+0000","last_change":"2024-08-19T01:33:23.202047+0000","last_active":"2024-08-19T01:33:24.184681+0000","last_peered":"2024-08-19T01:33:24.184681+0000","last_clean":"2024-08-19T01:33:24.184681+0000","last_became_active":"2024-08-19T01:33:23.201762+0000","last_became_peered":"2024-08-19T01:33:23.201762+0000","last_unstale":"2024-08-19T01:33:24.184681+0000","last_undegraded":"2024-08-19T01:33:24.184681+0000","last_fullsized":"2024-08-19T01:33:24.184681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:23:57.252284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184897+0000","last_change":"2024-08-19T01:33:23.202876+0000","last_active":"2024-08-19T01:33:24.184897+0000","last_peered":"2024-08-19T01:33:24.184897+0000","last_clean":"2024-08-19T01:33:24.184897+0000","last_became_active":"2024-08-19T01:33:23.202528+0000","last_became_peered":"2024-08-19T01:33:23.202528+0000","last_unstale":"2024-08-19T01:33:24.184897+0000","last_undegraded":"2024-08-19T01:33:24.184897+0000","last_fullsized":"2024-08-19T01:33:24.184897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:20.514390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185874+0000","last_change":"2024-08-19T01:33:23.199443+0000","last_active":"2024-08-19T01:33:24.185874+0000","last_peered":"2024-08-19T01:33:24.185874+0000","last_clean":"2024-08-19T01:33:24.185874+0000","last_became_active":"2024-08-19T01:33:23.198344+0000","last_became_peered":"2024-08-19T01:33:23.198344+0000","last_unstale":"2024-08-19T01:33:24.185874+0000","last_undegraded":"2024-08-19T01:33:24.185874+0000","last_fullsized":"2024-08-19T01:33:24.185874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:43:43.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183932+0000","last_change":"2024-08-19T01:33:23.191185+0000","last_active":"2024-08-19T01:33:24.183932+0000","last_peered":"2024-08-19T01:33:24.183932+0000","last_clean":"2024-08-19T01:33:24.183932+0000","last_became_active":"2024-08-19T01:33:23.190892+0000","last_became_peered":"2024-08-19T01:33:23.190892+0000","last_unstale":"2024-08-19T01:33:24.183932+0000","last_undegraded":"2024-08-19T01:33:24.183932+0000","last_fullsized":"2024-08-19T01:33:24.183932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:28.384609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.177164+0000","last_change":"2024-08-19T01:33:23.193036+0000","last_active":"2024-08-19T01:33:24.177164+0000","last_peered":"2024-08-19T01:33:24.177164+0000","last_clean":"2024-08-19T01:33:24.177164+0000","last_became_active":"2024-08-19T01:33:23.192686+0000","last_became_peered":"2024-08-19T01:33:23.192686+0000","last_unstale":"2024-08-19T01:33:24.177164+0000","last_undegraded":"2024-08-19T01:33:24.177164+0000","last_fullsized":"2024-08-19T01:33:24.177164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:46.235112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185812+0000","last_change":"2024-08-19T01:33:23.197636+0000","last_active":"2024-08-19T01:33:24.185812+0000","last_peered":"2024-08-19T01:33:24.185812+0000","last_clean":"2024-08-19T01:33:24.185812+0000","last_became_active":"2024-08-19T01:33:23.197323+0000","last_became_peered":"2024-08-19T01:33:23.197323+0000","last_unstale":"2024-08-19T01:33:24.185812+0000","last_undegraded":"2024-08-19T01:33:24.185812+0000","last_fullsized":"2024-08-19T01:33:24.185812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:46:36.168786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182499+0000","last_change":"2024-08-19T01:33:23.193717+0000","last_active":"2024-08-19T01:33:24.182499+0000","last_peered":"2024-08-19T01:33:24.182499+0000","last_clean":"2024-08-19T01:33:24.182499+0000","last_became_active":"2024-08-19T01:33:23.193376+0000","last_became_peered":"2024-08-19T01:33:23.193376+0000","last_unstale":"2024-08-19T01:33:24.182499+0000","last_undegraded":"2024-08-19T01:33:24.182499+0000","last_fullsized":"2024-08-19T01:33:24.182499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:16.290081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183386+0000","last_change":"2024-08-19T01:33:23.197956+0000","last_active":"2024-08-19T01:33:24.183386+0000","last_peered":"2024-08-19T01:33:24.183386+0000","last_clean":"2024-08-19T01:33:24.183386+0000","last_became_active":"2024-08-19T01:33:23.197667+0000","last_became_peered":"2024-08-19T01:33:23.197667+0000","last_unstale":"2024-08-19T01:33:24.183386+0000","last_undegraded":"2024-08-19T01:33:24.183386+0000","last_fullsized":"2024-08-19T01:33:24.183386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:16.841046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.177654+0000","last_change":"2024-08-19T01:33:23.194569+0000","last_active":"2024-08-19T01:33:24.177654+0000","last_peered":"2024-08-19T01:33:24.177654+0000","last_clean":"2024-08-19T01:33:24.177654+0000","last_became_active":"2024-08-19T01:33:23.194223+0000","last_became_peered":"2024-08-19T01:33:23.194223+0000","last_unstale":"2024-08-19T01:33:24.177654+0000","last_undegraded":"2024-08-19T01:33:24.177654+0000","last_fullsized":"2024-08-19T01:33:24.177654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:14:30.184738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186285+0000","last_change":"2024-08-19T01:33:23.202396+0000","last_active":"2024-08-19T01:33:24.186285+0000","last_peered":"2024-08-19T01:33:24.186285+0000","last_clean":"2024-08-19T01:33:24.186285+0000","last_became_active":"2024-08-19T01:33:23.202063+0000","last_became_peered":"2024-08-19T01:33:23.202063+0000","last_unstale":"2024-08-19T01:33:24.186285+0000","last_undegraded":"2024-08-19T01:33:24.186285+0000","last_fullsized":"2024-08-19T01:33:24.186285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:57.513639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185543+0000","last_change":"2024-08-19T01:33:23.200610+0000","last_active":"2024-08-19T01:33:24.185543+0000","last_peered":"2024-08-19T01:33:24.185543+0000","last_clean":"2024-08-19T01:33:24.185543+0000","last_became_active":"2024-08-19T01:33:23.200136+0000","last_became_peered":"2024-08-19T01:33:23.200136+0000","last_unstale":"2024-08-19T01:33:24.185543+0000","last_undegraded":"2024-08-19T01:33:24.185543+0000","last_fullsized":"2024-08-19T01:33:24.185543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:47:39.252702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174693+0000","last_change":"2024-08-19T01:33:23.196062+0000","last_active":"2024-08-19T01:33:24.174693+0000","last_peered":"2024-08-19T01:33:24.174693+0000","last_clean":"2024-08-19T01:33:24.174693+0000","last_became_active":"2024-08-19T01:33:23.195577+0000","last_became_peered":"2024-08-19T01:33:23.195577+0000","last_unstale":"2024-08-19T01:33:24.174693+0000","last_undegraded":"2024-08-19T01:33:24.174693+0000","last_fullsized":"2024-08-19T01:33:24.174693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:46:09.111323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184440+0000","last_change":"2024-08-19T01:33:23.201664+0000","last_active":"2024-08-19T01:33:24.184440+0000","last_peered":"2024-08-19T01:33:24.184440+0000","last_clean":"2024-08-19T01:33:24.184440+0000","last_became_active":"2024-08-19T01:33:23.200241+0000","last_became_peered":"2024-08-19T01:33:23.200241+0000","last_unstale":"2024-08-19T01:33:24.184440+0000","last_undegraded":"2024-08-19T01:33:24.184440+0000","last_fullsized":"2024-08-19T01:33:24.184440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:17.199776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184303+0000","last_change":"2024-08-19T01:33:23.198102+0000","last_active":"2024-08-19T01:33:24.184303+0000","last_peered":"2024-08-19T01:33:24.184303+0000","last_clean":"2024-08-19T01:33:24.184303+0000","last_became_active":"2024-08-19T01:33:23.197778+0000","last_became_peered":"2024-08-19T01:33:23.197778+0000","last_unstale":"2024-08-19T01:33:24.184303+0000","last_undegraded":"2024-08-19T01:33:24.184303+0000","last_fullsized":"2024-08-19T01:33:24.184303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:03.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182240+0000","last_change":"2024-08-19T01:33:23.194581+0000","last_active":"2024-08-19T01:33:24.182240+0000","last_peered":"2024-08-19T01:33:24.182240+0000","last_clean":"2024-08-19T01:33:24.182240+0000","last_became_active":"2024-08-19T01:33:23.194256+0000","last_became_peered":"2024-08-19T01:33:23.194256+0000","last_unstale":"2024-08-19T01:33:24.182240+0000","last_undegraded":"2024-08-19T01:33:24.182240+0000","last_fullsized":"2024-08-19T01:33:24.182240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:17:20.954190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.191140+0000","last_change":"2024-08-19T01:33:23.201242+0000","last_active":"2024-08-19T01:33:24.191140+0000","last_peered":"2024-08-19T01:33:24.191140+0000","last_clean":"2024-08-19T01:33:24.191140+0000","last_became_active":"2024-08-19T01:33:23.200835+0000","last_became_peered":"2024-08-19T01:33:23.200835+0000","last_unstale":"2024-08-19T01:33:24.191140+0000","last_undegraded":"2024-08-19T01:33:24.191140+0000","last_fullsized":"2024-08-19T01:33:24.191140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:35:54.312266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185247+0000","last_change":"2024-08-19T01:33:23.187728+0000","last_active":"2024-08-19T01:33:24.185247+0000","last_peered":"2024-08-19T01:33:24.185247+0000","last_clean":"2024-08-19T01:33:24.185247+0000","last_became_active":"2024-08-19T01:33:23.187352+0000","last_became_peered":"2024-08-19T01:33:23.187352+0000","last_unstale":"2024-08-19T01:33:24.185247+0000","last_undegraded":"2024-08-19T01:33:24.185247+0000","last_fullsized":"2024-08-19T01:33:24.185247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:06:03.041328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186903+0000","last_change":"2024-08-19T01:33:23.193146+0000","last_active":"2024-08-19T01:33:24.186903+0000","last_peered":"2024-08-19T01:33:24.186903+0000","last_clean":"2024-08-19T01:33:24.186903+0000","last_became_active":"2024-08-19T01:33:23.192810+0000","last_became_peered":"2024-08-19T01:33:23.192810+0000","last_unstale":"2024-08-19T01:33:24.186903+0000","last_undegraded":"2024-08-19T01:33:24.186903+0000","last_fullsized":"2024-08-19T01:33:24.186903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:12.044175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.189491+0000","last_change":"2024-08-19T01:33:23.194635+0000","last_active":"2024-08-19T01:33:24.189491+0000","last_peered":"2024-08-19T01:33:24.189491+0000","last_clean":"2024-08-19T01:33:24.189491+0000","last_became_active":"2024-08-19T01:33:23.194308+0000","last_became_peered":"2024-08-19T01:33:23.194308+0000","last_unstale":"2024-08-19T01:33:24.189491+0000","last_undegraded":"2024-08-19T01:33:24.189491+0000","last_fullsized":"2024-08-19T01:33:24.189491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:12.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174804+0000","last_change":"2024-08-19T01:33:23.193108+0000","last_active":"2024-08-19T01:33:24.174804+0000","last_peered":"2024-08-19T01:33:24.174804+0000","last_clean":"2024-08-19T01:33:24.174804+0000","last_became_active":"2024-08-19T01:33:23.192830+0000","last_became_peered":"2024-08-19T01:33:23.192830+0000","last_unstale":"2024-08-19T01:33:24.174804+0000","last_undegraded":"2024-08-19T01:33:24.174804+0000","last_fullsized":"2024-08-19T01:33:24.174804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:33.355498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182362+0000","last_change":"2024-08-19T01:33:23.195789+0000","last_active":"2024-08-19T01:33:24.182362+0000","last_peered":"2024-08-19T01:33:24.182362+0000","last_clean":"2024-08-19T01:33:24.182362+0000","last_became_active":"2024-08-19T01:33:23.195064+0000","last_became_peered":"2024-08-19T01:33:23.195064+0000","last_unstale":"2024-08-19T01:33:24.182362+0000","last_undegraded":"2024-08-19T01:33:24.182362+0000","last_fullsized":"2024-08-19T01:33:24.182362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:05:33.848953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182747+0000","last_change":"2024-08-19T01:33:23.200968+0000","last_active":"2024-08-19T01:33:24.182747+0000","last_peered":"2024-08-19T01:33:24.182747+0000","last_clean":"2024-08-19T01:33:24.182747+0000","last_became_active":"2024-08-19T01:33:23.200606+0000","last_became_peered":"2024-08-19T01:33:23.200606+0000","last_unstale":"2024-08-19T01:33:24.182747+0000","last_undegraded":"2024-08-19T01:33:24.182747+0000","last_fullsized":"2024-08-19T01:33:24.182747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:44.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175504+0000","last_change":"2024-08-19T01:33:23.191204+0000","last_active":"2024-08-19T01:33:24.175504+0000","last_peered":"2024-08-19T01:33:24.175504+0000","last_clean":"2024-08-19T01:33:24.175504+0000","last_became_active":"2024-08-19T01:33:23.190709+0000","last_became_peered":"2024-08-19T01:33:23.190709+0000","last_unstale":"2024-08-19T01:33:24.175504+0000","last_undegraded":"2024-08-19T01:33:24.175504+0000","last_fullsized":"2024-08-19T01:33:24.175504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:56:10.988967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183164+0000","last_change":"2024-08-19T01:33:23.197168+0000","last_active":"2024-08-19T01:33:24.183164+0000","last_peered":"2024-08-19T01:33:24.183164+0000","last_clean":"2024-08-19T01:33:24.183164+0000","last_became_active":"2024-08-19T01:33:23.196883+0000","last_became_peered":"2024-08-19T01:33:23.196883+0000","last_unstale":"2024-08-19T01:33:24.183164+0000","last_undegraded":"2024-08-19T01:33:24.183164+0000","last_fullsized":"2024-08-19T01:33:24.183164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:33:35.386768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.187306+0000","last_change":"2024-08-19T01:33:23.187309+0000","last_active":"2024-08-19T01:33:24.187306+0000","last_peered":"2024-08-19T01:33:24.187306+0000","last_clean":"2024-08-19T01:33:24.187306+0000","last_became_active":"2024-08-19T01:33:23.186976+0000","last_became_peered":"2024-08-19T01:33:23.186976+0000","last_unstale":"2024-08-19T01:33:24.187306+0000","last_undegraded":"2024-08-19T01:33:24.187306+0000","last_fullsized":"2024-08-19T01:33:24.187306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:46:02.852000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.190181+0000","last_change":"2024-08-19T01:33:23.190292+0000","last_active":"2024-08-19T01:33:24.190181+0000","last_peered":"2024-08-19T01:33:24.190181+0000","last_clean":"2024-08-19T01:33:24.190181+0000","last_became_active":"2024-08-19T01:33:23.189969+0000","last_became_peered":"2024-08-19T01:33:23.189969+0000","last_unstale":"2024-08-19T01:33:24.190181+0000","last_undegraded":"2024-08-19T01:33:24.190181+0000","last_fullsized":"2024-08-19T01:33:24.190181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:35:51.388150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174739+0000","last_change":"2024-08-19T01:33:23.192148+0000","last_active":"2024-08-19T01:33:24.174739+0000","last_peered":"2024-08-19T01:33:24.174739+0000","last_clean":"2024-08-19T01:33:24.174739+0000","last_became_active":"2024-08-19T01:33:23.191845+0000","last_became_peered":"2024-08-19T01:33:23.191845+0000","last_unstale":"2024-08-19T01:33:24.174739+0000","last_undegraded":"2024-08-19T01:33:24.174739+0000","last_fullsized":"2024-08-19T01:33:24.174739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:51.308393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184626+0000","last_change":"2024-08-19T01:33:23.195033+0000","last_active":"2024-08-19T01:33:24.184626+0000","last_peered":"2024-08-19T01:33:24.184626+0000","last_clean":"2024-08-19T01:33:24.184626+0000","last_became_active":"2024-08-19T01:33:23.194745+0000","last_became_peered":"2024-08-19T01:33:23.194745+0000","last_unstale":"2024-08-19T01:33:24.184626+0000","last_undegraded":"2024-08-19T01:33:24.184626+0000","last_fullsized":"2024-08-19T01:33:24.184626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:33:40.748311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186518+0000","last_change":"2024-08-19T01:33:23.195817+0000","last_active":"2024-08-19T01:33:24.186518+0000","last_peered":"2024-08-19T01:33:24.186518+0000","last_clean":"2024-08-19T01:33:24.186518+0000","last_became_active":"2024-08-19T01:33:23.194688+0000","last_became_peered":"2024-08-19T01:33:23.194688+0000","last_unstale":"2024-08-19T01:33:24.186518+0000","last_undegraded":"2024-08-19T01:33:24.186518+0000","last_fullsized":"2024-08-19T01:33:24.186518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:13:04.913921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.187934+0000","last_change":"2024-08-19T01:33:23.187604+0000","last_active":"2024-08-19T01:33:24.187934+0000","last_peered":"2024-08-19T01:33:24.187934+0000","last_clean":"2024-08-19T01:33:24.187934+0000","last_became_active":"2024-08-19T01:33:23.187242+0000","last_became_peered":"2024-08-19T01:33:23.187242+0000","last_unstale":"2024-08-19T01:33:24.187934+0000","last_undegraded":"2024-08-19T01:33:24.187934+0000","last_fullsized":"2024-08-19T01:33:24.187934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:28.592525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.189956+0000","last_change":"2024-08-19T01:33:23.191737+0000","last_active":"2024-08-19T01:33:24.189956+0000","last_peered":"2024-08-19T01:33:24.189956+0000","last_clean":"2024-08-19T01:33:24.189956+0000","last_became_active":"2024-08-19T01:33:23.191358+0000","last_became_peered":"2024-08-19T01:33:23.191358+0000","last_unstale":"2024-08-19T01:33:24.189956+0000","last_undegraded":"2024-08-19T01:33:24.189956+0000","last_fullsized":"2024-08-19T01:33:24.189956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:28:14.694157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183641+0000","last_change":"2024-08-19T01:33:23.193110+0000","last_active":"2024-08-19T01:33:24.183641+0000","last_peered":"2024-08-19T01:33:24.183641+0000","last_clean":"2024-08-19T01:33:24.183641+0000","last_became_active":"2024-08-19T01:33:23.192789+0000","last_became_peered":"2024-08-19T01:33:23.192789+0000","last_unstale":"2024-08-19T01:33:24.183641+0000","last_undegraded":"2024-08-19T01:33:24.183641+0000","last_fullsized":"2024-08-19T01:33:24.183641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:45:48.141403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:40.983245+0000","last_change":"2024-08-19T01:33:21.173518+0000","last_active":"2024-08-19T01:33:40.983245+0000","last_peered":"2024-08-19T01:33:40.983245+0000","last_clean":"2024-08-19T01:33:40.983245+0000","last_became_active":"2024-08-19T01:33:21.172288+0000","last_became_peered":"2024-08-19T01:33:21.172288+0000","last_unstale":"2024-08-19T01:33:40.983245+0000","last_undegraded":"2024-08-19T01:33:40.983245+0000","last_fullsized":"2024-08-19T01:33:40.983245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:59:42.864724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175781+0000","last_change":"2024-08-19T01:33:23.190160+0000","last_active":"2024-08-19T01:33:24.175781+0000","last_peered":"2024-08-19T01:33:24.175781+0000","last_clean":"2024-08-19T01:33:24.175781+0000","last_became_active":"2024-08-19T01:33:23.189802+0000","last_became_peered":"2024-08-19T01:33:23.189802+0000","last_unstale":"2024-08-19T01:33:24.175781+0000","last_undegraded":"2024-08-19T01:33:24.175781+0000","last_fullsized":"2024-08-19T01:33:24.175781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:53:10.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.950719+0000","last_change":"2024-08-19T01:33:21.167443+0000","last_active":"2024-08-19T01:33:32.950719+0000","last_peered":"2024-08-19T01:33:32.950719+0000","last_clean":"2024-08-19T01:33:32.950719+0000","last_became_active":"2024-08-19T01:33:21.167054+0000","last_became_peered":"2024-08-19T01:33:21.167054+0000","last_unstale":"2024-08-19T01:33:32.950719+0000","last_undegraded":"2024-08-19T01:33:32.950719+0000","last_fullsized":"2024-08-19T01:33:32.950719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:00:38.358807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:31.944589+0000","last_change":"2024-08-19T01:33:21.163097+0000","last_active":"2024-08-19T01:33:31.944589+0000","last_peered":"2024-08-19T01:33:31.944589+0000","last_clean":"2024-08-19T01:33:31.944589+0000","last_became_active":"2024-08-19T01:33:21.162802+0000","last_became_peered":"2024-08-19T01:33:21.162802+0000","last_unstale":"2024-08-19T01:33:31.944589+0000","last_undegraded":"2024-08-19T01:33:31.944589+0000","last_fullsized":"2024-08-19T01:33:31.944589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:15:41.244104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182649+0000","last_change":"2024-08-19T01:33:23.200975+0000","last_active":"2024-08-19T01:33:24.182649+0000","last_peered":"2024-08-19T01:33:24.182649+0000","last_clean":"2024-08-19T01:33:24.182649+0000","last_became_active":"2024-08-19T01:33:23.200667+0000","last_became_peered":"2024-08-19T01:33:23.200667+0000","last_unstale":"2024-08-19T01:33:24.182649+0000","last_undegraded":"2024-08-19T01:33:24.182649+0000","last_fullsized":"2024-08-19T01:33:24.182649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:35.105521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.960474+0000","last_change":"2024-08-19T01:33:21.171252+0000","last_active":"2024-08-19T01:33:33.960474+0000","last_peered":"2024-08-19T01:33:33.960474+0000","last_clean":"2024-08-19T01:33:33.960474+0000","last_became_active":"2024-08-19T01:33:21.170944+0000","last_became_peered":"2024-08-19T01:33:21.170944+0000","last_unstale":"2024-08-19T01:33:33.960474+0000","last_undegraded":"2024-08-19T01:33:33.960474+0000","last_fullsized":"2024-08-19T01:33:33.960474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:10:15.869452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185094+0000","last_change":"2024-08-19T01:33:23.191931+0000","last_active":"2024-08-19T01:33:24.185094+0000","last_peered":"2024-08-19T01:33:24.185094+0000","last_clean":"2024-08-19T01:33:24.185094+0000","last_became_active":"2024-08-19T01:33:23.191601+0000","last_became_peered":"2024-08-19T01:33:23.191601+0000","last_unstale":"2024-08-19T01:33:24.185094+0000","last_undegraded":"2024-08-19T01:33:24.185094+0000","last_fullsized":"2024-08-19T01:33:24.185094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:52.874546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:40.982697+0000","last_change":"2024-08-19T01:33:21.170714+0000","last_active":"2024-08-19T01:33:40.982697+0000","last_peered":"2024-08-19T01:33:40.982697+0000","last_clean":"2024-08-19T01:33:40.982697+0000","last_became_active":"2024-08-19T01:33:21.170379+0000","last_became_peered":"2024-08-19T01:33:21.170379+0000","last_unstale":"2024-08-19T01:33:40.982697+0000","last_undegraded":"2024-08-19T01:33:40.982697+0000","last_fullsized":"2024-08-19T01:33:40.982697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:27:27.417634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183469+0000","last_change":"2024-08-19T01:33:23.189680+0000","last_active":"2024-08-19T01:33:24.183469+0000","last_peered":"2024-08-19T01:33:24.183469+0000","last_clean":"2024-08-19T01:33:24.183469+0000","last_became_active":"2024-08-19T01:33:23.189029+0000","last_became_peered":"2024-08-19T01:33:23.189029+0000","last_unstale":"2024-08-19T01:33:24.183469+0000","last_undegraded":"2024-08-19T01:33:24.183469+0000","last_fullsized":"2024-08-19T01:33:24.183469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:39.667675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958346+0000","last_change":"2024-08-19T01:33:21.169233+0000","last_active":"2024-08-19T01:33:33.958346+0000","last_peered":"2024-08-19T01:33:33.958346+0000","last_clean":"2024-08-19T01:33:33.958346+0000","last_became_active":"2024-08-19T01:33:21.168761+0000","last_became_peered":"2024-08-19T01:33:21.168761+0000","last_unstale":"2024-08-19T01:33:33.958346+0000","last_undegraded":"2024-08-19T01:33:33.958346+0000","last_fullsized":"2024-08-19T01:33:33.958346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:31:56.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185364+0000","last_change":"2024-08-19T01:33:23.194250+0000","last_active":"2024-08-19T01:33:24.185364+0000","last_peered":"2024-08-19T01:33:24.185364+0000","last_clean":"2024-08-19T01:33:24.185364+0000","last_became_active":"2024-08-19T01:33:23.193939+0000","last_became_peered":"2024-08-19T01:33:23.193939+0000","last_unstale":"2024-08-19T01:33:24.185364+0000","last_undegraded":"2024-08-19T01:33:24.185364+0000","last_fullsized":"2024-08-19T01:33:24.185364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:19:09.703701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.950817+0000","last_change":"2024-08-19T01:33:21.287684+0000","last_active":"2024-08-19T01:33:32.950817+0000","last_peered":"2024-08-19T01:33:32.950817+0000","last_clean":"2024-08-19T01:33:32.950817+0000","last_became_active":"2024-08-19T01:33:21.287252+0000","last_became_peered":"2024-08-19T01:33:21.287252+0000","last_unstale":"2024-08-19T01:33:32.950817+0000","last_undegraded":"2024-08-19T01:33:32.950817+0000","last_fullsized":"2024-08-19T01:33:32.950817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:08.801345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182801+0000","last_change":"2024-08-19T01:33:23.195077+0000","last_active":"2024-08-19T01:33:24.182801+0000","last_peered":"2024-08-19T01:33:24.182801+0000","last_clean":"2024-08-19T01:33:24.182801+0000","last_became_active":"2024-08-19T01:33:23.194753+0000","last_became_peered":"2024-08-19T01:33:23.194753+0000","last_unstale":"2024-08-19T01:33:24.182801+0000","last_undegraded":"2024-08-19T01:33:24.182801+0000","last_fullsized":"2024-08-19T01:33:24.182801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:12:09.186261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'23","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:41.816958+0000","last_change":"2024-08-19T01:33:21.168845+0000","last_active":"2024-08-19T01:33:41.816958+0000","last_peered":"2024-08-19T01:33:41.816958+0000","last_clean":"2024-08-19T01:33:41.816958+0000","last_became_active":"2024-08-19T01:33:21.168479+0000","last_became_peered":"2024-08-19T01:33:21.168479+0000","last_unstale":"2024-08-19T01:33:41.816958+0000","last_undegraded":"2024-08-19T01:33:41.816958+0000","last_fullsized":"2024-08-19T01:33:41.816958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:59:50.296329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41062,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":30,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184294+0000","last_change":"2024-08-19T01:33:23.192252+0000","last_active":"2024-08-19T01:33:24.184294+0000","last_peered":"2024-08-19T01:33:24.184294+0000","last_clean":"2024-08-19T01:33:24.184294+0000","last_became_active":"2024-08-19T01:33:23.191934+0000","last_became_peered":"2024-08-19T01:33:23.191934+0000","last_unstale":"2024-08-19T01:33:24.184294+0000","last_undegraded":"2024-08-19T01:33:24.184294+0000","last_fullsized":"2024-08-19T01:33:24.184294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:31.134825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:34.971455+0000","last_change":"2024-08-19T01:33:21.168541+0000","last_active":"2024-08-19T01:33:34.971455+0000","last_peered":"2024-08-19T01:33:34.971455+0000","last_clean":"2024-08-19T01:33:34.971455+0000","last_became_active":"2024-08-19T01:33:21.168205+0000","last_became_peered":"2024-08-19T01:33:21.168205+0000","last_unstale":"2024-08-19T01:33:34.971455+0000","last_undegraded":"2024-08-19T01:33:34.971455+0000","last_fullsized":"2024-08-19T01:33:34.971455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:19:52.078947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174312+0000","last_change":"2024-08-19T01:33:23.193013+0000","last_active":"2024-08-19T01:33:24.174312+0000","last_peered":"2024-08-19T01:33:24.174312+0000","last_clean":"2024-08-19T01:33:24.174312+0000","last_became_active":"2024-08-19T01:33:23.192728+0000","last_became_peered":"2024-08-19T01:33:23.192728+0000","last_unstale":"2024-08-19T01:33:24.174312+0000","last_undegraded":"2024-08-19T01:33:24.174312+0000","last_fullsized":"2024-08-19T01:33:24.174312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:23:17.587527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958916+0000","last_change":"2024-08-19T01:33:21.170118+0000","last_active":"2024-08-19T01:33:33.958916+0000","last_peered":"2024-08-19T01:33:33.958916+0000","last_clean":"2024-08-19T01:33:33.958916+0000","last_became_active":"2024-08-19T01:33:21.169757+0000","last_became_peered":"2024-08-19T01:33:21.169757+0000","last_unstale":"2024-08-19T01:33:33.958916+0000","last_undegraded":"2024-08-19T01:33:33.958916+0000","last_fullsized":"2024-08-19T01:33:33.958916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:09:26.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186584+0000","last_change":"2024-08-19T01:33:23.191270+0000","last_active":"2024-08-19T01:33:24.186584+0000","last_peered":"2024-08-19T01:33:24.186584+0000","last_clean":"2024-08-19T01:33:24.186584+0000","last_became_active":"2024-08-19T01:33:23.190944+0000","last_became_peered":"2024-08-19T01:33:23.190944+0000","last_unstale":"2024-08-19T01:33:24.186584+0000","last_undegraded":"2024-08-19T01:33:24.186584+0000","last_fullsized":"2024-08-19T01:33:24.186584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:38:38.242601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:43.014441+0000","last_change":"2024-08-19T01:33:21.165204+0000","last_active":"2024-08-19T01:33:43.014441+0000","last_peered":"2024-08-19T01:33:43.014441+0000","last_clean":"2024-08-19T01:33:43.014441+0000","last_became_active":"2024-08-19T01:33:21.164826+0000","last_became_peered":"2024-08-19T01:33:21.164826+0000","last_unstale":"2024-08-19T01:33:43.014441+0000","last_undegraded":"2024-08-19T01:33:43.014441+0000","last_fullsized":"2024-08-19T01:33:43.014441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:45:23.212064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185598+0000","last_change":"2024-08-19T01:33:23.198176+0000","last_active":"2024-08-19T01:33:24.185598+0000","last_peered":"2024-08-19T01:33:24.185598+0000","last_clean":"2024-08-19T01:33:24.185598+0000","last_became_active":"2024-08-19T01:33:23.196187+0000","last_became_peered":"2024-08-19T01:33:23.196187+0000","last_unstale":"2024-08-19T01:33:24.185598+0000","last_undegraded":"2024-08-19T01:33:24.185598+0000","last_fullsized":"2024-08-19T01:33:24.185598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:53:05.943467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958274+0000","last_change":"2024-08-19T01:33:21.287768+0000","last_active":"2024-08-19T01:33:33.958274+0000","last_peered":"2024-08-19T01:33:33.958274+0000","last_clean":"2024-08-19T01:33:33.958274+0000","last_became_active":"2024-08-19T01:33:21.287407+0000","last_became_peered":"2024-08-19T01:33:21.287407+0000","last_unstale":"2024-08-19T01:33:33.958274+0000","last_undegraded":"2024-08-19T01:33:33.958274+0000","last_fullsized":"2024-08-19T01:33:33.958274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:05:13.802207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175345+0000","last_change":"2024-08-19T01:33:23.201104+0000","last_active":"2024-08-19T01:33:24.175345+0000","last_peered":"2024-08-19T01:33:24.175345+0000","last_clean":"2024-08-19T01:33:24.175345+0000","last_became_active":"2024-08-19T01:33:23.200809+0000","last_became_peered":"2024-08-19T01:33:23.200809+0000","last_unstale":"2024-08-19T01:33:24.175345+0000","last_undegraded":"2024-08-19T01:33:24.175345+0000","last_fullsized":"2024-08-19T01:33:24.175345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:54.901412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.941089+0000","last_change":"2024-08-19T01:33:21.172423+0000","last_active":"2024-08-19T01:33:32.941089+0000","last_peered":"2024-08-19T01:33:32.941089+0000","last_clean":"2024-08-19T01:33:32.941089+0000","last_became_active":"2024-08-19T01:33:21.171950+0000","last_became_peered":"2024-08-19T01:33:21.171950+0000","last_unstale":"2024-08-19T01:33:32.941089+0000","last_undegraded":"2024-08-19T01:33:32.941089+0000","last_fullsized":"2024-08-19T01:33:32.941089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:44:11.446875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.189818+0000","last_change":"2024-08-19T01:33:23.192933+0000","last_active":"2024-08-19T01:33:24.189818+0000","last_peered":"2024-08-19T01:33:24.189818+0000","last_clean":"2024-08-19T01:33:24.189818+0000","last_became_active":"2024-08-19T01:33:23.192601+0000","last_became_peered":"2024-08-19T01:33:23.192601+0000","last_unstale":"2024-08-19T01:33:24.189818+0000","last_undegraded":"2024-08-19T01:33:24.189818+0000","last_fullsized":"2024-08-19T01:33:24.189818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:09.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.957343+0000","last_change":"2024-08-19T01:33:21.168225+0000","last_active":"2024-08-19T01:33:33.957343+0000","last_peered":"2024-08-19T01:33:33.957343+0000","last_clean":"2024-08-19T01:33:33.957343+0000","last_became_active":"2024-08-19T01:33:21.167881+0000","last_became_peered":"2024-08-19T01:33:21.167881+0000","last_unstale":"2024-08-19T01:33:33.957343+0000","last_undegraded":"2024-08-19T01:33:33.957343+0000","last_fullsized":"2024-08-19T01:33:33.957343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:53:38.037036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.188188+0000","last_change":"2024-08-19T01:33:23.194079+0000","last_active":"2024-08-19T01:33:24.188188+0000","last_peered":"2024-08-19T01:33:24.188188+0000","last_clean":"2024-08-19T01:33:24.188188+0000","last_became_active":"2024-08-19T01:33:23.193749+0000","last_became_peered":"2024-08-19T01:33:23.193749+0000","last_unstale":"2024-08-19T01:33:24.188188+0000","last_undegraded":"2024-08-19T01:33:24.188188+0000","last_fullsized":"2024-08-19T01:33:24.188188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:53:41.102163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.960258+0000","last_change":"2024-08-19T01:33:21.168916+0000","last_active":"2024-08-19T01:33:33.960258+0000","last_peered":"2024-08-19T01:33:33.960258+0000","last_clean":"2024-08-19T01:33:33.960258+0000","last_became_active":"2024-08-19T01:33:21.168618+0000","last_became_peered":"2024-08-19T01:33:21.168618+0000","last_unstale":"2024-08-19T01:33:33.960258+0000","last_undegraded":"2024-08-19T01:33:33.960258+0000","last_fullsized":"2024-08-19T01:33:33.960258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:45:11.567779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186282+0000","last_change":"2024-08-19T01:33:23.195546+0000","last_active":"2024-08-19T01:33:24.186282+0000","last_peered":"2024-08-19T01:33:24.186282+0000","last_clean":"2024-08-19T01:33:24.186282+0000","last_became_active":"2024-08-19T01:33:23.195221+0000","last_became_peered":"2024-08-19T01:33:23.195221+0000","last_unstale":"2024-08-19T01:33:24.186282+0000","last_undegraded":"2024-08-19T01:33:24.186282+0000","last_fullsized":"2024-08-19T01:33:24.186282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:46:10.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:42.986633+0000","last_change":"2024-08-19T01:33:21.287946+0000","last_active":"2024-08-19T01:33:42.986633+0000","last_peered":"2024-08-19T01:33:42.986633+0000","last_clean":"2024-08-19T01:33:42.986633+0000","last_became_active":"2024-08-19T01:33:21.287594+0000","last_became_peered":"2024-08-19T01:33:21.287594+0000","last_unstale":"2024-08-19T01:33:42.986633+0000","last_undegraded":"2024-08-19T01:33:42.986633+0000","last_fullsized":"2024-08-19T01:33:42.986633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:10:06.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.188245+0000","last_change":"2024-08-19T01:33:23.200914+0000","last_active":"2024-08-19T01:33:24.188245+0000","last_peered":"2024-08-19T01:33:24.188245+0000","last_clean":"2024-08-19T01:33:24.188245+0000","last_became_active":"2024-08-19T01:33:23.200611+0000","last_became_peered":"2024-08-19T01:33:23.200611+0000","last_unstale":"2024-08-19T01:33:24.188245+0000","last_undegraded":"2024-08-19T01:33:24.188245+0000","last_fullsized":"2024-08-19T01:33:24.188245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:22:00.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185919+0000","last_change":"2024-08-19T01:32:09.926979+0000","last_active":"2024-08-19T01:33:24.185919+0000","last_peered":"2024-08-19T01:33:24.185919+0000","last_clean":"2024-08-19T01:33:24.185919+0000","last_became_active":"2024-08-19T01:32:09.926560+0000","last_became_peered":"2024-08-19T01:32:09.926560+0000","last_unstale":"2024-08-19T01:33:24.185919+0000","last_undegraded":"2024-08-19T01:33:24.185919+0000","last_fullsized":"2024-08-19T01:33:24.185919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_clean_scrub_stamp":"2024-08-19T01:29:05.356703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:12:23.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:34.971430+0000","last_change":"2024-08-19T01:33:21.165099+0000","last_active":"2024-08-19T01:33:34.971430+0000","last_peered":"2024-08-19T01:33:34.971430+0000","last_clean":"2024-08-19T01:33:34.971430+0000","last_became_active":"2024-08-19T01:33:21.164756+0000","last_became_peered":"2024-08-19T01:33:21.164756+0000","last_unstale":"2024-08-19T01:33:34.971430+0000","last_undegraded":"2024-08-19T01:33:34.971430+0000","last_fullsized":"2024-08-19T01:33:34.971430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:11:54.480263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182680+0000","last_change":"2024-08-19T01:33:23.200185+0000","last_active":"2024-08-19T01:33:24.182680+0000","last_peered":"2024-08-19T01:33:24.182680+0000","last_clean":"2024-08-19T01:33:24.182680+0000","last_became_active":"2024-08-19T01:33:23.199885+0000","last_became_peered":"2024-08-19T01:33:23.199885+0000","last_unstale":"2024-08-19T01:33:24.182680+0000","last_undegraded":"2024-08-19T01:33:24.182680+0000","last_fullsized":"2024-08-19T01:33:24.182680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:17.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:31.945206+0000","last_change":"2024-08-19T01:33:21.172587+0000","last_active":"2024-08-19T01:33:31.945206+0000","last_peered":"2024-08-19T01:33:31.945206+0000","last_clean":"2024-08-19T01:33:31.945206+0000","last_became_active":"2024-08-19T01:33:21.172233+0000","last_became_peered":"2024-08-19T01:33:21.172233+0000","last_unstale":"2024-08-19T01:33:31.945206+0000","last_undegraded":"2024-08-19T01:33:31.945206+0000","last_fullsized":"2024-08-19T01:33:31.945206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:26.690403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174217+0000","last_change":"2024-08-19T01:33:23.197272+0000","last_active":"2024-08-19T01:33:24.174217+0000","last_peered":"2024-08-19T01:33:24.174217+0000","last_clean":"2024-08-19T01:33:24.174217+0000","last_became_active":"2024-08-19T01:33:23.196869+0000","last_became_peered":"2024-08-19T01:33:23.196869+0000","last_unstale":"2024-08-19T01:33:24.174217+0000","last_undegraded":"2024-08-19T01:33:24.174217+0000","last_fullsized":"2024-08-19T01:33:24.174217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:30.020184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:44.014964+0000","last_change":"2024-08-19T01:33:21.287366+0000","last_active":"2024-08-19T01:33:44.014964+0000","last_peered":"2024-08-19T01:33:44.014964+0000","last_clean":"2024-08-19T01:33:44.014964+0000","last_became_active":"2024-08-19T01:33:21.287044+0000","last_became_peered":"2024-08-19T01:33:21.287044+0000","last_unstale":"2024-08-19T01:33:44.014964+0000","last_undegraded":"2024-08-19T01:33:44.014964+0000","last_fullsized":"2024-08-19T01:33:44.014964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:19:04.194284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175704+0000","last_change":"2024-08-19T01:33:23.198097+0000","last_active":"2024-08-19T01:33:24.175704+0000","last_peered":"2024-08-19T01:33:24.175704+0000","last_clean":"2024-08-19T01:33:24.175704+0000","last_became_active":"2024-08-19T01:33:23.197635+0000","last_became_peered":"2024-08-19T01:33:23.197635+0000","last_unstale":"2024-08-19T01:33:24.175704+0000","last_undegraded":"2024-08-19T01:33:24.175704+0000","last_fullsized":"2024-08-19T01:33:24.175704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:20:44.190002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:41.001624+0000","last_change":"2024-08-19T01:33:21.164639+0000","last_active":"2024-08-19T01:33:41.001624+0000","last_peered":"2024-08-19T01:33:41.001624+0000","last_clean":"2024-08-19T01:33:41.001624+0000","last_became_active":"2024-08-19T01:33:21.164330+0000","last_became_peered":"2024-08-19T01:33:21.164330+0000","last_unstale":"2024-08-19T01:33:41.001624+0000","last_undegraded":"2024-08-19T01:33:41.001624+0000","last_fullsized":"2024-08-19T01:33:41.001624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:26:04.035130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.177423+0000","last_change":"2024-08-19T01:33:23.195845+0000","last_active":"2024-08-19T01:33:24.177423+0000","last_peered":"2024-08-19T01:33:24.177423+0000","last_clean":"2024-08-19T01:33:24.177423+0000","last_became_active":"2024-08-19T01:33:23.195519+0000","last_became_peered":"2024-08-19T01:33:23.195519+0000","last_unstale":"2024-08-19T01:33:24.177423+0000","last_undegraded":"2024-08-19T01:33:24.177423+0000","last_fullsized":"2024-08-19T01:33:24.177423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:53:45.363489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.795883+0000","last_change":"2024-08-19T01:33:21.287906+0000","last_active":"2024-08-19T01:33:24.795883+0000","last_peered":"2024-08-19T01:33:24.795883+0000","last_clean":"2024-08-19T01:33:24.795883+0000","last_became_active":"2024-08-19T01:33:21.287597+0000","last_became_peered":"2024-08-19T01:33:21.287597+0000","last_unstale":"2024-08-19T01:33:24.795883+0000","last_undegraded":"2024-08-19T01:33:24.795883+0000","last_fullsized":"2024-08-19T01:33:24.795883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:30.605948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:26.860193+0000","last_change":"2024-08-19T01:33:23.191026+0000","last_active":"2024-08-19T01:33:26.860193+0000","last_peered":"2024-08-19T01:33:26.860193+0000","last_clean":"2024-08-19T01:33:26.860193+0000","last_became_active":"2024-08-19T01:33:23.190382+0000","last_became_peered":"2024-08-19T01:33:23.190382+0000","last_unstale":"2024-08-19T01:33:26.860193+0000","last_undegraded":"2024-08-19T01:33:26.860193+0000","last_fullsized":"2024-08-19T01:33:26.860193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:33:45.065441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.959997+0000","last_change":"2024-08-19T01:33:21.288008+0000","last_active":"2024-08-19T01:33:33.959997+0000","last_peered":"2024-08-19T01:33:33.959997+0000","last_clean":"2024-08-19T01:33:33.959997+0000","last_became_active":"2024-08-19T01:33:21.287613+0000","last_became_peered":"2024-08-19T01:33:21.287613+0000","last_unstale":"2024-08-19T01:33:33.959997+0000","last_undegraded":"2024-08-19T01:33:33.959997+0000","last_fullsized":"2024-08-19T01:33:33.959997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:35:18.169564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174837+0000","last_change":"2024-08-19T01:33:23.197201+0000","last_active":"2024-08-19T01:33:24.174837+0000","last_peered":"2024-08-19T01:33:24.174837+0000","last_clean":"2024-08-19T01:33:24.174837+0000","last_became_active":"2024-08-19T01:33:23.196946+0000","last_became_peered":"2024-08-19T01:33:23.196946+0000","last_unstale":"2024-08-19T01:33:24.174837+0000","last_undegraded":"2024-08-19T01:33:24.174837+0000","last_fullsized":"2024-08-19T01:33:24.174837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:14:31.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:42.180879+0000","last_change":"2024-08-19T01:33:21.167109+0000","last_active":"2024-08-19T01:33:42.180879+0000","last_peered":"2024-08-19T01:33:42.180879+0000","last_clean":"2024-08-19T01:33:42.180879+0000","last_became_active":"2024-08-19T01:33:21.166788+0000","last_became_peered":"2024-08-19T01:33:21.166788+0000","last_unstale":"2024-08-19T01:33:42.180879+0000","last_undegraded":"2024-08-19T01:33:42.180879+0000","last_fullsized":"2024-08-19T01:33:42.180879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:44.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183334+0000","last_change":"2024-08-19T01:33:23.199699+0000","last_active":"2024-08-19T01:33:24.183334+0000","last_peered":"2024-08-19T01:33:24.183334+0000","last_clean":"2024-08-19T01:33:24.183334+0000","last_became_active":"2024-08-19T01:33:23.199417+0000","last_became_peered":"2024-08-19T01:33:23.199417+0000","last_unstale":"2024-08-19T01:33:24.183334+0000","last_undegraded":"2024-08-19T01:33:24.183334+0000","last_fullsized":"2024-08-19T01:33:24.183334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:36.519135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958202+0000","last_change":"2024-08-19T01:33:21.164177+0000","last_active":"2024-08-19T01:33:33.958202+0000","last_peered":"2024-08-19T01:33:33.958202+0000","last_clean":"2024-08-19T01:33:33.958202+0000","last_became_active":"2024-08-19T01:33:21.163832+0000","last_became_peered":"2024-08-19T01:33:21.163832+0000","last_unstale":"2024-08-19T01:33:33.958202+0000","last_undegraded":"2024-08-19T01:33:33.958202+0000","last_fullsized":"2024-08-19T01:33:33.958202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:31.907677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184690+0000","last_change":"2024-08-19T01:33:23.195102+0000","last_active":"2024-08-19T01:33:24.184690+0000","last_peered":"2024-08-19T01:33:24.184690+0000","last_clean":"2024-08-19T01:33:24.184690+0000","last_became_active":"2024-08-19T01:33:23.194802+0000","last_became_peered":"2024-08-19T01:33:23.194802+0000","last_unstale":"2024-08-19T01:33:24.184690+0000","last_undegraded":"2024-08-19T01:33:24.184690+0000","last_fullsized":"2024-08-19T01:33:24.184690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:33.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:30.938765+0000","last_change":"2024-08-19T01:33:21.287933+0000","last_active":"2024-08-19T01:33:30.938765+0000","last_peered":"2024-08-19T01:33:30.938765+0000","last_clean":"2024-08-19T01:33:30.938765+0000","last_became_active":"2024-08-19T01:33:21.287568+0000","last_became_peered":"2024-08-19T01:33:21.287568+0000","last_unstale":"2024-08-19T01:33:30.938765+0000","last_undegraded":"2024-08-19T01:33:30.938765+0000","last_fullsized":"2024-08-19T01:33:30.938765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:23.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182861+0000","last_change":"2024-08-19T01:33:23.194320+0000","last_active":"2024-08-19T01:33:24.182861+0000","last_peered":"2024-08-19T01:33:24.182861+0000","last_clean":"2024-08-19T01:33:24.182861+0000","last_became_active":"2024-08-19T01:33:23.194002+0000","last_became_peered":"2024-08-19T01:33:23.194002+0000","last_unstale":"2024-08-19T01:33:24.182861+0000","last_undegraded":"2024-08-19T01:33:24.182861+0000","last_fullsized":"2024-08-19T01:33:24.182861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:59.198997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.956894+0000","last_change":"2024-08-19T01:33:21.165421+0000","last_active":"2024-08-19T01:33:33.956894+0000","last_peered":"2024-08-19T01:33:33.956894+0000","last_clean":"2024-08-19T01:33:33.956894+0000","last_became_active":"2024-08-19T01:33:21.165013+0000","last_became_peered":"2024-08-19T01:33:21.165013+0000","last_unstale":"2024-08-19T01:33:33.956894+0000","last_undegraded":"2024-08-19T01:33:33.956894+0000","last_fullsized":"2024-08-19T01:33:33.956894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:07:18.546099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.190190+0000","last_change":"2024-08-19T01:33:23.193544+0000","last_active":"2024-08-19T01:33:24.190190+0000","last_peered":"2024-08-19T01:33:24.190190+0000","last_clean":"2024-08-19T01:33:24.190190+0000","last_became_active":"2024-08-19T01:33:23.193213+0000","last_became_peered":"2024-08-19T01:33:23.193213+0000","last_unstale":"2024-08-19T01:33:24.190190+0000","last_undegraded":"2024-08-19T01:33:24.190190+0000","last_fullsized":"2024-08-19T01:33:24.190190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:20:54.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.951920+0000","last_change":"2024-08-19T01:33:21.173620+0000","last_active":"2024-08-19T01:33:32.951920+0000","last_peered":"2024-08-19T01:33:32.951920+0000","last_clean":"2024-08-19T01:33:32.951920+0000","last_became_active":"2024-08-19T01:33:21.173204+0000","last_became_peered":"2024-08-19T01:33:21.173204+0000","last_unstale":"2024-08-19T01:33:32.951920+0000","last_undegraded":"2024-08-19T01:33:32.951920+0000","last_fullsized":"2024-08-19T01:33:32.951920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:24:48.710659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182898+0000","last_change":"2024-08-19T01:33:23.196266+0000","last_active":"2024-08-19T01:33:24.182898+0000","last_peered":"2024-08-19T01:33:24.182898+0000","last_clean":"2024-08-19T01:33:24.182898+0000","last_became_active":"2024-08-19T01:33:23.194268+0000","last_became_peered":"2024-08-19T01:33:23.194268+0000","last_unstale":"2024-08-19T01:33:24.182898+0000","last_undegraded":"2024-08-19T01:33:24.182898+0000","last_fullsized":"2024-08-19T01:33:24.182898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:15:38.628746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:30.938559+0000","last_change":"2024-08-19T01:33:21.171634+0000","last_active":"2024-08-19T01:33:30.938559+0000","last_peered":"2024-08-19T01:33:30.938559+0000","last_clean":"2024-08-19T01:33:30.938559+0000","last_became_active":"2024-08-19T01:33:21.171262+0000","last_became_peered":"2024-08-19T01:33:21.171262+0000","last_unstale":"2024-08-19T01:33:30.938559+0000","last_undegraded":"2024-08-19T01:33:30.938559+0000","last_fullsized":"2024-08-19T01:33:30.938559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:04.538597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.187159+0000","last_change":"2024-08-19T01:33:23.202534+0000","last_active":"2024-08-19T01:33:24.187159+0000","last_peered":"2024-08-19T01:33:24.187159+0000","last_clean":"2024-08-19T01:33:24.187159+0000","last_became_active":"2024-08-19T01:33:23.202191+0000","last_became_peered":"2024-08-19T01:33:23.202191+0000","last_unstale":"2024-08-19T01:33:24.187159+0000","last_undegraded":"2024-08-19T01:33:24.187159+0000","last_fullsized":"2024-08-19T01:33:24.187159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:03.478905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175547+0000","last_change":"2024-08-19T01:33:23.201136+0000","last_active":"2024-08-19T01:33:24.175547+0000","last_peered":"2024-08-19T01:33:24.175547+0000","last_clean":"2024-08-19T01:33:24.175547+0000","last_became_active":"2024-08-19T01:33:23.200864+0000","last_became_peered":"2024-08-19T01:33:23.200864+0000","last_unstale":"2024-08-19T01:33:24.175547+0000","last_undegraded":"2024-08-19T01:33:24.175547+0000","last_fullsized":"2024-08-19T01:33:24.175547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:32:42.091880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.960302+0000","last_change":"2024-08-19T01:33:21.172149+0000","last_active":"2024-08-19T01:33:33.960302+0000","last_peered":"2024-08-19T01:33:33.960302+0000","last_clean":"2024-08-19T01:33:33.960302+0000","last_became_active":"2024-08-19T01:33:21.171833+0000","last_became_peered":"2024-08-19T01:33:21.171833+0000","last_unstale":"2024-08-19T01:33:33.960302+0000","last_undegraded":"2024-08-19T01:33:33.960302+0000","last_fullsized":"2024-08-19T01:33:33.960302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:25:11.097212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182669+0000","last_change":"2024-08-19T01:33:23.193675+0000","last_active":"2024-08-19T01:33:24.182669+0000","last_peered":"2024-08-19T01:33:24.182669+0000","last_clean":"2024-08-19T01:33:24.182669+0000","last_became_active":"2024-08-19T01:33:23.188719+0000","last_became_peered":"2024-08-19T01:33:23.188719+0000","last_unstale":"2024-08-19T01:33:24.182669+0000","last_undegraded":"2024-08-19T01:33:24.182669+0000","last_fullsized":"2024-08-19T01:33:24.182669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:54:12.698211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:34.972130+0000","last_change":"2024-08-19T01:33:21.172675+0000","last_active":"2024-08-19T01:33:34.972130+0000","last_peered":"2024-08-19T01:33:34.972130+0000","last_clean":"2024-08-19T01:33:34.972130+0000","last_became_active":"2024-08-19T01:33:21.172320+0000","last_became_peered":"2024-08-19T01:33:21.172320+0000","last_unstale":"2024-08-19T01:33:34.972130+0000","last_undegraded":"2024-08-19T01:33:34.972130+0000","last_fullsized":"2024-08-19T01:33:34.972130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:29:41.511220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185298+0000","last_change":"2024-08-19T01:33:23.203113+0000","last_active":"2024-08-19T01:33:24.185298+0000","last_peered":"2024-08-19T01:33:24.185298+0000","last_clean":"2024-08-19T01:33:24.185298+0000","last_became_active":"2024-08-19T01:33:23.202807+0000","last_became_peered":"2024-08-19T01:33:23.202807+0000","last_unstale":"2024-08-19T01:33:24.185298+0000","last_undegraded":"2024-08-19T01:33:24.185298+0000","last_fullsized":"2024-08-19T01:33:24.185298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:24:21.570929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186026+0000","last_change":"2024-08-19T01:33:21.287382+0000","last_active":"2024-08-19T01:33:24.186026+0000","last_peered":"2024-08-19T01:33:24.186026+0000","last_clean":"2024-08-19T01:33:24.186026+0000","last_became_active":"2024-08-19T01:33:21.287040+0000","last_became_peered":"2024-08-19T01:33:21.287040+0000","last_unstale":"2024-08-19T01:33:24.186026+0000","last_undegraded":"2024-08-19T01:33:24.186026+0000","last_fullsized":"2024-08-19T01:33:24.186026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:01:49.973969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183157+0000","last_change":"2024-08-19T01:33:23.197869+0000","last_active":"2024-08-19T01:33:24.183157+0000","last_peered":"2024-08-19T01:33:24.183157+0000","last_clean":"2024-08-19T01:33:24.183157+0000","last_became_active":"2024-08-19T01:33:23.197529+0000","last_became_peered":"2024-08-19T01:33:23.197529+0000","last_unstale":"2024-08-19T01:33:24.183157+0000","last_undegraded":"2024-08-19T01:33:24.183157+0000","last_fullsized":"2024-08-19T01:33:24.183157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:14.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:43.003729+0000","last_change":"2024-08-19T01:33:21.172180+0000","last_active":"2024-08-19T01:33:43.003729+0000","last_peered":"2024-08-19T01:33:43.003729+0000","last_clean":"2024-08-19T01:33:43.003729+0000","last_became_active":"2024-08-19T01:33:21.171895+0000","last_became_peered":"2024-08-19T01:33:21.171895+0000","last_unstale":"2024-08-19T01:33:43.003729+0000","last_undegraded":"2024-08-19T01:33:43.003729+0000","last_fullsized":"2024-08-19T01:33:43.003729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:51:38.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48654,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":122,"num_read_kb":98,"num_write":229,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":145962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":436932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710741,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1055295,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874266,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":436862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":437240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364835,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1030537,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528360,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":1029318,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":436830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":436287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":479164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":478944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":437918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:33:49.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:33:49.494 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:49.907 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:49 smithi103 ceph-mon[18436]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:49.908 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:49 smithi103 ceph-mon[18436]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 -- 172.21.15.103:0/1713782785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 msgr2=0x7f53c4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/1713782785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4100c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009930 tx=0x7f53b402a420 comp rx=0 tx=0).stop 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 -- 172.21.15.103:0/1713782785 shutdown_connections 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/1713782785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c4101150 0x7f53c410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/1713782785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4100c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/1713782785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c41067b0 0x7f53c4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 -- 172.21.15.103:0/1713782785 >> 172.21.15.103:0/1713782785 conn(0x7f53c40fc4d0 msgr2=0x7f53c40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 -- 172.21.15.103:0/1713782785 shutdown_connections 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.986+0000 7f53cbb9a640 1 -- 172.21.15.103:0/1713782785 wait complete. 2024-08-19T01:33:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 Processor -- start 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 -- start start 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c4101150 0x7f53c4197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c41067b0 0x7f53c419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4110a60 con 0x7f53c41007b0 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41108e0 con 0x7f53c4101150 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53cbb9a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4110be0 con 0x7f53c41067b0 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53c9936640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53c9936640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58966/0 (socket says 172.21.15.103:58966) 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53c9936640 1 -- 172.21.15.103:0/258917414 learned_addr learned my addr 172.21.15.103:0/258917414 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53c9135640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c4101150 0x7f53c4197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.987+0000 7f53ca137640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c41067b0 0x7f53c419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53c9936640 1 -- 172.21.15.103:0/258917414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c41067b0 msgr2=0x7f53c419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53c9936640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c41067b0 0x7f53c419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53c9936640 1 -- 172.21.15.103:0/258917414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c4101150 msgr2=0x7f53c4197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53c9936640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c4101150 0x7f53c4197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53c9936640 1 -- 172.21.15.103:0/258917414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c419bf90 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53c9936640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4196fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53c000d770 tx=0x7f53c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53baffd640 1 -- 172.21.15.103:0/258917414 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0017070 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53baffd640 1 -- 172.21.15.103:0/258917414 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c00054d0 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c419c280 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53baffd640 1 -- 172.21.15.103:0/258917414 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0004cf0 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.988+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c41a3b60 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.989+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c4100c10 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.991+0000 7f53baffd640 1 -- 172.21.15.103:0/258917414 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f53c0004e90 con 0x7f53c41007b0 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.991+0000 7f53baffd640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53b00a2c90 0x7f53b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.992+0000 7f53c9135640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53b00a2c90 0x7f53b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:50.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.992+0000 7f53baffd640 1 -- 172.21.15.103:0/258917414 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f53c00ceda0 con 0x7f53c41007b0 2024-08-19T01:33:50.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.992+0000 7f53c9135640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53b00a2c90 0x7f53b00a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009900 tx=0x7f53b4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:50.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:49.995+0000 7f53baffd640 1 -- 172.21.15.103:0/258917414 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f53c009bdd0 con 0x7f53c41007b0 2024-08-19T01:33:50.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:49 smithi190 ceph-mon[22032]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:50.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:49 smithi190 ceph-mon[22032]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-08-19T01:33:50.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.160+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53c4062360 con 0x7f53b00a2c90 2024-08-19T01:33:50.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.167+0000 7f53baffd640 1 -- 172.21.15.103:0/258917414 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258934 (secure 0 0 0) 0x7f53c4062360 con 0x7f53b00a2c90 2024-08-19T01:33:50.171 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:50.172 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T01:33:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.169+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53b00a2c90 msgr2=0x7f53b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.169+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53b00a2c90 0x7f53b00a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009900 tx=0x7f53b4000bc0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.169+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 msgr2=0x7f53c4196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.169+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4196fe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f53c000d770 tx=0x7f53c000dc30 comp rx=0 tx=0).stop 2024-08-19T01:33:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 shutdown_connections 2024-08-19T01:33:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53b00a2c90 0x7f53b00a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c41067b0 0x7f53c419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53c4101150 0x7f53c4197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 --2- 172.21.15.103:0/258917414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53c41007b0 0x7f53c4196fe0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 >> 172.21.15.103:0/258917414 conn(0x7f53c40fc4d0 msgr2=0x7f53c40fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 shutdown_connections 2024-08-19T01:33:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:50.170+0000 7f53cbb9a640 1 -- 172.21.15.103:0/258917414 wait complete. 2024-08-19T01:33:50.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:49 smithi146 ceph-mon[22098]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:50.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:49 smithi146 ceph-mon[22098]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-08-19T01:33:50.914 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-08-19T01:33:48.776841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652565,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":279,"num_read_kb":241,"num_write":416,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360372,"kb_used_data":11220,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581452,"statfs":{"total":1151940427776,"available":1151571406848,"internally_reserved":0,"allocated":11489280,"data_stored":7146648,"data_compressed":107457,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":55513,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":143,"num_read_kb":122,"num_write":61,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003114"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.190579+0000","last_change":"2024-08-19T01:33:23.202035+0000","last_active":"2024-08-19T01:33:24.190579+0000","last_peered":"2024-08-19T01:33:24.190579+0000","last_clean":"2024-08-19T01:33:24.190579+0000","last_became_active":"2024-08-19T01:33:23.201418+0000","last_became_peered":"2024-08-19T01:33:23.201418+0000","last_unstale":"2024-08-19T01:33:24.190579+0000","last_undegraded":"2024-08-19T01:33:24.190579+0000","last_fullsized":"2024-08-19T01:33:24.190579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:54:57.927424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184681+0000","last_change":"2024-08-19T01:33:23.202047+0000","last_active":"2024-08-19T01:33:24.184681+0000","last_peered":"2024-08-19T01:33:24.184681+0000","last_clean":"2024-08-19T01:33:24.184681+0000","last_became_active":"2024-08-19T01:33:23.201762+0000","last_became_peered":"2024-08-19T01:33:23.201762+0000","last_unstale":"2024-08-19T01:33:24.184681+0000","last_undegraded":"2024-08-19T01:33:24.184681+0000","last_fullsized":"2024-08-19T01:33:24.184681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:23:57.252284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184897+0000","last_change":"2024-08-19T01:33:23.202876+0000","last_active":"2024-08-19T01:33:24.184897+0000","last_peered":"2024-08-19T01:33:24.184897+0000","last_clean":"2024-08-19T01:33:24.184897+0000","last_became_active":"2024-08-19T01:33:23.202528+0000","last_became_peered":"2024-08-19T01:33:23.202528+0000","last_unstale":"2024-08-19T01:33:24.184897+0000","last_undegraded":"2024-08-19T01:33:24.184897+0000","last_fullsized":"2024-08-19T01:33:24.184897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:20.514390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185874+0000","last_change":"2024-08-19T01:33:23.199443+0000","last_active":"2024-08-19T01:33:24.185874+0000","last_peered":"2024-08-19T01:33:24.185874+0000","last_clean":"2024-08-19T01:33:24.185874+0000","last_became_active":"2024-08-19T01:33:23.198344+0000","last_became_peered":"2024-08-19T01:33:23.198344+0000","last_unstale":"2024-08-19T01:33:24.185874+0000","last_undegraded":"2024-08-19T01:33:24.185874+0000","last_fullsized":"2024-08-19T01:33:24.185874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:43:43.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183932+0000","last_change":"2024-08-19T01:33:23.191185+0000","last_active":"2024-08-19T01:33:24.183932+0000","last_peered":"2024-08-19T01:33:24.183932+0000","last_clean":"2024-08-19T01:33:24.183932+0000","last_became_active":"2024-08-19T01:33:23.190892+0000","last_became_peered":"2024-08-19T01:33:23.190892+0000","last_unstale":"2024-08-19T01:33:24.183932+0000","last_undegraded":"2024-08-19T01:33:24.183932+0000","last_fullsized":"2024-08-19T01:33:24.183932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:28.384609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.177164+0000","last_change":"2024-08-19T01:33:23.193036+0000","last_active":"2024-08-19T01:33:24.177164+0000","last_peered":"2024-08-19T01:33:24.177164+0000","last_clean":"2024-08-19T01:33:24.177164+0000","last_became_active":"2024-08-19T01:33:23.192686+0000","last_became_peered":"2024-08-19T01:33:23.192686+0000","last_unstale":"2024-08-19T01:33:24.177164+0000","last_undegraded":"2024-08-19T01:33:24.177164+0000","last_fullsized":"2024-08-19T01:33:24.177164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:46.235112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185812+0000","last_change":"2024-08-19T01:33:23.197636+0000","last_active":"2024-08-19T01:33:24.185812+0000","last_peered":"2024-08-19T01:33:24.185812+0000","last_clean":"2024-08-19T01:33:24.185812+0000","last_became_active":"2024-08-19T01:33:23.197323+0000","last_became_peered":"2024-08-19T01:33:23.197323+0000","last_unstale":"2024-08-19T01:33:24.185812+0000","last_undegraded":"2024-08-19T01:33:24.185812+0000","last_fullsized":"2024-08-19T01:33:24.185812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:46:36.168786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182499+0000","last_change":"2024-08-19T01:33:23.193717+0000","last_active":"2024-08-19T01:33:24.182499+0000","last_peered":"2024-08-19T01:33:24.182499+0000","last_clean":"2024-08-19T01:33:24.182499+0000","last_became_active":"2024-08-19T01:33:23.193376+0000","last_became_peered":"2024-08-19T01:33:23.193376+0000","last_unstale":"2024-08-19T01:33:24.182499+0000","last_undegraded":"2024-08-19T01:33:24.182499+0000","last_fullsized":"2024-08-19T01:33:24.182499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:16.290081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183386+0000","last_change":"2024-08-19T01:33:23.197956+0000","last_active":"2024-08-19T01:33:24.183386+0000","last_peered":"2024-08-19T01:33:24.183386+0000","last_clean":"2024-08-19T01:33:24.183386+0000","last_became_active":"2024-08-19T01:33:23.197667+0000","last_became_peered":"2024-08-19T01:33:23.197667+0000","last_unstale":"2024-08-19T01:33:24.183386+0000","last_undegraded":"2024-08-19T01:33:24.183386+0000","last_fullsized":"2024-08-19T01:33:24.183386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:16.841046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.177654+0000","last_change":"2024-08-19T01:33:23.194569+0000","last_active":"2024-08-19T01:33:24.177654+0000","last_peered":"2024-08-19T01:33:24.177654+0000","last_clean":"2024-08-19T01:33:24.177654+0000","last_became_active":"2024-08-19T01:33:23.194223+0000","last_became_peered":"2024-08-19T01:33:23.194223+0000","last_unstale":"2024-08-19T01:33:24.177654+0000","last_undegraded":"2024-08-19T01:33:24.177654+0000","last_fullsized":"2024-08-19T01:33:24.177654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:14:30.184738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186285+0000","last_change":"2024-08-19T01:33:23.202396+0000","last_active":"2024-08-19T01:33:24.186285+0000","last_peered":"2024-08-19T01:33:24.186285+0000","last_clean":"2024-08-19T01:33:24.186285+0000","last_became_active":"2024-08-19T01:33:23.202063+0000","last_became_peered":"2024-08-19T01:33:23.202063+0000","last_unstale":"2024-08-19T01:33:24.186285+0000","last_undegraded":"2024-08-19T01:33:24.186285+0000","last_fullsized":"2024-08-19T01:33:24.186285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:57.513639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185543+0000","last_change":"2024-08-19T01:33:23.200610+0000","last_active":"2024-08-19T01:33:24.185543+0000","last_peered":"2024-08-19T01:33:24.185543+0000","last_clean":"2024-08-19T01:33:24.185543+0000","last_became_active":"2024-08-19T01:33:23.200136+0000","last_became_peered":"2024-08-19T01:33:23.200136+0000","last_unstale":"2024-08-19T01:33:24.185543+0000","last_undegraded":"2024-08-19T01:33:24.185543+0000","last_fullsized":"2024-08-19T01:33:24.185543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:47:39.252702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174693+0000","last_change":"2024-08-19T01:33:23.196062+0000","last_active":"2024-08-19T01:33:24.174693+0000","last_peered":"2024-08-19T01:33:24.174693+0000","last_clean":"2024-08-19T01:33:24.174693+0000","last_became_active":"2024-08-19T01:33:23.195577+0000","last_became_peered":"2024-08-19T01:33:23.195577+0000","last_unstale":"2024-08-19T01:33:24.174693+0000","last_undegraded":"2024-08-19T01:33:24.174693+0000","last_fullsized":"2024-08-19T01:33:24.174693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:46:09.111323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184440+0000","last_change":"2024-08-19T01:33:23.201664+0000","last_active":"2024-08-19T01:33:24.184440+0000","last_peered":"2024-08-19T01:33:24.184440+0000","last_clean":"2024-08-19T01:33:24.184440+0000","last_became_active":"2024-08-19T01:33:23.200241+0000","last_became_peered":"2024-08-19T01:33:23.200241+0000","last_unstale":"2024-08-19T01:33:24.184440+0000","last_undegraded":"2024-08-19T01:33:24.184440+0000","last_fullsized":"2024-08-19T01:33:24.184440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:17.199776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184303+0000","last_change":"2024-08-19T01:33:23.198102+0000","last_active":"2024-08-19T01:33:24.184303+0000","last_peered":"2024-08-19T01:33:24.184303+0000","last_clean":"2024-08-19T01:33:24.184303+0000","last_became_active":"2024-08-19T01:33:23.197778+0000","last_became_peered":"2024-08-19T01:33:23.197778+0000","last_unstale":"2024-08-19T01:33:24.184303+0000","last_undegraded":"2024-08-19T01:33:24.184303+0000","last_fullsized":"2024-08-19T01:33:24.184303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:03.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182240+0000","last_change":"2024-08-19T01:33:23.194581+0000","last_active":"2024-08-19T01:33:24.182240+0000","last_peered":"2024-08-19T01:33:24.182240+0000","last_clean":"2024-08-19T01:33:24.182240+0000","last_became_active":"2024-08-19T01:33:23.194256+0000","last_became_peered":"2024-08-19T01:33:23.194256+0000","last_unstale":"2024-08-19T01:33:24.182240+0000","last_undegraded":"2024-08-19T01:33:24.182240+0000","last_fullsized":"2024-08-19T01:33:24.182240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:17:20.954190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.191140+0000","last_change":"2024-08-19T01:33:23.201242+0000","last_active":"2024-08-19T01:33:24.191140+0000","last_peered":"2024-08-19T01:33:24.191140+0000","last_clean":"2024-08-19T01:33:24.191140+0000","last_became_active":"2024-08-19T01:33:23.200835+0000","last_became_peered":"2024-08-19T01:33:23.200835+0000","last_unstale":"2024-08-19T01:33:24.191140+0000","last_undegraded":"2024-08-19T01:33:24.191140+0000","last_fullsized":"2024-08-19T01:33:24.191140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:35:54.312266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185247+0000","last_change":"2024-08-19T01:33:23.187728+0000","last_active":"2024-08-19T01:33:24.185247+0000","last_peered":"2024-08-19T01:33:24.185247+0000","last_clean":"2024-08-19T01:33:24.185247+0000","last_became_active":"2024-08-19T01:33:23.187352+0000","last_became_peered":"2024-08-19T01:33:23.187352+0000","last_unstale":"2024-08-19T01:33:24.185247+0000","last_undegraded":"2024-08-19T01:33:24.185247+0000","last_fullsized":"2024-08-19T01:33:24.185247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:06:03.041328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186903+0000","last_change":"2024-08-19T01:33:23.193146+0000","last_active":"2024-08-19T01:33:24.186903+0000","last_peered":"2024-08-19T01:33:24.186903+0000","last_clean":"2024-08-19T01:33:24.186903+0000","last_became_active":"2024-08-19T01:33:23.192810+0000","last_became_peered":"2024-08-19T01:33:23.192810+0000","last_unstale":"2024-08-19T01:33:24.186903+0000","last_undegraded":"2024-08-19T01:33:24.186903+0000","last_fullsized":"2024-08-19T01:33:24.186903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:12.044175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.189491+0000","last_change":"2024-08-19T01:33:23.194635+0000","last_active":"2024-08-19T01:33:24.189491+0000","last_peered":"2024-08-19T01:33:24.189491+0000","last_clean":"2024-08-19T01:33:24.189491+0000","last_became_active":"2024-08-19T01:33:23.194308+0000","last_became_peered":"2024-08-19T01:33:23.194308+0000","last_unstale":"2024-08-19T01:33:24.189491+0000","last_undegraded":"2024-08-19T01:33:24.189491+0000","last_fullsized":"2024-08-19T01:33:24.189491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:12.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174804+0000","last_change":"2024-08-19T01:33:23.193108+0000","last_active":"2024-08-19T01:33:24.174804+0000","last_peered":"2024-08-19T01:33:24.174804+0000","last_clean":"2024-08-19T01:33:24.174804+0000","last_became_active":"2024-08-19T01:33:23.192830+0000","last_became_peered":"2024-08-19T01:33:23.192830+0000","last_unstale":"2024-08-19T01:33:24.174804+0000","last_undegraded":"2024-08-19T01:33:24.174804+0000","last_fullsized":"2024-08-19T01:33:24.174804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:33.355498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182362+0000","last_change":"2024-08-19T01:33:23.195789+0000","last_active":"2024-08-19T01:33:24.182362+0000","last_peered":"2024-08-19T01:33:24.182362+0000","last_clean":"2024-08-19T01:33:24.182362+0000","last_became_active":"2024-08-19T01:33:23.195064+0000","last_became_peered":"2024-08-19T01:33:23.195064+0000","last_unstale":"2024-08-19T01:33:24.182362+0000","last_undegraded":"2024-08-19T01:33:24.182362+0000","last_fullsized":"2024-08-19T01:33:24.182362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:05:33.848953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182747+0000","last_change":"2024-08-19T01:33:23.200968+0000","last_active":"2024-08-19T01:33:24.182747+0000","last_peered":"2024-08-19T01:33:24.182747+0000","last_clean":"2024-08-19T01:33:24.182747+0000","last_became_active":"2024-08-19T01:33:23.200606+0000","last_became_peered":"2024-08-19T01:33:23.200606+0000","last_unstale":"2024-08-19T01:33:24.182747+0000","last_undegraded":"2024-08-19T01:33:24.182747+0000","last_fullsized":"2024-08-19T01:33:24.182747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:44.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175504+0000","last_change":"2024-08-19T01:33:23.191204+0000","last_active":"2024-08-19T01:33:24.175504+0000","last_peered":"2024-08-19T01:33:24.175504+0000","last_clean":"2024-08-19T01:33:24.175504+0000","last_became_active":"2024-08-19T01:33:23.190709+0000","last_became_peered":"2024-08-19T01:33:23.190709+0000","last_unstale":"2024-08-19T01:33:24.175504+0000","last_undegraded":"2024-08-19T01:33:24.175504+0000","last_fullsized":"2024-08-19T01:33:24.175504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:56:10.988967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183164+0000","last_change":"2024-08-19T01:33:23.197168+0000","last_active":"2024-08-19T01:33:24.183164+0000","last_peered":"2024-08-19T01:33:24.183164+0000","last_clean":"2024-08-19T01:33:24.183164+0000","last_became_active":"2024-08-19T01:33:23.196883+0000","last_became_peered":"2024-08-19T01:33:23.196883+0000","last_unstale":"2024-08-19T01:33:24.183164+0000","last_undegraded":"2024-08-19T01:33:24.183164+0000","last_fullsized":"2024-08-19T01:33:24.183164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:33:35.386768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.187306+0000","last_change":"2024-08-19T01:33:23.187309+0000","last_active":"2024-08-19T01:33:24.187306+0000","last_peered":"2024-08-19T01:33:24.187306+0000","last_clean":"2024-08-19T01:33:24.187306+0000","last_became_active":"2024-08-19T01:33:23.186976+0000","last_became_peered":"2024-08-19T01:33:23.186976+0000","last_unstale":"2024-08-19T01:33:24.187306+0000","last_undegraded":"2024-08-19T01:33:24.187306+0000","last_fullsized":"2024-08-19T01:33:24.187306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:46:02.852000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.190181+0000","last_change":"2024-08-19T01:33:23.190292+0000","last_active":"2024-08-19T01:33:24.190181+0000","last_peered":"2024-08-19T01:33:24.190181+0000","last_clean":"2024-08-19T01:33:24.190181+0000","last_became_active":"2024-08-19T01:33:23.189969+0000","last_became_peered":"2024-08-19T01:33:23.189969+0000","last_unstale":"2024-08-19T01:33:24.190181+0000","last_undegraded":"2024-08-19T01:33:24.190181+0000","last_fullsized":"2024-08-19T01:33:24.190181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:35:51.388150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174739+0000","last_change":"2024-08-19T01:33:23.192148+0000","last_active":"2024-08-19T01:33:24.174739+0000","last_peered":"2024-08-19T01:33:24.174739+0000","last_clean":"2024-08-19T01:33:24.174739+0000","last_became_active":"2024-08-19T01:33:23.191845+0000","last_became_peered":"2024-08-19T01:33:23.191845+0000","last_unstale":"2024-08-19T01:33:24.174739+0000","last_undegraded":"2024-08-19T01:33:24.174739+0000","last_fullsized":"2024-08-19T01:33:24.174739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:51.308393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184626+0000","last_change":"2024-08-19T01:33:23.195033+0000","last_active":"2024-08-19T01:33:24.184626+0000","last_peered":"2024-08-19T01:33:24.184626+0000","last_clean":"2024-08-19T01:33:24.184626+0000","last_became_active":"2024-08-19T01:33:23.194745+0000","last_became_peered":"2024-08-19T01:33:23.194745+0000","last_unstale":"2024-08-19T01:33:24.184626+0000","last_undegraded":"2024-08-19T01:33:24.184626+0000","last_fullsized":"2024-08-19T01:33:24.184626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:33:40.748311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186518+0000","last_change":"2024-08-19T01:33:23.195817+0000","last_active":"2024-08-19T01:33:24.186518+0000","last_peered":"2024-08-19T01:33:24.186518+0000","last_clean":"2024-08-19T01:33:24.186518+0000","last_became_active":"2024-08-19T01:33:23.194688+0000","last_became_peered":"2024-08-19T01:33:23.194688+0000","last_unstale":"2024-08-19T01:33:24.186518+0000","last_undegraded":"2024-08-19T01:33:24.186518+0000","last_fullsized":"2024-08-19T01:33:24.186518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:13:04.913921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.187934+0000","last_change":"2024-08-19T01:33:23.187604+0000","last_active":"2024-08-19T01:33:24.187934+0000","last_peered":"2024-08-19T01:33:24.187934+0000","last_clean":"2024-08-19T01:33:24.187934+0000","last_became_active":"2024-08-19T01:33:23.187242+0000","last_became_peered":"2024-08-19T01:33:23.187242+0000","last_unstale":"2024-08-19T01:33:24.187934+0000","last_undegraded":"2024-08-19T01:33:24.187934+0000","last_fullsized":"2024-08-19T01:33:24.187934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:28.592525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.189956+0000","last_change":"2024-08-19T01:33:23.191737+0000","last_active":"2024-08-19T01:33:24.189956+0000","last_peered":"2024-08-19T01:33:24.189956+0000","last_clean":"2024-08-19T01:33:24.189956+0000","last_became_active":"2024-08-19T01:33:23.191358+0000","last_became_peered":"2024-08-19T01:33:23.191358+0000","last_unstale":"2024-08-19T01:33:24.189956+0000","last_undegraded":"2024-08-19T01:33:24.189956+0000","last_fullsized":"2024-08-19T01:33:24.189956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:28:14.694157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183641+0000","last_change":"2024-08-19T01:33:23.193110+0000","last_active":"2024-08-19T01:33:24.183641+0000","last_peered":"2024-08-19T01:33:24.183641+0000","last_clean":"2024-08-19T01:33:24.183641+0000","last_became_active":"2024-08-19T01:33:23.192789+0000","last_became_peered":"2024-08-19T01:33:23.192789+0000","last_unstale":"2024-08-19T01:33:24.183641+0000","last_undegraded":"2024-08-19T01:33:24.183641+0000","last_fullsized":"2024-08-19T01:33:24.183641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:45:48.141403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:45.991426+0000","last_change":"2024-08-19T01:33:21.173518+0000","last_active":"2024-08-19T01:33:45.991426+0000","last_peered":"2024-08-19T01:33:45.991426+0000","last_clean":"2024-08-19T01:33:45.991426+0000","last_became_active":"2024-08-19T01:33:21.172288+0000","last_became_peered":"2024-08-19T01:33:21.172288+0000","last_unstale":"2024-08-19T01:33:45.991426+0000","last_undegraded":"2024-08-19T01:33:45.991426+0000","last_fullsized":"2024-08-19T01:33:45.991426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:59:42.864724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175781+0000","last_change":"2024-08-19T01:33:23.190160+0000","last_active":"2024-08-19T01:33:24.175781+0000","last_peered":"2024-08-19T01:33:24.175781+0000","last_clean":"2024-08-19T01:33:24.175781+0000","last_became_active":"2024-08-19T01:33:23.189802+0000","last_became_peered":"2024-08-19T01:33:23.189802+0000","last_unstale":"2024-08-19T01:33:24.175781+0000","last_undegraded":"2024-08-19T01:33:24.175781+0000","last_fullsized":"2024-08-19T01:33:24.175781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:53:10.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.950719+0000","last_change":"2024-08-19T01:33:21.167443+0000","last_active":"2024-08-19T01:33:32.950719+0000","last_peered":"2024-08-19T01:33:32.950719+0000","last_clean":"2024-08-19T01:33:32.950719+0000","last_became_active":"2024-08-19T01:33:21.167054+0000","last_became_peered":"2024-08-19T01:33:21.167054+0000","last_unstale":"2024-08-19T01:33:32.950719+0000","last_undegraded":"2024-08-19T01:33:32.950719+0000","last_fullsized":"2024-08-19T01:33:32.950719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:00:38.358807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:31.944589+0000","last_change":"2024-08-19T01:33:21.163097+0000","last_active":"2024-08-19T01:33:31.944589+0000","last_peered":"2024-08-19T01:33:31.944589+0000","last_clean":"2024-08-19T01:33:31.944589+0000","last_became_active":"2024-08-19T01:33:21.162802+0000","last_became_peered":"2024-08-19T01:33:21.162802+0000","last_unstale":"2024-08-19T01:33:31.944589+0000","last_undegraded":"2024-08-19T01:33:31.944589+0000","last_fullsized":"2024-08-19T01:33:31.944589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:15:41.244104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182649+0000","last_change":"2024-08-19T01:33:23.200975+0000","last_active":"2024-08-19T01:33:24.182649+0000","last_peered":"2024-08-19T01:33:24.182649+0000","last_clean":"2024-08-19T01:33:24.182649+0000","last_became_active":"2024-08-19T01:33:23.200667+0000","last_became_peered":"2024-08-19T01:33:23.200667+0000","last_unstale":"2024-08-19T01:33:24.182649+0000","last_undegraded":"2024-08-19T01:33:24.182649+0000","last_fullsized":"2024-08-19T01:33:24.182649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:35.105521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.960474+0000","last_change":"2024-08-19T01:33:21.171252+0000","last_active":"2024-08-19T01:33:33.960474+0000","last_peered":"2024-08-19T01:33:33.960474+0000","last_clean":"2024-08-19T01:33:33.960474+0000","last_became_active":"2024-08-19T01:33:21.170944+0000","last_became_peered":"2024-08-19T01:33:21.170944+0000","last_unstale":"2024-08-19T01:33:33.960474+0000","last_undegraded":"2024-08-19T01:33:33.960474+0000","last_fullsized":"2024-08-19T01:33:33.960474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:10:15.869452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185094+0000","last_change":"2024-08-19T01:33:23.191931+0000","last_active":"2024-08-19T01:33:24.185094+0000","last_peered":"2024-08-19T01:33:24.185094+0000","last_clean":"2024-08-19T01:33:24.185094+0000","last_became_active":"2024-08-19T01:33:23.191601+0000","last_became_peered":"2024-08-19T01:33:23.191601+0000","last_unstale":"2024-08-19T01:33:24.185094+0000","last_undegraded":"2024-08-19T01:33:24.185094+0000","last_fullsized":"2024-08-19T01:33:24.185094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:52.874546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:45.990434+0000","last_change":"2024-08-19T01:33:21.170714+0000","last_active":"2024-08-19T01:33:45.990434+0000","last_peered":"2024-08-19T01:33:45.990434+0000","last_clean":"2024-08-19T01:33:45.990434+0000","last_became_active":"2024-08-19T01:33:21.170379+0000","last_became_peered":"2024-08-19T01:33:21.170379+0000","last_unstale":"2024-08-19T01:33:45.990434+0000","last_undegraded":"2024-08-19T01:33:45.990434+0000","last_fullsized":"2024-08-19T01:33:45.990434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:27:27.417634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183469+0000","last_change":"2024-08-19T01:33:23.189680+0000","last_active":"2024-08-19T01:33:24.183469+0000","last_peered":"2024-08-19T01:33:24.183469+0000","last_clean":"2024-08-19T01:33:24.183469+0000","last_became_active":"2024-08-19T01:33:23.189029+0000","last_became_peered":"2024-08-19T01:33:23.189029+0000","last_unstale":"2024-08-19T01:33:24.183469+0000","last_undegraded":"2024-08-19T01:33:24.183469+0000","last_fullsized":"2024-08-19T01:33:24.183469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:39.667675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958346+0000","last_change":"2024-08-19T01:33:21.169233+0000","last_active":"2024-08-19T01:33:33.958346+0000","last_peered":"2024-08-19T01:33:33.958346+0000","last_clean":"2024-08-19T01:33:33.958346+0000","last_became_active":"2024-08-19T01:33:21.168761+0000","last_became_peered":"2024-08-19T01:33:21.168761+0000","last_unstale":"2024-08-19T01:33:33.958346+0000","last_undegraded":"2024-08-19T01:33:33.958346+0000","last_fullsized":"2024-08-19T01:33:33.958346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:31:56.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185364+0000","last_change":"2024-08-19T01:33:23.194250+0000","last_active":"2024-08-19T01:33:24.185364+0000","last_peered":"2024-08-19T01:33:24.185364+0000","last_clean":"2024-08-19T01:33:24.185364+0000","last_became_active":"2024-08-19T01:33:23.193939+0000","last_became_peered":"2024-08-19T01:33:23.193939+0000","last_unstale":"2024-08-19T01:33:24.185364+0000","last_undegraded":"2024-08-19T01:33:24.185364+0000","last_fullsized":"2024-08-19T01:33:24.185364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:19:09.703701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.950817+0000","last_change":"2024-08-19T01:33:21.287684+0000","last_active":"2024-08-19T01:33:32.950817+0000","last_peered":"2024-08-19T01:33:32.950817+0000","last_clean":"2024-08-19T01:33:32.950817+0000","last_became_active":"2024-08-19T01:33:21.287252+0000","last_became_peered":"2024-08-19T01:33:21.287252+0000","last_unstale":"2024-08-19T01:33:32.950817+0000","last_undegraded":"2024-08-19T01:33:32.950817+0000","last_fullsized":"2024-08-19T01:33:32.950817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:08.801345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182801+0000","last_change":"2024-08-19T01:33:23.195077+0000","last_active":"2024-08-19T01:33:24.182801+0000","last_peered":"2024-08-19T01:33:24.182801+0000","last_clean":"2024-08-19T01:33:24.182801+0000","last_became_active":"2024-08-19T01:33:23.194753+0000","last_became_peered":"2024-08-19T01:33:23.194753+0000","last_unstale":"2024-08-19T01:33:24.182801+0000","last_undegraded":"2024-08-19T01:33:24.182801+0000","last_fullsized":"2024-08-19T01:33:24.182801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:12:09.186261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'27","reported_seq":74,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:46.340977+0000","last_change":"2024-08-19T01:33:21.168845+0000","last_active":"2024-08-19T01:33:46.340977+0000","last_peered":"2024-08-19T01:33:46.340977+0000","last_clean":"2024-08-19T01:33:46.340977+0000","last_became_active":"2024-08-19T01:33:21.168479+0000","last_became_peered":"2024-08-19T01:33:21.168479+0000","last_unstale":"2024-08-19T01:33:46.340977+0000","last_undegraded":"2024-08-19T01:33:46.340977+0000","last_fullsized":"2024-08-19T01:33:46.340977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:59:50.296329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54371,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":54,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184294+0000","last_change":"2024-08-19T01:33:23.192252+0000","last_active":"2024-08-19T01:33:24.184294+0000","last_peered":"2024-08-19T01:33:24.184294+0000","last_clean":"2024-08-19T01:33:24.184294+0000","last_became_active":"2024-08-19T01:33:23.191934+0000","last_became_peered":"2024-08-19T01:33:23.191934+0000","last_unstale":"2024-08-19T01:33:24.184294+0000","last_undegraded":"2024-08-19T01:33:24.184294+0000","last_fullsized":"2024-08-19T01:33:24.184294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:31.134825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:34.971455+0000","last_change":"2024-08-19T01:33:21.168541+0000","last_active":"2024-08-19T01:33:34.971455+0000","last_peered":"2024-08-19T01:33:34.971455+0000","last_clean":"2024-08-19T01:33:34.971455+0000","last_became_active":"2024-08-19T01:33:21.168205+0000","last_became_peered":"2024-08-19T01:33:21.168205+0000","last_unstale":"2024-08-19T01:33:34.971455+0000","last_undegraded":"2024-08-19T01:33:34.971455+0000","last_fullsized":"2024-08-19T01:33:34.971455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:19:52.078947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174312+0000","last_change":"2024-08-19T01:33:23.193013+0000","last_active":"2024-08-19T01:33:24.174312+0000","last_peered":"2024-08-19T01:33:24.174312+0000","last_clean":"2024-08-19T01:33:24.174312+0000","last_became_active":"2024-08-19T01:33:23.192728+0000","last_became_peered":"2024-08-19T01:33:23.192728+0000","last_unstale":"2024-08-19T01:33:24.174312+0000","last_undegraded":"2024-08-19T01:33:24.174312+0000","last_fullsized":"2024-08-19T01:33:24.174312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:23:17.587527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958916+0000","last_change":"2024-08-19T01:33:21.170118+0000","last_active":"2024-08-19T01:33:33.958916+0000","last_peered":"2024-08-19T01:33:33.958916+0000","last_clean":"2024-08-19T01:33:33.958916+0000","last_became_active":"2024-08-19T01:33:21.169757+0000","last_became_peered":"2024-08-19T01:33:21.169757+0000","last_unstale":"2024-08-19T01:33:33.958916+0000","last_undegraded":"2024-08-19T01:33:33.958916+0000","last_fullsized":"2024-08-19T01:33:33.958916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:09:26.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186584+0000","last_change":"2024-08-19T01:33:23.191270+0000","last_active":"2024-08-19T01:33:24.186584+0000","last_peered":"2024-08-19T01:33:24.186584+0000","last_clean":"2024-08-19T01:33:24.186584+0000","last_became_active":"2024-08-19T01:33:23.190944+0000","last_became_peered":"2024-08-19T01:33:23.190944+0000","last_unstale":"2024-08-19T01:33:24.186584+0000","last_undegraded":"2024-08-19T01:33:24.186584+0000","last_fullsized":"2024-08-19T01:33:24.186584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:38:38.242601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:43.014441+0000","last_change":"2024-08-19T01:33:21.165204+0000","last_active":"2024-08-19T01:33:43.014441+0000","last_peered":"2024-08-19T01:33:43.014441+0000","last_clean":"2024-08-19T01:33:43.014441+0000","last_became_active":"2024-08-19T01:33:21.164826+0000","last_became_peered":"2024-08-19T01:33:21.164826+0000","last_unstale":"2024-08-19T01:33:43.014441+0000","last_undegraded":"2024-08-19T01:33:43.014441+0000","last_fullsized":"2024-08-19T01:33:43.014441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:45:23.212064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185598+0000","last_change":"2024-08-19T01:33:23.198176+0000","last_active":"2024-08-19T01:33:24.185598+0000","last_peered":"2024-08-19T01:33:24.185598+0000","last_clean":"2024-08-19T01:33:24.185598+0000","last_became_active":"2024-08-19T01:33:23.196187+0000","last_became_peered":"2024-08-19T01:33:23.196187+0000","last_unstale":"2024-08-19T01:33:24.185598+0000","last_undegraded":"2024-08-19T01:33:24.185598+0000","last_fullsized":"2024-08-19T01:33:24.185598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:53:05.943467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958274+0000","last_change":"2024-08-19T01:33:21.287768+0000","last_active":"2024-08-19T01:33:33.958274+0000","last_peered":"2024-08-19T01:33:33.958274+0000","last_clean":"2024-08-19T01:33:33.958274+0000","last_became_active":"2024-08-19T01:33:21.287407+0000","last_became_peered":"2024-08-19T01:33:21.287407+0000","last_unstale":"2024-08-19T01:33:33.958274+0000","last_undegraded":"2024-08-19T01:33:33.958274+0000","last_fullsized":"2024-08-19T01:33:33.958274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:05:13.802207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175345+0000","last_change":"2024-08-19T01:33:23.201104+0000","last_active":"2024-08-19T01:33:24.175345+0000","last_peered":"2024-08-19T01:33:24.175345+0000","last_clean":"2024-08-19T01:33:24.175345+0000","last_became_active":"2024-08-19T01:33:23.200809+0000","last_became_peered":"2024-08-19T01:33:23.200809+0000","last_unstale":"2024-08-19T01:33:24.175345+0000","last_undegraded":"2024-08-19T01:33:24.175345+0000","last_fullsized":"2024-08-19T01:33:24.175345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:54.901412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.941089+0000","last_change":"2024-08-19T01:33:21.172423+0000","last_active":"2024-08-19T01:33:32.941089+0000","last_peered":"2024-08-19T01:33:32.941089+0000","last_clean":"2024-08-19T01:33:32.941089+0000","last_became_active":"2024-08-19T01:33:21.171950+0000","last_became_peered":"2024-08-19T01:33:21.171950+0000","last_unstale":"2024-08-19T01:33:32.941089+0000","last_undegraded":"2024-08-19T01:33:32.941089+0000","last_fullsized":"2024-08-19T01:33:32.941089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:44:11.446875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.189818+0000","last_change":"2024-08-19T01:33:23.192933+0000","last_active":"2024-08-19T01:33:24.189818+0000","last_peered":"2024-08-19T01:33:24.189818+0000","last_clean":"2024-08-19T01:33:24.189818+0000","last_became_active":"2024-08-19T01:33:23.192601+0000","last_became_peered":"2024-08-19T01:33:23.192601+0000","last_unstale":"2024-08-19T01:33:24.189818+0000","last_undegraded":"2024-08-19T01:33:24.189818+0000","last_fullsized":"2024-08-19T01:33:24.189818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:09.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.957343+0000","last_change":"2024-08-19T01:33:21.168225+0000","last_active":"2024-08-19T01:33:33.957343+0000","last_peered":"2024-08-19T01:33:33.957343+0000","last_clean":"2024-08-19T01:33:33.957343+0000","last_became_active":"2024-08-19T01:33:21.167881+0000","last_became_peered":"2024-08-19T01:33:21.167881+0000","last_unstale":"2024-08-19T01:33:33.957343+0000","last_undegraded":"2024-08-19T01:33:33.957343+0000","last_fullsized":"2024-08-19T01:33:33.957343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:53:38.037036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.188188+0000","last_change":"2024-08-19T01:33:23.194079+0000","last_active":"2024-08-19T01:33:24.188188+0000","last_peered":"2024-08-19T01:33:24.188188+0000","last_clean":"2024-08-19T01:33:24.188188+0000","last_became_active":"2024-08-19T01:33:23.193749+0000","last_became_peered":"2024-08-19T01:33:23.193749+0000","last_unstale":"2024-08-19T01:33:24.188188+0000","last_undegraded":"2024-08-19T01:33:24.188188+0000","last_fullsized":"2024-08-19T01:33:24.188188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:53:41.102163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.960258+0000","last_change":"2024-08-19T01:33:21.168916+0000","last_active":"2024-08-19T01:33:33.960258+0000","last_peered":"2024-08-19T01:33:33.960258+0000","last_clean":"2024-08-19T01:33:33.960258+0000","last_became_active":"2024-08-19T01:33:21.168618+0000","last_became_peered":"2024-08-19T01:33:21.168618+0000","last_unstale":"2024-08-19T01:33:33.960258+0000","last_undegraded":"2024-08-19T01:33:33.960258+0000","last_fullsized":"2024-08-19T01:33:33.960258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:45:11.567779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186282+0000","last_change":"2024-08-19T01:33:23.195546+0000","last_active":"2024-08-19T01:33:24.186282+0000","last_peered":"2024-08-19T01:33:24.186282+0000","last_clean":"2024-08-19T01:33:24.186282+0000","last_became_active":"2024-08-19T01:33:23.195221+0000","last_became_peered":"2024-08-19T01:33:23.195221+0000","last_unstale":"2024-08-19T01:33:24.186282+0000","last_undegraded":"2024-08-19T01:33:24.186282+0000","last_fullsized":"2024-08-19T01:33:24.186282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:46:10.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:42.986633+0000","last_change":"2024-08-19T01:33:21.287946+0000","last_active":"2024-08-19T01:33:42.986633+0000","last_peered":"2024-08-19T01:33:42.986633+0000","last_clean":"2024-08-19T01:33:42.986633+0000","last_became_active":"2024-08-19T01:33:21.287594+0000","last_became_peered":"2024-08-19T01:33:21.287594+0000","last_unstale":"2024-08-19T01:33:42.986633+0000","last_undegraded":"2024-08-19T01:33:42.986633+0000","last_fullsized":"2024-08-19T01:33:42.986633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:10:06.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.188245+0000","last_change":"2024-08-19T01:33:23.200914+0000","last_active":"2024-08-19T01:33:24.188245+0000","last_peered":"2024-08-19T01:33:24.188245+0000","last_clean":"2024-08-19T01:33:24.188245+0000","last_became_active":"2024-08-19T01:33:23.200611+0000","last_became_peered":"2024-08-19T01:33:23.200611+0000","last_unstale":"2024-08-19T01:33:24.188245+0000","last_undegraded":"2024-08-19T01:33:24.188245+0000","last_fullsized":"2024-08-19T01:33:24.188245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:22:00.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185919+0000","last_change":"2024-08-19T01:32:09.926979+0000","last_active":"2024-08-19T01:33:24.185919+0000","last_peered":"2024-08-19T01:33:24.185919+0000","last_clean":"2024-08-19T01:33:24.185919+0000","last_became_active":"2024-08-19T01:32:09.926560+0000","last_became_peered":"2024-08-19T01:32:09.926560+0000","last_unstale":"2024-08-19T01:33:24.185919+0000","last_undegraded":"2024-08-19T01:33:24.185919+0000","last_fullsized":"2024-08-19T01:33:24.185919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_clean_scrub_stamp":"2024-08-19T01:29:05.356703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:12:23.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:34.971430+0000","last_change":"2024-08-19T01:33:21.165099+0000","last_active":"2024-08-19T01:33:34.971430+0000","last_peered":"2024-08-19T01:33:34.971430+0000","last_clean":"2024-08-19T01:33:34.971430+0000","last_became_active":"2024-08-19T01:33:21.164756+0000","last_became_peered":"2024-08-19T01:33:21.164756+0000","last_unstale":"2024-08-19T01:33:34.971430+0000","last_undegraded":"2024-08-19T01:33:34.971430+0000","last_fullsized":"2024-08-19T01:33:34.971430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:11:54.480263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182680+0000","last_change":"2024-08-19T01:33:23.200185+0000","last_active":"2024-08-19T01:33:24.182680+0000","last_peered":"2024-08-19T01:33:24.182680+0000","last_clean":"2024-08-19T01:33:24.182680+0000","last_became_active":"2024-08-19T01:33:23.199885+0000","last_became_peered":"2024-08-19T01:33:23.199885+0000","last_unstale":"2024-08-19T01:33:24.182680+0000","last_undegraded":"2024-08-19T01:33:24.182680+0000","last_fullsized":"2024-08-19T01:33:24.182680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:17.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:31.945206+0000","last_change":"2024-08-19T01:33:21.172587+0000","last_active":"2024-08-19T01:33:31.945206+0000","last_peered":"2024-08-19T01:33:31.945206+0000","last_clean":"2024-08-19T01:33:31.945206+0000","last_became_active":"2024-08-19T01:33:21.172233+0000","last_became_peered":"2024-08-19T01:33:21.172233+0000","last_unstale":"2024-08-19T01:33:31.945206+0000","last_undegraded":"2024-08-19T01:33:31.945206+0000","last_fullsized":"2024-08-19T01:33:31.945206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:26.690403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174217+0000","last_change":"2024-08-19T01:33:23.197272+0000","last_active":"2024-08-19T01:33:24.174217+0000","last_peered":"2024-08-19T01:33:24.174217+0000","last_clean":"2024-08-19T01:33:24.174217+0000","last_became_active":"2024-08-19T01:33:23.196869+0000","last_became_peered":"2024-08-19T01:33:23.196869+0000","last_unstale":"2024-08-19T01:33:24.174217+0000","last_undegraded":"2024-08-19T01:33:24.174217+0000","last_fullsized":"2024-08-19T01:33:24.174217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:30.020184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:44.014964+0000","last_change":"2024-08-19T01:33:21.287366+0000","last_active":"2024-08-19T01:33:44.014964+0000","last_peered":"2024-08-19T01:33:44.014964+0000","last_clean":"2024-08-19T01:33:44.014964+0000","last_became_active":"2024-08-19T01:33:21.287044+0000","last_became_peered":"2024-08-19T01:33:21.287044+0000","last_unstale":"2024-08-19T01:33:44.014964+0000","last_undegraded":"2024-08-19T01:33:44.014964+0000","last_fullsized":"2024-08-19T01:33:44.014964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:19:04.194284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175704+0000","last_change":"2024-08-19T01:33:23.198097+0000","last_active":"2024-08-19T01:33:24.175704+0000","last_peered":"2024-08-19T01:33:24.175704+0000","last_clean":"2024-08-19T01:33:24.175704+0000","last_became_active":"2024-08-19T01:33:23.197635+0000","last_became_peered":"2024-08-19T01:33:23.197635+0000","last_unstale":"2024-08-19T01:33:24.175704+0000","last_undegraded":"2024-08-19T01:33:24.175704+0000","last_fullsized":"2024-08-19T01:33:24.175704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:20:44.190002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:46.016806+0000","last_change":"2024-08-19T01:33:21.164639+0000","last_active":"2024-08-19T01:33:46.016806+0000","last_peered":"2024-08-19T01:33:46.016806+0000","last_clean":"2024-08-19T01:33:46.016806+0000","last_became_active":"2024-08-19T01:33:21.164330+0000","last_became_peered":"2024-08-19T01:33:21.164330+0000","last_unstale":"2024-08-19T01:33:46.016806+0000","last_undegraded":"2024-08-19T01:33:46.016806+0000","last_fullsized":"2024-08-19T01:33:46.016806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:26:04.035130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.177423+0000","last_change":"2024-08-19T01:33:23.195845+0000","last_active":"2024-08-19T01:33:24.177423+0000","last_peered":"2024-08-19T01:33:24.177423+0000","last_clean":"2024-08-19T01:33:24.177423+0000","last_became_active":"2024-08-19T01:33:23.195519+0000","last_became_peered":"2024-08-19T01:33:23.195519+0000","last_unstale":"2024-08-19T01:33:24.177423+0000","last_undegraded":"2024-08-19T01:33:24.177423+0000","last_fullsized":"2024-08-19T01:33:24.177423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:53:45.363489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.795883+0000","last_change":"2024-08-19T01:33:21.287906+0000","last_active":"2024-08-19T01:33:24.795883+0000","last_peered":"2024-08-19T01:33:24.795883+0000","last_clean":"2024-08-19T01:33:24.795883+0000","last_became_active":"2024-08-19T01:33:21.287597+0000","last_became_peered":"2024-08-19T01:33:21.287597+0000","last_unstale":"2024-08-19T01:33:24.795883+0000","last_undegraded":"2024-08-19T01:33:24.795883+0000","last_fullsized":"2024-08-19T01:33:24.795883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:30.605948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:26.860193+0000","last_change":"2024-08-19T01:33:23.191026+0000","last_active":"2024-08-19T01:33:26.860193+0000","last_peered":"2024-08-19T01:33:26.860193+0000","last_clean":"2024-08-19T01:33:26.860193+0000","last_became_active":"2024-08-19T01:33:23.190382+0000","last_became_peered":"2024-08-19T01:33:23.190382+0000","last_unstale":"2024-08-19T01:33:26.860193+0000","last_undegraded":"2024-08-19T01:33:26.860193+0000","last_fullsized":"2024-08-19T01:33:26.860193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:33:45.065441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.959997+0000","last_change":"2024-08-19T01:33:21.288008+0000","last_active":"2024-08-19T01:33:33.959997+0000","last_peered":"2024-08-19T01:33:33.959997+0000","last_clean":"2024-08-19T01:33:33.959997+0000","last_became_active":"2024-08-19T01:33:21.287613+0000","last_became_peered":"2024-08-19T01:33:21.287613+0000","last_unstale":"2024-08-19T01:33:33.959997+0000","last_undegraded":"2024-08-19T01:33:33.959997+0000","last_fullsized":"2024-08-19T01:33:33.959997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:35:18.169564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.174837+0000","last_change":"2024-08-19T01:33:23.197201+0000","last_active":"2024-08-19T01:33:24.174837+0000","last_peered":"2024-08-19T01:33:24.174837+0000","last_clean":"2024-08-19T01:33:24.174837+0000","last_became_active":"2024-08-19T01:33:23.196946+0000","last_became_peered":"2024-08-19T01:33:23.196946+0000","last_unstale":"2024-08-19T01:33:24.174837+0000","last_undegraded":"2024-08-19T01:33:24.174837+0000","last_fullsized":"2024-08-19T01:33:24.174837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:14:31.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:42.180879+0000","last_change":"2024-08-19T01:33:21.167109+0000","last_active":"2024-08-19T01:33:42.180879+0000","last_peered":"2024-08-19T01:33:42.180879+0000","last_clean":"2024-08-19T01:33:42.180879+0000","last_became_active":"2024-08-19T01:33:21.166788+0000","last_became_peered":"2024-08-19T01:33:21.166788+0000","last_unstale":"2024-08-19T01:33:42.180879+0000","last_undegraded":"2024-08-19T01:33:42.180879+0000","last_fullsized":"2024-08-19T01:33:42.180879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:44.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183334+0000","last_change":"2024-08-19T01:33:23.199699+0000","last_active":"2024-08-19T01:33:24.183334+0000","last_peered":"2024-08-19T01:33:24.183334+0000","last_clean":"2024-08-19T01:33:24.183334+0000","last_became_active":"2024-08-19T01:33:23.199417+0000","last_became_peered":"2024-08-19T01:33:23.199417+0000","last_unstale":"2024-08-19T01:33:24.183334+0000","last_undegraded":"2024-08-19T01:33:24.183334+0000","last_fullsized":"2024-08-19T01:33:24.183334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:36.519135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.958202+0000","last_change":"2024-08-19T01:33:21.164177+0000","last_active":"2024-08-19T01:33:33.958202+0000","last_peered":"2024-08-19T01:33:33.958202+0000","last_clean":"2024-08-19T01:33:33.958202+0000","last_became_active":"2024-08-19T01:33:21.163832+0000","last_became_peered":"2024-08-19T01:33:21.163832+0000","last_unstale":"2024-08-19T01:33:33.958202+0000","last_undegraded":"2024-08-19T01:33:33.958202+0000","last_fullsized":"2024-08-19T01:33:33.958202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:31.907677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.184690+0000","last_change":"2024-08-19T01:33:23.195102+0000","last_active":"2024-08-19T01:33:24.184690+0000","last_peered":"2024-08-19T01:33:24.184690+0000","last_clean":"2024-08-19T01:33:24.184690+0000","last_became_active":"2024-08-19T01:33:23.194802+0000","last_became_peered":"2024-08-19T01:33:23.194802+0000","last_unstale":"2024-08-19T01:33:24.184690+0000","last_undegraded":"2024-08-19T01:33:24.184690+0000","last_fullsized":"2024-08-19T01:33:24.184690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:33.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:30.938765+0000","last_change":"2024-08-19T01:33:21.287933+0000","last_active":"2024-08-19T01:33:30.938765+0000","last_peered":"2024-08-19T01:33:30.938765+0000","last_clean":"2024-08-19T01:33:30.938765+0000","last_became_active":"2024-08-19T01:33:21.287568+0000","last_became_peered":"2024-08-19T01:33:21.287568+0000","last_unstale":"2024-08-19T01:33:30.938765+0000","last_undegraded":"2024-08-19T01:33:30.938765+0000","last_fullsized":"2024-08-19T01:33:30.938765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:23.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182861+0000","last_change":"2024-08-19T01:33:23.194320+0000","last_active":"2024-08-19T01:33:24.182861+0000","last_peered":"2024-08-19T01:33:24.182861+0000","last_clean":"2024-08-19T01:33:24.182861+0000","last_became_active":"2024-08-19T01:33:23.194002+0000","last_became_peered":"2024-08-19T01:33:23.194002+0000","last_unstale":"2024-08-19T01:33:24.182861+0000","last_undegraded":"2024-08-19T01:33:24.182861+0000","last_fullsized":"2024-08-19T01:33:24.182861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:59.198997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.956894+0000","last_change":"2024-08-19T01:33:21.165421+0000","last_active":"2024-08-19T01:33:33.956894+0000","last_peered":"2024-08-19T01:33:33.956894+0000","last_clean":"2024-08-19T01:33:33.956894+0000","last_became_active":"2024-08-19T01:33:21.165013+0000","last_became_peered":"2024-08-19T01:33:21.165013+0000","last_unstale":"2024-08-19T01:33:33.956894+0000","last_undegraded":"2024-08-19T01:33:33.956894+0000","last_fullsized":"2024-08-19T01:33:33.956894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:07:18.546099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.190190+0000","last_change":"2024-08-19T01:33:23.193544+0000","last_active":"2024-08-19T01:33:24.190190+0000","last_peered":"2024-08-19T01:33:24.190190+0000","last_clean":"2024-08-19T01:33:24.190190+0000","last_became_active":"2024-08-19T01:33:23.193213+0000","last_became_peered":"2024-08-19T01:33:23.193213+0000","last_unstale":"2024-08-19T01:33:24.190190+0000","last_undegraded":"2024-08-19T01:33:24.190190+0000","last_fullsized":"2024-08-19T01:33:24.190190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:20:54.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:32.951920+0000","last_change":"2024-08-19T01:33:21.173620+0000","last_active":"2024-08-19T01:33:32.951920+0000","last_peered":"2024-08-19T01:33:32.951920+0000","last_clean":"2024-08-19T01:33:32.951920+0000","last_became_active":"2024-08-19T01:33:21.173204+0000","last_became_peered":"2024-08-19T01:33:21.173204+0000","last_unstale":"2024-08-19T01:33:32.951920+0000","last_undegraded":"2024-08-19T01:33:32.951920+0000","last_fullsized":"2024-08-19T01:33:32.951920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:24:48.710659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182898+0000","last_change":"2024-08-19T01:33:23.196266+0000","last_active":"2024-08-19T01:33:24.182898+0000","last_peered":"2024-08-19T01:33:24.182898+0000","last_clean":"2024-08-19T01:33:24.182898+0000","last_became_active":"2024-08-19T01:33:23.194268+0000","last_became_peered":"2024-08-19T01:33:23.194268+0000","last_unstale":"2024-08-19T01:33:24.182898+0000","last_undegraded":"2024-08-19T01:33:24.182898+0000","last_fullsized":"2024-08-19T01:33:24.182898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:15:38.628746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:30.938559+0000","last_change":"2024-08-19T01:33:21.171634+0000","last_active":"2024-08-19T01:33:30.938559+0000","last_peered":"2024-08-19T01:33:30.938559+0000","last_clean":"2024-08-19T01:33:30.938559+0000","last_became_active":"2024-08-19T01:33:21.171262+0000","last_became_peered":"2024-08-19T01:33:21.171262+0000","last_unstale":"2024-08-19T01:33:30.938559+0000","last_undegraded":"2024-08-19T01:33:30.938559+0000","last_fullsized":"2024-08-19T01:33:30.938559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:04.538597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.187159+0000","last_change":"2024-08-19T01:33:23.202534+0000","last_active":"2024-08-19T01:33:24.187159+0000","last_peered":"2024-08-19T01:33:24.187159+0000","last_clean":"2024-08-19T01:33:24.187159+0000","last_became_active":"2024-08-19T01:33:23.202191+0000","last_became_peered":"2024-08-19T01:33:23.202191+0000","last_unstale":"2024-08-19T01:33:24.187159+0000","last_undegraded":"2024-08-19T01:33:24.187159+0000","last_fullsized":"2024-08-19T01:33:24.187159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:03.478905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.175547+0000","last_change":"2024-08-19T01:33:23.201136+0000","last_active":"2024-08-19T01:33:24.175547+0000","last_peered":"2024-08-19T01:33:24.175547+0000","last_clean":"2024-08-19T01:33:24.175547+0000","last_became_active":"2024-08-19T01:33:23.200864+0000","last_became_peered":"2024-08-19T01:33:23.200864+0000","last_unstale":"2024-08-19T01:33:24.175547+0000","last_undegraded":"2024-08-19T01:33:24.175547+0000","last_fullsized":"2024-08-19T01:33:24.175547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:32:42.091880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:33.960302+0000","last_change":"2024-08-19T01:33:21.172149+0000","last_active":"2024-08-19T01:33:33.960302+0000","last_peered":"2024-08-19T01:33:33.960302+0000","last_clean":"2024-08-19T01:33:33.960302+0000","last_became_active":"2024-08-19T01:33:21.171833+0000","last_became_peered":"2024-08-19T01:33:21.171833+0000","last_unstale":"2024-08-19T01:33:33.960302+0000","last_undegraded":"2024-08-19T01:33:33.960302+0000","last_fullsized":"2024-08-19T01:33:33.960302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:25:11.097212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.182669+0000","last_change":"2024-08-19T01:33:23.193675+0000","last_active":"2024-08-19T01:33:24.182669+0000","last_peered":"2024-08-19T01:33:24.182669+0000","last_clean":"2024-08-19T01:33:24.182669+0000","last_became_active":"2024-08-19T01:33:23.188719+0000","last_became_peered":"2024-08-19T01:33:23.188719+0000","last_unstale":"2024-08-19T01:33:24.182669+0000","last_undegraded":"2024-08-19T01:33:24.182669+0000","last_fullsized":"2024-08-19T01:33:24.182669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:54:12.698211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:34.972130+0000","last_change":"2024-08-19T01:33:21.172675+0000","last_active":"2024-08-19T01:33:34.972130+0000","last_peered":"2024-08-19T01:33:34.972130+0000","last_clean":"2024-08-19T01:33:34.972130+0000","last_became_active":"2024-08-19T01:33:21.172320+0000","last_became_peered":"2024-08-19T01:33:21.172320+0000","last_unstale":"2024-08-19T01:33:34.972130+0000","last_undegraded":"2024-08-19T01:33:34.972130+0000","last_fullsized":"2024-08-19T01:33:34.972130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:29:41.511220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.185298+0000","last_change":"2024-08-19T01:33:23.203113+0000","last_active":"2024-08-19T01:33:24.185298+0000","last_peered":"2024-08-19T01:33:24.185298+0000","last_clean":"2024-08-19T01:33:24.185298+0000","last_became_active":"2024-08-19T01:33:23.202807+0000","last_became_peered":"2024-08-19T01:33:23.202807+0000","last_unstale":"2024-08-19T01:33:24.185298+0000","last_undegraded":"2024-08-19T01:33:24.185298+0000","last_fullsized":"2024-08-19T01:33:24.185298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:24:21.570929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.186026+0000","last_change":"2024-08-19T01:33:21.287382+0000","last_active":"2024-08-19T01:33:24.186026+0000","last_peered":"2024-08-19T01:33:24.186026+0000","last_clean":"2024-08-19T01:33:24.186026+0000","last_became_active":"2024-08-19T01:33:21.287040+0000","last_became_peered":"2024-08-19T01:33:21.287040+0000","last_unstale":"2024-08-19T01:33:24.186026+0000","last_undegraded":"2024-08-19T01:33:24.186026+0000","last_fullsized":"2024-08-19T01:33:24.186026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:01:49.973969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:24.183157+0000","last_change":"2024-08-19T01:33:23.197869+0000","last_active":"2024-08-19T01:33:24.183157+0000","last_peered":"2024-08-19T01:33:24.183157+0000","last_clean":"2024-08-19T01:33:24.183157+0000","last_became_active":"2024-08-19T01:33:23.197529+0000","last_became_peered":"2024-08-19T01:33:23.197529+0000","last_unstale":"2024-08-19T01:33:24.183157+0000","last_undegraded":"2024-08-19T01:33:24.183157+0000","last_fullsized":"2024-08-19T01:33:24.183157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:14.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-08-19T01:33:43.003729+0000","last_change":"2024-08-19T01:33:21.172180+0000","last_active":"2024-08-19T01:33:43.003729+0000","last_peered":"2024-08-19T01:33:43.003729+0000","last_clean":"2024-08-19T01:33:43.003729+0000","last_became_active":"2024-08-19T01:33:21.171895+0000","last_became_peered":"2024-08-19T01:33:21.171895+0000","last_unstale":"2024-08-19T01:33:43.003729+0000","last_undegraded":"2024-08-19T01:33:43.003729+0000","last_fullsized":"2024-08-19T01:33:43.003729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:51:38.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61963,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":153,"num_read_kb":132,"num_write":233,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":159271,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":436932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710742,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1069413,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874266,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":436862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":437240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1030537,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528360,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":436969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1029435,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":436830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":436404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":479164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":478944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":437918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":56497,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:33:50.919 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-19T01:33:50.920 INFO:tasks.cephadm:Setting up client nodes... 2024-08-19T01:33:50.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-19T01:33:51.171 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:51.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/67858925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98102b30 msgr2=0x7f6d98102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:51.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/67858925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98102b30 0x7f6d98102fb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009900 tx=0x7f6d8002a410 comp rx=0 tx=0).stop 2024-08-19T01:33:51.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/67858925 shutdown_connections 2024-08-19T01:33:51.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/67858925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d981034f0 0x7f6d98109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/67858925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98102b30 0x7f6d98102fb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/67858925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d98101930 0x7f6d98101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/67858925 >> 172.21.15.103:0/67858925 conn(0x7f6d980fd0c0 msgr2=0x7f6d980ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/67858925 shutdown_connections 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.687+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/67858925 wait complete. 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 Processor -- start 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 -- start start 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98101930 0x7f6d98199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d98102b30 0x7f6d9819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d981034f0 0x7f6d981a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810d4e0 con 0x7f6d98101930 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810d360 con 0x7f6d981034f0 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d9d7a0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810d660 con 0x7f6d98102b30 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d96ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98101930 0x7f6d98199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:51.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d96ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98101930 0x7f6d98199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58990/0 (socket says 172.21.15.103:58990) 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d96ffd640 1 -- 172.21.15.103:0/2384580855 learned_addr learned my addr 172.21.15.103:0/2384580855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d977fe640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d981034f0 0x7f6d981a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.688+0000 7f6d8e7fc640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d98102b30 0x7f6d9819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d96ffd640 1 -- 172.21.15.103:0/2384580855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d98102b30 msgr2=0x7f6d9819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d96ffd640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d98102b30 0x7f6d9819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d96ffd640 1 -- 172.21.15.103:0/2384580855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d981034f0 msgr2=0x7f6d981a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d96ffd640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d981034f0 0x7f6d981a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d96ffd640 1 -- 172.21.15.103:0/2384580855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d981a1a10 con 0x7f6d98101930 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d8e7fc640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d98102b30 0x7f6d9819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d96ffd640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98101930 0x7f6d98199de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b450 tx=0x7f6d8800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d94ff9640 1 -- 172.21.15.103:0/2384580855 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88015020 con 0x7f6d98101930 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d94ff9640 1 -- 172.21.15.103:0/2384580855 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d88011d40 con 0x7f6d98101930 2024-08-19T01:33:51.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d94ff9640 1 -- 172.21.15.103:0/2384580855 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800f340 con 0x7f6d98101930 2024-08-19T01:33:51.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d981a1d00 con 0x7f6d98101930 2024-08-19T01:33:51.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.689+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d981a2190 con 0x7f6d98101930 2024-08-19T01:33:51.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.690+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d981a2420 con 0x7f6d98101930 2024-08-19T01:33:51.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.691+0000 7f6d94ff9640 1 -- 172.21.15.103:0/2384580855 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6d88018020 con 0x7f6d98101930 2024-08-19T01:33:51.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.692+0000 7f6d94ff9640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6d840a2c40 0x7f6d840a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:51.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.692+0000 7f6d8e7fc640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6d840a2c40 0x7f6d840a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:51.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.692+0000 7f6d94ff9640 1 -- 172.21.15.103:0/2384580855 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f6d880cf0f0 con 0x7f6d98101930 2024-08-19T01:33:51.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.693+0000 7f6d8e7fc640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6d840a2c40 0x7f6d840a5100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006010 tx=0x7f6d80000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:51.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.695+0000 7f6d94ff9640 1 -- 172.21.15.103:0/2384580855 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6d8801f020 con 0x7f6d98101930 2024-08-19T01:33:51.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.926+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6d981a2770 con 0x7f6d98101930 2024-08-19T01:33:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.930+0000 7f6d94ff9640 1 -- 172.21.15.103:0/2384580855 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7f6d8800d040 con 0x7f6d98101930 2024-08-19T01:33:51.932 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-08-19T01:33:51.932 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQD/oMJmWU9YNxAAj+oKnrxYJyGzXftJqzhckw== 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6d840a2c40 msgr2=0x7f6d840a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6d840a2c40 0x7f6d840a5100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006010 tx=0x7f6d80000bc0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98101930 msgr2=0x7f6d98199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98101930 0x7f6d98199de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b450 tx=0x7f6d8800b910 comp rx=0 tx=0).stop 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 shutdown_connections 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6d840a2c40 0x7f6d840a5100 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d981034f0 0x7f6d981a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.932+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d98102b30 0x7f6d9819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.933+0000 7f6d9d7a0640 1 --2- 172.21.15.103:0/2384580855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d98101930 0x7f6d98199de0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:51.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.933+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 >> 172.21.15.103:0/2384580855 conn(0x7f6d980fd0c0 msgr2=0x7f6d98108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:51.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.933+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 shutdown_connections 2024-08-19T01:33:51.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:51.933+0000 7f6d9d7a0640 1 -- 172.21.15.103:0/2384580855 wait complete. 2024-08-19T01:33:52.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:51 smithi190 ceph-mon[22032]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:52.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:51 smithi190 ceph-mon[22032]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-08-19T01:33:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:51 smithi146 ceph-mon[22098]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:51 smithi146 ceph-mon[22098]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-08-19T01:33:52.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:51 smithi103 ceph-mon[18436]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:33:52.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:51 smithi103 ceph-mon[18436]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-08-19T01:33:52.684 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:33:52.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-19T01:33:52.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-19T01:33:52.721 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-19T01:33:52.721 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-19T01:33:52.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph mgr dump --format=json 2024-08-19T01:33:53.011 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:52 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2384580855' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:52 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2384580855' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-19T01:33:53.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:52 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2384580855' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:53.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:52 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2384580855' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-19T01:33:53.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:52 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2384580855' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-19T01:33:53.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:52 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2384580855' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-19T01:33:53.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 -- 172.21.15.103:0/3266612836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 msgr2=0x7f33e8101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 --2- 172.21.15.103:0/3266612836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e8101d00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009930 tx=0x7f33dc02a420 comp rx=0 tx=0).stop 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 -- 172.21.15.103:0/3266612836 shutdown_connections 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 --2- 172.21.15.103:0/3266612836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e8102240 0x7f33e810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 --2- 172.21.15.103:0/3266612836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e8101d00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 --2- 172.21.15.103:0/3266612836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33e81078a0 0x7f33e8107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 -- 172.21.15.103:0/3266612836 >> 172.21.15.103:0/3266612836 conn(0x7f33e80fd5e0 msgr2=0x7f33e80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 -- 172.21.15.103:0/3266612836 shutdown_connections 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.513+0000 7f33eddf2640 1 -- 172.21.15.103:0/3266612836 wait complete. 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 Processor -- start 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 -- start start 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33e8102240 0x7f33e8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e81078a0 0x7f33e819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33e8111b70 con 0x7f33e81018a0 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f33e81119f0 con 0x7f33e8102240 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33eddf2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f33e8111cf0 con 0x7f33e81078a0 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e81980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43902/0 (socket says 172.21.15.103:43902) 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33e77fe640 1 -- 172.21.15.103:0/2502667288 learned_addr learned my addr 172.21.15.103:0/2502667288 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.514+0000 7f33e6ffd640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33e8102240 0x7f33e8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e7fff640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e81078a0 0x7f33e819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e77fe640 1 -- 172.21.15.103:0/2502667288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e81078a0 msgr2=0x7f33e819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e77fe640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e81078a0 0x7f33e819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e77fe640 1 -- 172.21.15.103:0/2502667288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33e8102240 msgr2=0x7f33e8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e77fe640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33e8102240 0x7f33e8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e77fe640 1 -- 172.21.15.103:0/2502667288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e819d080 con 0x7f33e81018a0 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e77fe640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e81980d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f33d800c920 tx=0x7f33d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:53.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e4ff9640 1 -- 172.21.15.103:0/2502667288 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8007c70 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e4ff9640 1 -- 172.21.15.103:0/2502667288 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33d8007e10 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e819d370 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33e4ff9640 1 -- 172.21.15.103:0/2502667288 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8018820 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.515+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e81a4c50 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.516+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e8101d00 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.517+0000 7f33e4ff9640 1 -- 172.21.15.103:0/2502667288 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f33d80189c0 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.518+0000 7f33e4ff9640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f33d00a2c10 0x7f33d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.518+0000 7f33e6ffd640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f33d00a2c10 0x7f33d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.518+0000 7f33e4ff9640 1 -- 172.21.15.103:0/2502667288 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f33d80cf240 con 0x7f33e81018a0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.518+0000 7f33e6ffd640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f33d00a2c10 0x7f33d00a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000c00 tx=0x7f33dc031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:53.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.522+0000 7f33e4ff9640 1 -- 172.21.15.103:0/2502667288 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f33d8014030 con 0x7f33e81018a0 2024-08-19T01:33:53.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.734+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f33e8109e90 con 0x7f33e81018a0 2024-08-19T01:33:53.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.741+0000 7f33e4ff9640 1 -- 172.21.15.103:0/2502667288 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263208 (secure 0 0 0) 0x7f33d809c1f0 con 0x7f33e81018a0 2024-08-19T01:33:53.746 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:53.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f33d00a2c10 msgr2=0x7f33d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f33d00a2c10 0x7f33d00a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000c00 tx=0x7f33dc031400 comp rx=0 tx=0).stop 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 msgr2=0x7f33e81980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e81980d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f33d800c920 tx=0x7f33d800cde0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 shutdown_connections 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f33d00a2c10 0x7f33d00a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33e81078a0 0x7f33e819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f33e8102240 0x7f33e8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 --2- 172.21.15.103:0/2502667288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33e81018a0 0x7f33e81980d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 >> 172.21.15.103:0/2502667288 conn(0x7f33e80fd5e0 msgr2=0x7f33e80fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 shutdown_connections 2024-08-19T01:33:53.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:53.744+0000 7f33eddf2640 1 -- 172.21.15.103:0/2502667288 wait complete. 2024-08-19T01:33:54.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:53 smithi103 ceph-mon[18436]: pgmap v280: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-08-19T01:33:54.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:53 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2502667288' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-19T01:33:54.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:53 smithi190 ceph-mon[22032]: pgmap v280: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-08-19T01:33:54.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:53 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2502667288' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-19T01:33:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:53 smithi146 ceph-mon[22098]: pgmap v280: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-08-19T01:33:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:53 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2502667288' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-19T01:33:54.519 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2656970648},{"type":"v1","addr":"172.21.15.103:6801","nonce":2656970648}]},"active_addr":"172.21.15.103:6801/2656970648","active_change":"2024-08-19T01:27:00.697997+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.103:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":1423964731}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":927767623}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":3263898905}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":3433232620}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":721501221}]}]} 2024-08-19T01:33:54.527 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-19T01:33:54.527 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-19T01:33:54.527 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:54.778 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:55.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/829054536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 msgr2=0x7f3e04107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/829054536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e04107c90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009930 tx=0x7f3df802a420 comp rx=0 tx=0).stop 2024-08-19T01:33:55.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/829054536 shutdown_connections 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/829054536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e04102250 0x7f3e0410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/829054536 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e041018b0 0x7f3e04101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/829054536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e04107c90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/829054536 >> 172.21.15.103:0/829054536 conn(0x7f3e040fd5f0 msgr2=0x7f3e040ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/829054536 shutdown_connections 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/829054536 wait complete. 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 Processor -- start 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.272+0000 7f3e0ab4f640 1 -- start start 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e0ab4f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e041018b0 0x7f3e04198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e0ab4f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e04102250 0x7f3e04198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e0ab4f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e0419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e0ab4f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04111c00 con 0x7f3e041078b0 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e0ab4f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04111a80 con 0x7f3e041018b0 2024-08-19T01:33:55.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e0ab4f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04111d80 con 0x7f3e04102250 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e0419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e0419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43926/0 (socket says 172.21.15.103:43926) 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 -- 172.21.15.103:0/592663745 learned_addr learned my addr 172.21.15.103:0/592663745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e088eb640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e041018b0 0x7f3e04198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e03fff640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e04102250 0x7f3e04198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 -- 172.21.15.103:0/592663745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e04102250 msgr2=0x7f3e04198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e04102250 0x7f3e04198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 -- 172.21.15.103:0/592663745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e041018b0 msgr2=0x7f3e04198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e041018b0 0x7f3e04198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 -- 172.21.15.103:0/592663745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0419d1f0 con 0x7f3e041078b0 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e088eb640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e041018b0 0x7f3e04198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:55.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e03fff640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e04102250 0x7f3e04198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e090ec640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e0419cb10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b970 tx=0x7f3dfc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e01ffb640 1 -- 172.21.15.103:0/592663745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc019070 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0419d4e0 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e01ffb640 1 -- 172.21.15.103:0/592663745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dfc014440 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.273+0000 7f3e01ffb640 1 -- 172.21.15.103:0/592663745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0075d0 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.274+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a4dc0 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.275+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e04101d90 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.275+0000 7f3e01ffb640 1 -- 172.21.15.103:0/592663745 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3dfc01c070 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.276+0000 7f3e01ffb640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3df40a2c10 0x7f3df40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.276+0000 7f3e01ffb640 1 -- 172.21.15.103:0/592663745 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f3dfc0ce510 con 0x7f3e041078b0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.276+0000 7f3e088eb640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3df40a2c10 0x7f3df40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:55.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.277+0000 7f3e088eb640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3df40a2c10 0x7f3df40a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df8031cf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:55.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.280+0000 7f3e01ffb640 1 -- 172.21.15.103:0/592663745 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3dfc09b5d0 con 0x7f3e041078b0 2024-08-19T01:33:55.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.447+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e04109e80 con 0x7f3e041078b0 2024-08-19T01:33:55.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.448+0000 7f3e01ffb640 1 -- 172.21.15.103:0/592663745 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22301 (secure 0 0 0) 0x7f3dfc09aed0 con 0x7f3e041078b0 2024-08-19T01:33:55.451 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:55.451 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:33:23.991024+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:55.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3df40a2c10 msgr2=0x7f3df40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3df40a2c10 0x7f3df40a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df8031cf0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 msgr2=0x7f3e0419cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:55.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e0419cb10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b970 tx=0x7f3dfc00be30 comp rx=0 tx=0).stop 2024-08-19T01:33:55.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 shutdown_connections 2024-08-19T01:33:55.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3df40a2c10 0x7f3df40a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e041078b0 0x7f3e0419cb10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e04102250 0x7f3e04198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 --2- 172.21.15.103:0/592663745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e041018b0 0x7f3e04198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:55.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 >> 172.21.15.103:0/592663745 conn(0x7f3e040fd5f0 msgr2=0x7f3e040fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:55.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 shutdown_connections 2024-08-19T01:33:55.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:55.451+0000 7f3e0ab4f640 1 -- 172.21.15.103:0/592663745 wait complete. 2024-08-19T01:33:56.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:55 smithi190 ceph-mon[22032]: pgmap v281: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-19T01:33:56.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:55 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/592663745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:56.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:55 smithi103 ceph-mon[18436]: pgmap v281: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-19T01:33:56.151 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:55 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/592663745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:56.183 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-19T01:33:56.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:33:56.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:55 smithi146 ceph-mon[22098]: pgmap v281: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-19T01:33:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:55 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/592663745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:56.436 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:56.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 -- 172.21.15.103:0/2668768012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 msgr2=0x7f894410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 --2- 172.21.15.103:0/2668768012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f894410d680 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f893c009930 tx=0x7f893c02a420 comp rx=0 tx=0).stop 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 -- 172.21.15.103:0/2668768012 shutdown_connections 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 --2- 172.21.15.103:0/2668768012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f894410d680 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 --2- 172.21.15.103:0/2668768012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89441007b0 0x7f8944100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 --2- 172.21.15.103:0/2668768012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89441067b0 0x7f8944106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 -- 172.21.15.103:0/2668768012 >> 172.21.15.103:0/2668768012 conn(0x7f89440fc4d0 msgr2=0x7f89440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 -- 172.21.15.103:0/2668768012 shutdown_connections 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.942+0000 7f894944a640 1 -- 172.21.15.103:0/2668768012 wait complete. 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 Processor -- start 2024-08-19T01:33:56.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 -- start start 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89441007b0 0x7f8944196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f89441974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89441067b0 0x7f894419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8944110a60 con 0x7f8944101150 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f89441108e0 con 0x7f89441007b0 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f894944a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8944110be0 con 0x7f89441067b0 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f89427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f89441974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f89427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f89441974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43940/0 (socket says 172.21.15.103:43940) 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.943+0000 7f89427fc640 1 -- 172.21.15.103:0/508907393 learned_addr learned my addr 172.21.15.103:0/508907393 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f8942ffd640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89441007b0 0x7f8944196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:56.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f89427fc640 1 -- 172.21.15.103:0/508907393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89441067b0 msgr2=0x7f894419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f89427fc640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89441067b0 0x7f894419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f89427fc640 1 -- 172.21.15.103:0/508907393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89441007b0 msgr2=0x7f8944196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f89427fc640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89441007b0 0x7f8944196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f89427fc640 1 -- 172.21.15.103:0/508907393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894419bf20 con 0x7f8944101150 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f8942ffd640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89441007b0 0x7f8944196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f89427fc640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f89441974b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f892c00d950 tx=0x7f892c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f893bfff640 1 -- 172.21.15.103:0/508907393 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c017070 con 0x7f8944101150 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f893bfff640 1 -- 172.21.15.103:0/508907393 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f892c009c30 con 0x7f8944101150 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894419c210 con 0x7f8944101150 2024-08-19T01:33:56.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f893bfff640 1 -- 172.21.15.103:0/508907393 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c00f620 con 0x7f8944101150 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.944+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89441a3af0 con 0x7f8944101150 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.945+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8944100c10 con 0x7f8944101150 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.946+0000 7f893bfff640 1 -- 172.21.15.103:0/508907393 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f892c0109a0 con 0x7f8944101150 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.947+0000 7f893bfff640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f89300a2c10 0x7f89300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.947+0000 7f8942ffd640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f89300a2c10 0x7f89300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.947+0000 7f893bfff640 1 -- 172.21.15.103:0/508907393 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f892c0ced20 con 0x7f8944101150 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.947+0000 7f8942ffd640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f89300a2c10 0x7f89300a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f893400a300 tx=0x7f893400a290 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:56.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:56.951+0000 7f893bfff640 1 -- 172.21.15.103:0/508907393 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f892c009de0 con 0x7f8944101150 2024-08-19T01:33:57.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.119+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f894402d5e0 con 0x7f8944101150 2024-08-19T01:33:57.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.121+0000 7f893bfff640 1 -- 172.21.15.103:0/508907393 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22301 (secure 0 0 0) 0x7f892c09bcd0 con 0x7f8944101150 2024-08-19T01:33:57.123 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:33:57.124 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:33:23.991024+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:33:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.122+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f89300a2c10 msgr2=0x7f89300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:57.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.122+0000 7f894944a640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f89300a2c10 0x7f89300a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f893400a300 tx=0x7f893400a290 comp rx=0 tx=0).stop 2024-08-19T01:33:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.122+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 msgr2=0x7f89441974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.122+0000 7f894944a640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f89441974b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f892c00d950 tx=0x7f892c00de10 comp rx=0 tx=0).stop 2024-08-19T01:33:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 shutdown_connections 2024-08-19T01:33:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f89300a2c10 0x7f89300a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89441067b0 0x7f894419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8944101150 0x7f89441974b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 --2- 172.21.15.103:0/508907393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f89441007b0 0x7f8944196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:57.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 >> 172.21.15.103:0/508907393 conn(0x7f89440fc4d0 msgr2=0x7f89440fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:57.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 shutdown_connections 2024-08-19T01:33:57.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:57.123+0000 7f894944a640 1 -- 172.21.15.103:0/508907393 wait complete. 2024-08-19T01:33:57.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-19T01:33:57.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-19T01:33:57.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-19T01:33:57.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-19T01:33:57.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-19T01:33:57.850 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-19T01:33:57.851 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-19T01:33:57.851 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-19T01:33:57.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-19T01:33:57.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-19T01:33:57.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-19T01:33:57.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-19T01:33:58.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:57 smithi190 ceph-mon[22032]: pgmap v282: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:33:58.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:57 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/508907393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:57 smithi146 ceph-mon[22098]: pgmap v282: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:33:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:57 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/508907393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:58.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:57 smithi103 ceph-mon[18436]: pgmap v282: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:33:58.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:57 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/508907393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:33:58.507 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.615 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.625 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.625 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.635 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.635 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.638 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:58.641 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:33:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 -- 172.21.15.103:0/2093215509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc8102250 msgr2=0x7facc810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 --2- 172.21.15.103:0/2093215509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc8102250 0x7facc810e780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7facc0009cd0 tx=0x7facc002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 -- 172.21.15.103:0/2093215509 shutdown_connections 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 --2- 172.21.15.103:0/2093215509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc8102250 0x7facc810e780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 --2- 172.21.15.103:0/2093215509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc81018b0 0x7facc8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 --2- 172.21.15.103:0/2093215509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc81078b0 0x7facc8107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 -- 172.21.15.103:0/2093215509 >> 172.21.15.103:0/2093215509 conn(0x7facc80fd5f0 msgr2=0x7facc80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 -- 172.21.15.103:0/2093215509 shutdown_connections 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.382+0000 7facccc25640 1 -- 172.21.15.103:0/2093215509 wait complete. 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 Processor -- start 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 -- start start 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc81018b0 0x7facc81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc8102250 0x7facc8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc81078b0 0x7facc819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7facc8111b60 con 0x7facc81018b0 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7facc81119e0 con 0x7facc81078b0 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facccc25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7facc8111ce0 con 0x7facc8102250 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facc659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc81018b0 0x7facc81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facc659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc81018b0 0x7facc81980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43972/0 (socket says 172.21.15.103:43972) 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facc659c640 1 -- 172.21.15.103:0/1648205828 learned_addr learned my addr 172.21.15.103:0/1648205828 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.383+0000 7facc6d9d640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc81078b0 0x7facc819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc5d9b640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc8102250 0x7facc8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc659c640 1 -- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc8102250 msgr2=0x7facc8198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc659c640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc8102250 0x7facc8198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc659c640 1 -- 172.21.15.103:0/1648205828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc81078b0 msgr2=0x7facc819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc659c640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc81078b0 0x7facc819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc659c640 1 -- 172.21.15.103:0/1648205828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc819d070 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc5d9b640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facc8102250 0x7facc8198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facc659c640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc81018b0 0x7facc81980c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facb800d950 tx=0x7facb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8017070 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facb8009c30 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc819d360 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb800f620 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.384+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81a4c40 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.385+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facc80626b0 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.386+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7facb80109a0 con 0x7facc81018b0 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.387+0000 7facb77fe640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7facb00a2c10 0x7facb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.387+0000 7facc5d9b640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7facb00a2c10 0x7facb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.387+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7facb80cef90 con 0x7facc81018b0 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.387+0000 7facb77fe640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] conn(0x7facb00ac460 0x7facb00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.387+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 --> [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facb8005600 con 0x7facb00ac460 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.387+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7facb80cf470 con 0x7facc81018b0 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.388+0000 7facc5d9b640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7facb00a2c10 0x7facb00a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7facc80ffc50 tx=0x7facbc005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.388+0000 7facc6d9d640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] conn(0x7facb00ac460 0x7facb00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.388+0000 7facc6d9d640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] conn(0x7facb00ac460 0x7facb00ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.389+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== osd.8 v2:172.21.15.190:6816/2035395456 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facb8005600 con 0x7facb00ac460 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.402+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 --> [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7facc8062ab0 con 0x7facb00ac460 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facb77fe640 1 -- 172.21.15.103:0/1648205828 <== osd.8 v2:172.21.15.190:6816/2035395456 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7facc8062ab0 con 0x7facb00ac460 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] conn(0x7facb00ac460 msgr2=0x7facb00ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facccc25640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] conn(0x7facb00ac460 0x7facb00ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7facb00a2c10 msgr2=0x7facb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facccc25640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7facb00a2c10 0x7facb00a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7facc80ffc50 tx=0x7facbc005e50 comp rx=0 tx=0).stop 2024-08-19T01:33:59.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc81018b0 msgr2=0x7facc81980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facccc25640 1 --2- 172.21.15.103:0/1648205828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facc81018b0 0x7facc81980c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facb800d950 tx=0x7facb800de10 comp rx=0 tx=0).stop 2024-08-19T01:33:59.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.403+0000 7facc6d9d640 1 -- 172.21.15.103:0/1648205828 reap_dead start 2024-08-19T01:33:59.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.404+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 shutdown_connections 2024-08-19T01:33:59.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.404+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 >> 172.21.15.103:0/1648205828 conn(0x7facc80fd5f0 msgr2=0x7facc80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:59.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.404+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 shutdown_connections 2024-08-19T01:33:59.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.404+0000 7facccc25640 1 -- 172.21.15.103:0/1648205828 wait complete. 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 -- 172.21.15.103:0/2044329744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168102c20 msgr2=0x7f816810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 --2- 172.21.15.103:0/2044329744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168102c20 0x7f816810d770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8160009ca0 tx=0x7f816002a950 comp rx=0 tx=0).stop 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 -- 172.21.15.103:0/2044329744 shutdown_connections 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 --2- 172.21.15.103:0/2044329744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168102c20 0x7f816810d770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 --2- 172.21.15.103:0/2044329744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81681022a0 0x7f81681026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 --2- 172.21.15.103:0/2044329744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8168100b50 0x7f8168100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 -- 172.21.15.103:0/2044329744 >> 172.21.15.103:0/2044329744 conn(0x7f81680fc500 msgr2=0x7f81680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 -- 172.21.15.103:0/2044329744 shutdown_connections 2024-08-19T01:33:59.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 -- 172.21.15.103:0/2044329744 wait complete. 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 Processor -- start 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.871+0000 7f816d3f5640 1 -- start start 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168100b50 0x7f8168198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81681022a0 0x7f81681986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8168102c20 0x7f816819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110bb0 con 0x7f8168100b50 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110a30 con 0x7f8168102c20 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110d30 con 0x7f81681022a0 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168100b50 0x7f8168198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168100b50 0x7f8168198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43984/0 (socket says 172.21.15.103:43984) 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 -- 172.21.15.103:0/664721973 learned_addr learned my addr 172.21.15.103:0/664721973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f815ffff640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81681022a0 0x7f81681986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f81677fe640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8168102c20 0x7f816819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 -- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81681022a0 msgr2=0x7f81681986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81681022a0 0x7f81681986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 -- 172.21.15.103:0/664721973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8168102c20 msgr2=0x7f816819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8168102c20 0x7f816819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 -- 172.21.15.103:0/664721973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816819d110 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f815ffff640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81681022a0 0x7f81681986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8166ffd640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168100b50 0x7f8168198160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f815800e9f0 tx=0x7f815800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815800cd80 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81580054d0 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816819d400 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158010490 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.872+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a4ce0 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.873+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81680626b0 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.874+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8158005020 con 0x7f8168100b50 2024-08-19T01:33:59.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f8164ff9640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f81540a2c90 0x7f81540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f815ffff640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f81540a2c90 0x7f81540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f81580cf460 con 0x7f8168100b50 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f8164ff9640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] conn(0x7f81540ac4e0 0x7f81540ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 --> [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81540aefb0 con 0x7f81540ac4e0 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f81580d6050 con 0x7f8168100b50 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f815ffff640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f81540a2c90 0x7f81540a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81680ffce0 tx=0x7f8150005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.875+0000 7f81677fe640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] conn(0x7f81540ac4e0 0x7f81540ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:33:59.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.876+0000 7f81677fe640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] conn(0x7f81540ac4e0 0x7f81540ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.877+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== osd.2 v2:172.21.15.190:6800/2941836968 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81540aefb0 con 0x7f81540ac4e0 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.890+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 --> [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8168062ab0 con 0x7f81540ac4e0 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f8164ff9640 1 -- 172.21.15.103:0/664721973 <== osd.2 v2:172.21.15.190:6800/2941836968 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8168062ab0 con 0x7f81540ac4e0 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] conn(0x7f81540ac4e0 msgr2=0x7f81540ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] conn(0x7f81540ac4e0 0x7f81540ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f81540a2c90 msgr2=0x7f81540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f81540a2c90 0x7f81540a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81680ffce0 tx=0x7f8150005f90 comp rx=0 tx=0).stop 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168100b50 msgr2=0x7f8168198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 --2- 172.21.15.103:0/664721973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8168100b50 0x7f8168198160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f815800e9f0 tx=0x7f815800edf0 comp rx=0 tx=0).stop 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f81677fe640 1 -- 172.21.15.103:0/664721973 reap_dead start 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 shutdown_connections 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 >> 172.21.15.103:0/664721973 conn(0x7f81680fc500 msgr2=0x7f8168069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:33:59.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 shutdown_connections 2024-08-19T01:33:59.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:33:59.891+0000 7f816d3f5640 1 -- 172.21.15.103:0/664721973 wait complete. 2024-08-19T01:34:00.002 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:33:59 smithi103 ceph-mon[18436]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-08-19T01:34:00.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:33:59 smithi190 ceph-mon[22032]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-08-19T01:34:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:33:59 smithi146 ceph-mon[22098]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-08-19T01:34:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-19T01:34:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-08-19T01:34:01.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]: dispatch 2024-08-19T01:34:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-08-19T01:34:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-08-19T01:34:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-08-19T01:34:01.055 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:00 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-08-19T01:34:01.124 INFO:teuthology.orchestra.run.smithi103.stdout:244813135906 2024-08-19T01:34:01.124 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-19T01:34:01.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-19T01:34:01.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-08-19T01:34:01.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]: dispatch 2024-08-19T01:34:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-08-19T01:34:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-08-19T01:34:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-08-19T01:34:01.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:00 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-08-19T01:34:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-19T01:34:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-08-19T01:34:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]: dispatch 2024-08-19T01:34:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-08-19T01:34:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-08-19T01:34:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-08-19T01:34:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:00 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-08-19T01:34:01.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 -- 172.21.15.103:0/3679686215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc0ffe30 msgr2=0x7f98cc100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 --2- 172.21.15.103:0/3679686215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc0ffe30 0x7f98cc100290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009900 tx=0x7f98c002a410 comp rx=0 tx=0).stop 2024-08-19T01:34:01.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 -- 172.21.15.103:0/3679686215 shutdown_connections 2024-08-19T01:34:01.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 --2- 172.21.15.103:0/3679686215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc1007d0 0x7f98cc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 --2- 172.21.15.103:0/3679686215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc0ffe30 0x7f98cc100290 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 --2- 172.21.15.103:0/3679686215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98cc1068b0 0x7f98cc106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 -- 172.21.15.103:0/3679686215 >> 172.21.15.103:0/3679686215 conn(0x7f98cc076f60 msgr2=0x7f98cc0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 -- 172.21.15.103:0/3679686215 shutdown_connections 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 -- 172.21.15.103:0/3679686215 wait complete. 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.278+0000 7f98d17af640 1 Processor -- start 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98d17af640 1 -- start start 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98d17af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98cc0ffe30 0x7f98cc198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98d17af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc1007d0 0x7f98cc198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98d17af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc1068b0 0x7f98cc19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98d17af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc111c50 con 0x7f98cc1068b0 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98d17af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc111ad0 con 0x7f98cc1007d0 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98d17af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc111dd0 con 0x7f98cc0ffe30 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc1068b0 0x7f98cc19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc1068b0 0x7f98cc19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44022/0 (socket says 172.21.15.103:44022) 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 -- 172.21.15.103:0/2135529560 learned_addr learned my addr 172.21.15.103:0/2135529560 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:01.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98caffd640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98cc0ffe30 0x7f98cc198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98ca7fc640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc1007d0 0x7f98cc198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 -- 172.21.15.103:0/2135529560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98cc0ffe30 msgr2=0x7f98cc198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98cc0ffe30 0x7f98cc198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 -- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc1007d0 msgr2=0x7f98cc198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc1007d0 0x7f98cc198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 -- 172.21.15.103:0/2135529560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19d0e0 con 0x7f98cc1068b0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98ca7fc640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98cc1007d0 0x7f98cc198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.279+0000 7f98cb7fe640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc1068b0 0x7f98cc19ca00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98c400ccb0 tx=0x7f98c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.280+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4003970 con 0x7f98cc1068b0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.280+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc19d3d0 con 0x7f98cc1068b0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.280+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c4004030 con 0x7f98cc1068b0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.280+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c400f6b0 con 0x7f98cc1068b0 2024-08-19T01:34:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.280+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a4cb0 con 0x7f98cc1068b0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.281+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98cc062630 con 0x7f98cc1068b0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.282+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f98c40041e0 con 0x7f98cc1068b0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.282+0000 7f98b3fff640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f98b40a2c10 0x7f98b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.283+0000 7f98caffd640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f98b40a2c10 0x7f98b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.283+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8190+0+0 (secure 0 0 0) 0x7f98c40cf6f0 con 0x7f98cc1068b0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.283+0000 7f98b3fff640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] conn(0x7f98b40ac740 0x7f98b40aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.283+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 --> [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98c4012ec0 con 0x7f98b40ac740 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.283+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f98c40cfb40 con 0x7f98cc1068b0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.283+0000 7f98caffd640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f98b40a2c10 0x7f98b40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009890 tx=0x7f98bc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.283+0000 7f98ca7fc640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] conn(0x7f98b40ac740 0x7f98b40aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.284+0000 7f98ca7fc640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] conn(0x7f98b40ac740 0x7f98b40aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.285+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== osd.7 v2:172.21.15.146:6816/2549773085 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98c4012ec0 con 0x7f98b40ac740 2024-08-19T01:34:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.298+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 --> [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98cc062ab0 con 0x7f98b40ac740 2024-08-19T01:34:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98b3fff640 1 -- 172.21.15.103:0/2135529560 <== osd.7 v2:172.21.15.146:6816/2549773085 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f98cc062ab0 con 0x7f98b40ac740 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] conn(0x7f98b40ac740 msgr2=0x7f98b40aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] conn(0x7f98b40ac740 0x7f98b40aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f98b40a2c10 msgr2=0x7f98b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f98b40a2c10 0x7f98b40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009890 tx=0x7f98bc008040 comp rx=0 tx=0).stop 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc1068b0 msgr2=0x7f98cc19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 --2- 172.21.15.103:0/2135529560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98cc1068b0 0x7f98cc19ca00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f98c400ccb0 tx=0x7f98c400a620 comp rx=0 tx=0).stop 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98cb7fe640 1 -- 172.21.15.103:0/2135529560 reap_dead start 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 shutdown_connections 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.299+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 >> 172.21.15.103:0/2135529560 conn(0x7f98cc076f60 msgr2=0x7f98cc104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.300+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 shutdown_connections 2024-08-19T01:34:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:01.300+0000 7f98d17af640 1 -- 172.21.15.103:0/2135529560 wait complete. 2024-08-19T01:34:01.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.519+0000 7fd6d8582640 1 -- 172.21.15.103:0/3665834129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d0103390 msgr2=0x7fd6d010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.519+0000 7fd6d8582640 1 --2- 172.21.15.103:0/3665834129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d0103390 0x7fd6d010b880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009900 tx=0x7fd6c002a480 comp rx=0 tx=0).stop 2024-08-19T01:34:01.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.519+0000 7fd6d8582640 1 -- 172.21.15.103:0/3665834129 shutdown_connections 2024-08-19T01:34:01.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.519+0000 7fd6d8582640 1 --2- 172.21.15.103:0/3665834129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d010bea0 0x7fd6d010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.519+0000 7fd6d8582640 1 --2- 172.21.15.103:0/3665834129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d0103390 0x7fd6d010b880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.519+0000 7fd6d8582640 1 --2- 172.21.15.103:0/3665834129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6d0102a70 0x7fd6d0102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 -- 172.21.15.103:0/3665834129 >> 172.21.15.103:0/3665834129 conn(0x7fd6d00fc4d0 msgr2=0x7fd6d00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 -- 172.21.15.103:0/3665834129 shutdown_connections 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 -- 172.21.15.103:0/3665834129 wait complete. 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 Processor -- start 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 -- start start 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6d0102a70 0x7fd6d01006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d0103390 0x7fd6d0100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d010bea0 0x7fd6d0101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0111b50 con 0x7fd6d010bea0 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01119d0 con 0x7fd6d0102a70 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d8582640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0111cd0 con 0x7fd6d0103390 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.520+0000 7fd6d6b1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d010bea0 0x7fd6d0101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d010bea0 0x7fd6d0101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44010/0 (socket says 172.21.15.103:44010) 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 -- 172.21.15.103:0/1561364515 learned_addr learned my addr 172.21.15.103:0/1561364515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:01.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d5b1d640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d0103390 0x7fd6d0100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d631e640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6d0102a70 0x7fd6d01006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 -- 172.21.15.103:0/1561364515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d0103390 msgr2=0x7fd6d0100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6d0103390 0x7fd6d0100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 -- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6d0102a70 msgr2=0x7fd6d01006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6d0102a70 0x7fd6d01006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 -- 172.21.15.103:0/1561364515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01a6510 con 0x7fd6d010bea0 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d631e640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6d0102a70 0x7fd6d01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:01.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d6b1f640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d010bea0 0x7fd6d0101150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c970 tx=0x7fd6c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8007bb0 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6c8007d50 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01a6800 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8018700 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.521+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a6cc0 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.522+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6d0062630 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.523+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd6c800fa20 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6c77fe640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd6bc0a2c10 0x7fd6bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6d631e640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd6bc0a2c10 0x7fd6bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fd6c80cf3d0 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6c77fe640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] conn(0x7fd6bc0ac460 0x7fd6bc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 --> [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6bc0aef30 con 0x7fd6bc0ac460 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fd6c80cf8b0 con 0x7fd6d010bea0 2024-08-19T01:34:01.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6d631e640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd6bc0a2c10 0x7fd6bc0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc005fd0 tx=0x7fd6cc0093e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.524+0000 7fd6d5b1d640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] conn(0x7fd6bc0ac460 0x7fd6bc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.525+0000 7fd6d5b1d640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] conn(0x7fd6bc0ac460 0x7fd6bc0ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.526+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== osd.10 v2:172.21.15.146:6824/1357662473 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6bc0aef30 con 0x7fd6bc0ac460 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.539+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 --> [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6d0062ab0 con 0x7fd6bc0ac460 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.539+0000 7fd6c77fe640 1 -- 172.21.15.103:0/1561364515 <== osd.10 v2:172.21.15.146:6824/1357662473 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd6d0062ab0 con 0x7fd6bc0ac460 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] conn(0x7fd6bc0ac460 msgr2=0x7fd6bc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] conn(0x7fd6bc0ac460 0x7fd6bc0ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd6bc0a2c10 msgr2=0x7fd6bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd6bc0a2c10 0x7fd6bc0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc005fd0 tx=0x7fd6cc0093e0 comp rx=0 tx=0).stop 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d010bea0 msgr2=0x7fd6d0101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 --2- 172.21.15.103:0/1561364515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6d010bea0 0x7fd6d0101150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c970 tx=0x7fd6c800ce30 comp rx=0 tx=0).stop 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d6b1f640 1 -- 172.21.15.103:0/1561364515 reap_dead start 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 shutdown_connections 2024-08-19T01:34:01.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 >> 172.21.15.103:0/1561364515 conn(0x7fd6d00fc4d0 msgr2=0x7fd6d00fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:01.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 shutdown_connections 2024-08-19T01:34:01.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:00.540+0000 7fd6d8582640 1 -- 172.21.15.103:0/1561364515 wait complete. 2024-08-19T01:34:01.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-08-19T01:34:01.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-19T01:34:01.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-08-19T01:34:01.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]': finished 2024-08-19T01:34:01.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-08-19T01:34:01.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-08-19T01:34:01.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-19T01:34:01.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-08-19T01:34:01.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:01 smithi103 ceph-mon[18436]: osdmap e85: 12 total, 12 up, 12 in 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]': finished 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-08-19T01:34:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:01 smithi146 ceph-mon[22098]: osdmap e85: 12 total, 12 up, 12 in 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11, 2, 1]}]': finished 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-08-19T01:34:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:01 smithi190 ceph-mon[22032]: osdmap e85: 12 total, 12 up, 12 in 2024-08-19T01:34:02.689 INFO:teuthology.orchestra.run.smithi103.stdout:300647710745 2024-08-19T01:34:02.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.852+0000 7f24fbb98640 1 -- 172.21.15.103:0/3397487831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4106ed0 msgr2=0x7f24f410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.852+0000 7f24fbb98640 1 --2- 172.21.15.103:0/3397487831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4106ed0 0x7f24f410d6b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009ca0 tx=0x7f24ec02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- 172.21.15.103:0/3397487831 shutdown_connections 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 --2- 172.21.15.103:0/3397487831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4106ed0 0x7f24f410d6b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 --2- 172.21.15.103:0/3397487831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4106530 0x7f24f4106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 --2- 172.21.15.103:0/3397487831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4105880 0x7f24f4105c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- 172.21.15.103:0/3397487831 >> 172.21.15.103:0/3397487831 conn(0x7f24f4076f40 msgr2=0x7f24f4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- 172.21.15.103:0/3397487831 shutdown_connections 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- 172.21.15.103:0/3397487831 wait complete. 2024-08-19T01:34:02.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 Processor -- start 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- start start 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4105880 0x7f24f4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4106530 0x7f24f41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4106ed0 0x7f24f41a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4110aa0 con 0x7f24f4105880 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4110920 con 0x7f24f4106530 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.853+0000 7f24fbb98640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4110c20 con 0x7f24f4106ed0 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4105880 0x7f24f4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4105880 0x7f24f4198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45976/0 (socket says 172.21.15.103:45976) 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 -- 172.21.15.103:0/277445730 learned_addr learned my addr 172.21.15.103:0/277445730 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9133640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4106530 0x7f24f41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:02.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24fa135640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4106ed0 0x7f24f41a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 -- 172.21.15.103:0/277445730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4106ed0 msgr2=0x7f24f41a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24f4106ed0 0x7f24f41a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 -- 172.21.15.103:0/277445730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4106530 msgr2=0x7f24f41993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f4106530 0x7f24f41993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 -- 172.21.15.103:0/277445730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f41a4450 con 0x7f24f4105880 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24f9934640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4105880 0x7f24f4198e90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c920 tx=0x7f24f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0007c70 con 0x7f24f4105880 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24f0007e10 con 0x7f24f4105880 2024-08-19T01:34:02.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0018820 con 0x7f24f4105880 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f41a4620 con 0x7f24f4105880 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.854+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f41a4b60 con 0x7f24f4105880 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.855+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24f40ff960 con 0x7f24f4105880 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.857+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f24f00189c0 con 0x7f24f4105880 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.857+0000 7f24eaffd640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f24e00a2c90 0x7f24e00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.857+0000 7f24f9133640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f24e00a2c90 0x7f24e00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.858+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f24f00cf7a0 con 0x7f24f4105880 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.858+0000 7f24eaffd640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] conn(0x7f24e00ac7c0 0x7f24e00aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.858+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 --> [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24f0015e70 con 0x7f24e00ac7c0 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.858+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f24f00cfbf0 con 0x7f24f4105880 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.858+0000 7f24f9133640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f24e00a2c90 0x7f24e00a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24e4005fd0 tx=0x7f24e400b510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.858+0000 7f24fa135640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] conn(0x7f24e00ac7c0 0x7f24e00aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.858+0000 7f24fa135640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] conn(0x7f24e00ac7c0 0x7f24e00aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.859+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== osd.0 v2:172.21.15.103:6802/2060644281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24f0015e70 con 0x7f24e00ac7c0 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.872+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 --> [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24f4069f10 con 0x7f24e00ac7c0 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24eaffd640 1 -- 172.21.15.103:0/277445730 <== osd.0 v2:172.21.15.103:6802/2060644281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f24f4069f10 con 0x7f24e00ac7c0 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] conn(0x7f24e00ac7c0 msgr2=0x7f24e00aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] conn(0x7f24e00ac7c0 0x7f24e00aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f24e00a2c90 msgr2=0x7f24e00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f24e00a2c90 0x7f24e00a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24e4005fd0 tx=0x7f24e400b510 comp rx=0 tx=0).stop 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4105880 msgr2=0x7f24f4198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 --2- 172.21.15.103:0/277445730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24f4105880 0x7f24f4198e90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c920 tx=0x7f24f000cde0 comp rx=0 tx=0).stop 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fa135640 1 -- 172.21.15.103:0/277445730 reap_dead start 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 shutdown_connections 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 >> 172.21.15.103:0/277445730 conn(0x7f24f4076f40 msgr2=0x7f24f40fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 shutdown_connections 2024-08-19T01:34:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:02.873+0000 7f24fbb98640 1 -- 172.21.15.103:0/277445730 wait complete. 2024-08-19T01:34:03.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:02 smithi103 ceph-mon[18436]: osdmap e86: 12 total, 12 up, 12 in 2024-08-19T01:34:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:02 smithi146 ceph-mon[22098]: osdmap e86: 12 total, 12 up, 12 in 2024-08-19T01:34:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:02 smithi190 ceph-mon[22032]: osdmap e86: 12 total, 12 up, 12 in 2024-08-19T01:34:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 -- 172.21.15.103:0/2854596782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a0101150 msgr2=0x7f94a010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 --2- 172.21.15.103:0/2854596782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a0101150 0x7f94a010d680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9498009cd0 tx=0x7f949802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 -- 172.21.15.103:0/2854596782 shutdown_connections 2024-08-19T01:34:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 --2- 172.21.15.103:0/2854596782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a0101150 0x7f94a010d680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 --2- 172.21.15.103:0/2854596782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a01007b0 0x7f94a0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 --2- 172.21.15.103:0/2854596782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94a01067b0 0x7f94a0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 -- 172.21.15.103:0/2854596782 >> 172.21.15.103:0/2854596782 conn(0x7f94a00fc4d0 msgr2=0x7f94a00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 -- 172.21.15.103:0/2854596782 shutdown_connections 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.508+0000 7f94a5f51640 1 -- 172.21.15.103:0/2854596782 wait complete. 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 Processor -- start 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 -- start start 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a01007b0 0x7f94a01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94a0101150 0x7f94a0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a01067b0 0x7f94a019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0111b80 con 0x7f94a01067b0 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0111a00 con 0x7f94a0101150 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f94a5f51640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0111d00 con 0x7f94a01007b0 2024-08-19T01:34:03.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a01067b0 0x7f94a019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a01067b0 0x7f94a019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45994/0 (socket says 172.21.15.103:45994) 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 -- 172.21.15.103:0/4055214673 learned_addr learned my addr 172.21.15.103:0/4055214673 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949f7fe640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a01007b0 0x7f94a01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f9497fff640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94a0101150 0x7f94a0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 -- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a01007b0 msgr2=0x7f94a01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a01007b0 0x7f94a01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 -- 172.21.15.103:0/4055214673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94a0101150 msgr2=0x7f94a0198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94a0101150 0x7f94a0198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949ffff640 1 -- 172.21.15.103:0/4055214673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a019d070 con 0x7f94a01067b0 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f949f7fe640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94a01007b0 0x7f94a01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.509+0000 7f9497fff640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94a0101150 0x7f94a0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.510+0000 7f949ffff640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a01067b0 0x7f94a019c990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9498005cc0 tx=0x7f9498003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.510+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498004240 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.510+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a019d300 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.510+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94980305a0 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.510+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94980466c0 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.510+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a01a4c40 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.511+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94a00626b0 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.512+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9498030ac0 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.512+0000 7f949d7fa640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f948c0a2c10 0x7f948c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.513+0000 7f949f7fe640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f948c0a2c10 0x7f948c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.513+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f949802e030 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.513+0000 7f949d7fa640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] conn(0x7f948c0ac740 0x7f948c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.513+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 --> [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f948c0af2b0 con 0x7f948c0ac740 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.513+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f94980f0c10 con 0x7f94a01067b0 2024-08-19T01:34:03.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.513+0000 7f949f7fe640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f948c0a2c10 0x7f948c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f949000baa0 tx=0x7f9490005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.513+0000 7f9497fff640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] conn(0x7f948c0ac740 0x7f948c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.514+0000 7f9497fff640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] conn(0x7f948c0ac740 0x7f948c0aebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.514+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== osd.5 v2:172.21.15.190:6808/1607717439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f948c0af2b0 con 0x7f948c0ac740 2024-08-19T01:34:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.528+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 --> [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94a0062ab0 con 0x7f948c0ac740 2024-08-19T01:34:03.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.528+0000 7f949d7fa640 1 -- 172.21.15.103:0/4055214673 <== osd.5 v2:172.21.15.190:6808/1607717439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f94a0062ab0 con 0x7f948c0ac740 2024-08-19T01:34:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] conn(0x7f948c0ac740 msgr2=0x7f948c0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] conn(0x7f948c0ac740 0x7f948c0aebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f948c0a2c10 msgr2=0x7f948c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f948c0a2c10 0x7f948c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f949000baa0 tx=0x7f9490005d90 comp rx=0 tx=0).stop 2024-08-19T01:34:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a01067b0 msgr2=0x7f94a019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 --2- 172.21.15.103:0/4055214673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a01067b0 0x7f94a019c990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9498005cc0 tx=0x7f9498003ba0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f949ffff640 1 -- 172.21.15.103:0/4055214673 reap_dead start 2024-08-19T01:34:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 shutdown_connections 2024-08-19T01:34:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 >> 172.21.15.103:0/4055214673 conn(0x7f94a00fc4d0 msgr2=0x7f94a00fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 shutdown_connections 2024-08-19T01:34:03.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.529+0000 7f94a5f51640 1 -- 172.21.15.103:0/4055214673 wait complete. 2024-08-19T01:34:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 -- 172.21.15.103:0/4244548963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720100790 msgr2=0x7f1720100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 --2- 172.21.15.103:0/4244548963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720100790 0x7f1720100bf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1710009930 tx=0x7f171002a420 comp rx=0 tx=0).stop 2024-08-19T01:34:03.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 -- 172.21.15.103:0/4244548963 shutdown_connections 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 --2- 172.21.15.103:0/4244548963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1720101130 0x7f172010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 --2- 172.21.15.103:0/4244548963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720100790 0x7f1720100bf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 --2- 172.21.15.103:0/4244548963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1720106790 0x7f1720106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 -- 172.21.15.103:0/4244548963 >> 172.21.15.103:0/4244548963 conn(0x7f17200fc4b0 msgr2=0x7f17200fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 -- 172.21.15.103:0/4244548963 shutdown_connections 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.917+0000 7f17278e3640 1 -- 172.21.15.103:0/4244548963 wait complete. 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 Processor -- start 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 -- start start 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1720100790 0x7f1720196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1720101130 0x7f1720197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720106790 0x7f172019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1720110a40 con 0x7f1720106790 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17201108c0 con 0x7f1720100790 2024-08-19T01:34:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f17278e3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1720110bc0 con 0x7f1720101130 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f1725e80640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720106790 0x7f172019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f1725e80640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720106790 0x7f172019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46010/0 (socket says 172.21.15.103:46010) 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f1725e80640 1 -- 172.21.15.103:0/1488088707 learned_addr learned my addr 172.21.15.103:0/1488088707 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f1724e7e640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1720101130 0x7f1720197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.918+0000 7f172567f640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1720100790 0x7f1720196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f1725e80640 1 -- 172.21.15.103:0/1488088707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1720101130 msgr2=0x7f1720197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f1725e80640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1720101130 0x7f1720197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f1725e80640 1 -- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1720100790 msgr2=0x7f1720196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f1725e80640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1720100790 0x7f1720196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f1725e80640 1 -- 172.21.15.103:0/1488088707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172019bf90 con 0x7f1720106790 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f172567f640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1720100790 0x7f1720196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f1725e80640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720106790 0x7f172019b8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f171800eb10 tx=0x7f171800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718004b20 con 0x7f1720106790 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1718004cc0 con 0x7f1720106790 2024-08-19T01:34:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172019c280 con 0x7f1720106790 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718010690 con 0x7f1720106790 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.919+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172006a4c0 con 0x7f1720106790 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.920+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f172006aad0 con 0x7f1720106790 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.921+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1718005020 con 0x7f1720106790 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f17167fc640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f170c0a2c90 0x7f170c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f172567f640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f170c0a2c90 0x7f170c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f17180cfe90 con 0x7f1720106790 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f17167fc640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] conn(0x7f170c0ac7c0 0x7f170c0aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 --> [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f170c0af330 con 0x7f170c0ac7c0 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f17180d02e0 con 0x7f1720106790 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f172567f640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f170c0a2c90 0x7f170c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.922+0000 7f1724e7e640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] conn(0x7f170c0ac7c0 0x7f170c0aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.923+0000 7f1724e7e640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] conn(0x7f170c0ac7c0 0x7f170c0aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.924+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== osd.1 v2:172.21.15.146:6800/351745310 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f170c0af330 con 0x7f170c0ac7c0 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.937+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 --> [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f172006af50 con 0x7f170c0ac7c0 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17167fc640 1 -- 172.21.15.103:0/1488088707 <== osd.1 v2:172.21.15.146:6800/351745310 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f172006af50 con 0x7f170c0ac7c0 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] conn(0x7f170c0ac7c0 msgr2=0x7f170c0aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] conn(0x7f170c0ac7c0 0x7f170c0aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f170c0a2c90 msgr2=0x7f170c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f170c0a2c90 0x7f170c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c005850 comp rx=0 tx=0).stop 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720106790 msgr2=0x7f172019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 --2- 172.21.15.103:0/1488088707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1720106790 0x7f172019b8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f171800eb10 tx=0x7f171800c2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:03.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f1725e80640 1 -- 172.21.15.103:0/1488088707 reap_dead start 2024-08-19T01:34:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 shutdown_connections 2024-08-19T01:34:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 >> 172.21.15.103:0/1488088707 conn(0x7f17200fc4b0 msgr2=0x7f17200fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 shutdown_connections 2024-08-19T01:34:03.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:03.938+0000 7f17278e3640 1 -- 172.21.15.103:0/1488088707 wait complete. 2024-08-19T01:34:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:03 smithi146 ceph-mon[22098]: pgmap v287: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 383 B/s wr, 12 op/s 2024-08-19T01:34:04.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:03 smithi103 ceph-mon[18436]: pgmap v287: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 383 B/s wr, 12 op/s 2024-08-19T01:34:04.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:03 smithi190 ceph-mon[22032]: pgmap v287: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 383 B/s wr, 12 op/s 2024-08-19T01:34:04.758 INFO:teuthology.orchestra.run.smithi103.stdout:34359738440 2024-08-19T01:34:04.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-19T01:34:05.168 INFO:teuthology.orchestra.run.smithi103.stdout:55834574915 2024-08-19T01:34:05.168 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-19T01:34:05.936 INFO:teuthology.orchestra.run.smithi103.stdout:214748364839 2024-08-19T01:34:05.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-19T01:34:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:05 smithi146 ceph-mon[22098]: pgmap v288: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s 2024-08-19T01:34:06.233 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:05 smithi103 ceph-mon[18436]: pgmap v288: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s 2024-08-19T01:34:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:05 smithi190 ceph-mon[22032]: pgmap v288: 97 pgs: 3 peering, 94 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s 2024-08-19T01:34:07.176 INFO:teuthology.orchestra.run.smithi103.stdout:163208757297 2024-08-19T01:34:07.177 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-19T01:34:07.199 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:06 smithi103 ceph-mon[18436]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-08-19T01:34:07.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:06 smithi146 ceph-mon[22098]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-08-19T01:34:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:06 smithi190 ceph-mon[22032]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-08-19T01:34:07.644 INFO:teuthology.orchestra.run.smithi103.stdout:77309411390 2024-08-19T01:34:07.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-19T01:34:07.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 -- 172.21.15.103:0/1958087393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f01018b0 msgr2=0x7f13f0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 --2- 172.21.15.103:0/1958087393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f01018b0 0x7f13f0101d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009930 tx=0x7f13e002a420 comp rx=0 tx=0).stop 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 -- 172.21.15.103:0/1958087393 shutdown_connections 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 --2- 172.21.15.103:0/1958087393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f0102250 0x7f13f010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 --2- 172.21.15.103:0/1958087393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f01018b0 0x7f13f0101d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 --2- 172.21.15.103:0/1958087393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f01078b0 0x7f13f0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 -- 172.21.15.103:0/1958087393 >> 172.21.15.103:0/1958087393 conn(0x7f13f00fd5f0 msgr2=0x7f13f00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 -- 172.21.15.103:0/1958087393 shutdown_connections 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 -- 172.21.15.103:0/1958087393 wait complete. 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.925+0000 7f13f87cd640 1 Processor -- start 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f87cd640 1 -- start start 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f87cd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f01018b0 0x7f13f0074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f87cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f0102250 0x7f13f0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f87cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f01078b0 0x7f13f0074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f87cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0111b80 con 0x7f13f0102250 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f87cd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0111a00 con 0x7f13f01018b0 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f87cd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0111d00 con 0x7f13f01078b0 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f0102250 0x7f13f0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f0102250 0x7f13f0074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46034/0 (socket says 172.21.15.103:46034) 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 -- 172.21.15.103:0/456210200 learned_addr learned my addr 172.21.15.103:0/456210200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f6569640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f01018b0 0x7f13f0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f6d6a640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f01078b0 0x7f13f0074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 -- 172.21.15.103:0/456210200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f01078b0 msgr2=0x7f13f0074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13f01078b0 0x7f13f0074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 -- 172.21.15.103:0/456210200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f01018b0 msgr2=0x7f13f0074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f13f01018b0 0x7f13f0074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 -- 172.21.15.103:0/456210200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f01a6660 con 0x7f13f0102250 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.926+0000 7f13f5d68640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f0102250 0x7f13f0074760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f13e0004770 tx=0x7f13e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.927+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e003f070 con 0x7f13f0102250 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.927+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f0197f00 con 0x7f13f0102250 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.927+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13e003b460 con 0x7f13f0102250 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.927+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0035070 con 0x7f13f0102250 2024-08-19T01:34:07.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.927+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f01983e0 con 0x7f13f0102250 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.928+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13f0062630 con 0x7f13f0102250 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.929+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f13e003b610 con 0x7f13f0102250 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.929+0000 7f13e77fe640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13dc0a2c10 0x7f13dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13f6569640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13dc0a2c10 0x7f13dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f13e00efc00 con 0x7f13f0102250 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13e77fe640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] conn(0x7f13dc0ac740 0x7f13dc0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 --> [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13e0001d30 con 0x7f13dc0ac740 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f13e00f0050 con 0x7f13f0102250 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13f6d6a640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] conn(0x7f13dc0ac740 0x7f13dc0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13f6569640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13dc0a2c10 0x7f13dc0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13ec005fd0 tx=0x7f13ec009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.930+0000 7f13f6d6a640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] conn(0x7f13dc0ac740 0x7f13dc0aebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.931+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== osd.6 v2:172.21.15.103:6818/3250967398 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13e0001d30 con 0x7f13dc0ac740 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.944+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 --> [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13f0062a30 con 0x7f13dc0ac740 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.944+0000 7f13e77fe640 1 -- 172.21.15.103:0/456210200 <== osd.6 v2:172.21.15.103:6818/3250967398 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f13f0062a30 con 0x7f13dc0ac740 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] conn(0x7f13dc0ac740 msgr2=0x7f13dc0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] conn(0x7f13dc0ac740 0x7f13dc0aebe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13dc0a2c10 msgr2=0x7f13dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13dc0a2c10 0x7f13dc0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13ec005fd0 tx=0x7f13ec009680 comp rx=0 tx=0).stop 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f0102250 msgr2=0x7f13f0074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:07.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 --2- 172.21.15.103:0/456210200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f0102250 0x7f13f0074760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f13e0004770 tx=0x7f13e00042f0 comp rx=0 tx=0).stop 2024-08-19T01:34:07.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f6d6a640 1 -- 172.21.15.103:0/456210200 reap_dead start 2024-08-19T01:34:07.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 shutdown_connections 2024-08-19T01:34:07.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 >> 172.21.15.103:0/456210200 conn(0x7f13f00fd5f0 msgr2=0x7f13f00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:07.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 shutdown_connections 2024-08-19T01:34:07.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:07.945+0000 7f13f87cd640 1 -- 172.21.15.103:0/456210200 wait complete. 2024-08-19T01:34:08.634 INFO:teuthology.orchestra.run.smithi103.stdout:193273528365 2024-08-19T01:34:08.635 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 -- 172.21.15.103:0/2562598984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 msgr2=0x7f2e4c10e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 --2- 172.21.15.103:0/2562598984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 0x7f2e4c10e4c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009ca0 tx=0x7f2e4402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 -- 172.21.15.103:0/2562598984 shutdown_connections 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 --2- 172.21.15.103:0/2562598984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 0x7f2e4c10e4c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 --2- 172.21.15.103:0/2562598984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e4c1034d0 0x7f2e4c10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 --2- 172.21.15.103:0/2562598984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c102b20 0x7f2e4c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 -- 172.21.15.103:0/2562598984 >> 172.21.15.103:0/2562598984 conn(0x7f2e4c0fc500 msgr2=0x7f2e4c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 -- 172.21.15.103:0/2562598984 shutdown_connections 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.882+0000 7f2e5436e640 1 -- 172.21.15.103:0/2562598984 wait complete. 2024-08-19T01:34:08.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 Processor -- start 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 -- start start 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e4c102b20 0x7f2e4c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c1034d0 0x7f2e4c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 0x7f2e4c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111db0 con 0x7f2e4c10c020 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111c30 con 0x7f2e4c102b20 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5436e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111f30 con 0x7f2e4c1034d0 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 0x7f2e4c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 0x7f2e4c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46050/0 (socket says 172.21.15.103:46050) 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 -- 172.21.15.103:0/968030129 learned_addr learned my addr 172.21.15.103:0/968030129 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e51909640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c1034d0 0x7f2e4c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5210a640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e4c102b20 0x7f2e4c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:08.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 -- 172.21.15.103:0/968030129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c1034d0 msgr2=0x7f2e4c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c1034d0 0x7f2e4c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 -- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e4c102b20 msgr2=0x7f2e4c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2e4c102b20 0x7f2e4c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.883+0000 7f2e5290b640 1 -- 172.21.15.103:0/968030129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c19d280 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.884+0000 7f2e5290b640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 0x7f2e4c19cba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e44033040 tx=0x7f2e44003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.884+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44004240 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.884+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e44003e10 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.884+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c19d510 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.884+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44030e10 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.884+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c1a4e50 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.885+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e4c062670 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.886+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2e440305e0 con 0x7f2e4c10c020 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.886+0000 7f2e437fe640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2e380a2c10 0x7f2e380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:08.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e5210a640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2e380a2c10 0x7f2e380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f2e440f07b0 con 0x7f2e4c10c020 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e437fe640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] conn(0x7f2e380ac740 0x7f2e380aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 --> [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e440020b0 con 0x7f2e380ac740 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f2e440f0c00 con 0x7f2e4c10c020 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e5210a640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2e380a2c10 0x7f2e380a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009890 tx=0x7f2e48008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e51909640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] conn(0x7f2e380ac740 0x7f2e380aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.887+0000 7f2e51909640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] conn(0x7f2e380ac740 0x7f2e380aebe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.888+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== osd.4 v2:172.21.15.146:6808/3000848263 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e440020b0 con 0x7f2e380ac740 2024-08-19T01:34:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.902+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 --> [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e4c062af0 con 0x7f2e380ac740 2024-08-19T01:34:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.902+0000 7f2e437fe640 1 -- 172.21.15.103:0/968030129 <== osd.4 v2:172.21.15.146:6808/3000848263 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2e4c062af0 con 0x7f2e380ac740 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] conn(0x7f2e380ac740 msgr2=0x7f2e380aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] conn(0x7f2e380ac740 0x7f2e380aebe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2e380a2c10 msgr2=0x7f2e380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2e380a2c10 0x7f2e380a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009890 tx=0x7f2e48008040 comp rx=0 tx=0).stop 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 msgr2=0x7f2e4c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 --2- 172.21.15.103:0/968030129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4c10c020 0x7f2e4c19cba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e44033040 tx=0x7f2e44003c20 comp rx=0 tx=0).stop 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5290b640 1 -- 172.21.15.103:0/968030129 reap_dead start 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 shutdown_connections 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 >> 172.21.15.103:0/968030129 conn(0x7f2e4c0fc500 msgr2=0x7f2e4c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 shutdown_connections 2024-08-19T01:34:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:08.903+0000 7f2e5436e640 1 -- 172.21.15.103:0/968030129 wait complete. 2024-08-19T01:34:09.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.090+0000 7f967196e640 1 -- 172.21.15.103:0/1368942705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 msgr2=0x7f966c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.090+0000 7f967196e640 1 --2- 172.21.15.103:0/1368942705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 0x7f966c1026e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9654009900 tx=0x7f965402a480 comp rx=0 tx=0).stop 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 -- 172.21.15.103:0/1368942705 shutdown_connections 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 --2- 172.21.15.103:0/1368942705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f966c102c20 0x7f966c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 --2- 172.21.15.103:0/1368942705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 0x7f966c1026e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 --2- 172.21.15.103:0/1368942705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f966c100b50 0x7f966c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 -- 172.21.15.103:0/1368942705 >> 172.21.15.103:0/1368942705 conn(0x7f966c0fc500 msgr2=0x7f966c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 -- 172.21.15.103:0/1368942705 shutdown_connections 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 -- 172.21.15.103:0/1368942705 wait complete. 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 Processor -- start 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.091+0000 7f967196e640 1 -- start start 2024-08-19T01:34:09.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f966c100b50 0x7f966c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 0x7f966c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f966c102c20 0x7f966c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f966c110ba0 con 0x7f966c1022a0 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f966c110a20 con 0x7f966c100b50 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f966c110d20 con 0x7f966c102c20 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 0x7f966c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 0x7f966c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46066/0 (socket says 172.21.15.103:46066) 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 -- 172.21.15.103:0/310168354 learned_addr learned my addr 172.21.15.103:0/310168354 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966affd640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f966c100b50 0x7f966c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966b7fe640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f966c102c20 0x7f966c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 -- 172.21.15.103:0/310168354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f966c102c20 msgr2=0x7f966c19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f966c102c20 0x7f966c19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 -- 172.21.15.103:0/310168354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f966c100b50 msgr2=0x7f966c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f966c100b50 0x7f966c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 -- 172.21.15.103:0/310168354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c19d220 con 0x7f966c1022a0 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966affd640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f966c100b50 0x7f966c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966b7fe640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f966c102c20 0x7f966c19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f966a7fc640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 0x7f966c1987b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9654006010 tx=0x7f96540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965403f030 con 0x7f966c1022a0 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9654003cb0 con 0x7f966c1022a0 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c19d4b0 con 0x7f966c1022a0 2024-08-19T01:34:09.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654035070 con 0x7f966c1022a0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.092+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c1a4df0 con 0x7f966c1022a0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.093+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f966c0626b0 con 0x7f966c1022a0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.095+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9654042030 con 0x7f966c1022a0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.095+0000 7f967096c640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96580a2c10 0x7f96580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.095+0000 7f966affd640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96580a2c10 0x7f96580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.096+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f96540efe40 con 0x7f966c1022a0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.096+0000 7f967096c640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] conn(0x7f96580ac740 0x7f96580aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.096+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 --> [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9654005640 con 0x7f96580ac740 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.096+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f96540f0290 con 0x7f966c1022a0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.096+0000 7f966b7fe640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] conn(0x7f96580ac740 0x7f96580aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.096+0000 7f966affd640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96580a2c10 0x7f96580a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9664005fd0 tx=0x7f96640093e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.096+0000 7f966b7fe640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] conn(0x7f96580ac740 0x7f96580aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.097+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== osd.9 v2:172.21.15.103:6826/1538313167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9654005640 con 0x7f96580ac740 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.110+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 --> [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f966c062b30 con 0x7f96580ac740 2024-08-19T01:34:09.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.110+0000 7f967096c640 1 -- 172.21.15.103:0/310168354 <== osd.9 v2:172.21.15.103:6826/1538313167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f966c062b30 con 0x7f96580ac740 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.110+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] conn(0x7f96580ac740 msgr2=0x7f96580aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.110+0000 7f967196e640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] conn(0x7f96580ac740 0x7f96580aebe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96580a2c10 msgr2=0x7f96580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96580a2c10 0x7f96580a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9664005fd0 tx=0x7f96640093e0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 msgr2=0x7f966c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 --2- 172.21.15.103:0/310168354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f966c1022a0 0x7f966c1987b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9654006010 tx=0x7f96540042f0 comp rx=0 tx=0).stop 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f966b7fe640 1 -- 172.21.15.103:0/310168354 reap_dead start 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 shutdown_connections 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 >> 172.21.15.103:0/310168354 conn(0x7f966c0fc500 msgr2=0x7f966c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 shutdown_connections 2024-08-19T01:34:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:09.111+0000 7f967196e640 1 -- 172.21.15.103:0/310168354 wait complete. 2024-08-19T01:34:10.093 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:09 smithi103 ceph-mon[18436]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-08-19T01:34:10.094 INFO:teuthology.orchestra.run.smithi103.stdout:279172874271 2024-08-19T01:34:10.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-19T01:34:10.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:09 smithi190 ceph-mon[22032]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-08-19T01:34:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:09 smithi146 ceph-mon[22098]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 13 B/s, 1 objects/s recovering 2024-08-19T01:34:10.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 -- 172.21.15.103:0/848796694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790102c20 msgr2=0x7fd79010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 --2- 172.21.15.103:0/848796694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790102c20 0x7fd79010d770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd788009ca0 tx=0x7fd78802a950 comp rx=0 tx=0).stop 2024-08-19T01:34:10.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 -- 172.21.15.103:0/848796694 shutdown_connections 2024-08-19T01:34:10.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 --2- 172.21.15.103:0/848796694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790102c20 0x7fd79010d770 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 --2- 172.21.15.103:0/848796694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7901022a0 0x7fd7901026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 --2- 172.21.15.103:0/848796694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd790100b50 0x7fd790100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 -- 172.21.15.103:0/848796694 >> 172.21.15.103:0/848796694 conn(0x7fd7900fc500 msgr2=0x7fd7900fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 -- 172.21.15.103:0/848796694 shutdown_connections 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 -- 172.21.15.103:0/848796694 wait complete. 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 Processor -- start 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.293+0000 7fd7986c2640 1 -- start start 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790100b50 0x7fd7900fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7901022a0 0x7fd7900fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd790102c20 0x7fd7900fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd790110c00 con 0x7fd790100b50 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd790110a80 con 0x7fd7901022a0 2024-08-19T01:34:10.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd790110d80 con 0x7fd790102c20 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790100b50 0x7fd7900fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790100b50 0x7fd7900fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46096/0 (socket says 172.21.15.103:46096) 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 -- 172.21.15.103:0/3696738218 learned_addr learned my addr 172.21.15.103:0/3696738218 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd796c5f640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd790102c20 0x7fd7900fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 -- 172.21.15.103:0/3696738218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd790102c20 msgr2=0x7fd7900fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd790102c20 0x7fd7900fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 -- 172.21.15.103:0/3696738218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7901022a0 msgr2=0x7fd7900fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd795c5d640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7901022a0 0x7fd7900fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7901022a0 0x7fd7900fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 -- 172.21.15.103:0/3696738218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7901a6710 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd796c5f640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd790102c20 0x7fd7900fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd79645e640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790100b50 0x7fd7900fc9b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00b4a0 tx=0x7fd78c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c014070 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7901a6a00 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd78c010ac0 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c017c30 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.294+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901a6ef0 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.295+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd790106f90 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.297+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd78c017070 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.297+0000 7fd7877fe640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.297+0000 7fd795c5d640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.297+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd78c0ce8f0 con 0x7fd790100b50 2024-08-19T01:34:10.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.298+0000 7fd7877fe640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] conn(0x7fd77c0ac740 0x7fd77c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.298+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 --> [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd78c01be50 con 0x7fd77c0ac740 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.298+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd78c0ced40 con 0x7fd790100b50 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.298+0000 7fd796c5f640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] conn(0x7fd77c0ac740 0x7fd77c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.298+0000 7fd795c5d640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd7900fe440 tx=0x7fd780008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.298+0000 7fd796c5f640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] conn(0x7fd77c0ac740 0x7fd77c0aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.299+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== osd.3 v2:172.21.15.103:6810/2225524779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd78c01be50 con 0x7fd77c0ac740 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.312+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 --> [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd790101ee0 con 0x7fd77c0ac740 2024-08-19T01:34:10.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7877fe640 1 -- 172.21.15.103:0/3696738218 <== osd.3 v2:172.21.15.103:6810/2225524779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd790101ee0 con 0x7fd77c0ac740 2024-08-19T01:34:10.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] conn(0x7fd77c0ac740 msgr2=0x7fd77c0aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] conn(0x7fd77c0ac740 0x7fd77c0aebe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:10.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd77c0a2c10 msgr2=0x7fd77c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd77c0a2c10 0x7fd77c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd7900fe440 tx=0x7fd780008040 comp rx=0 tx=0).stop 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790100b50 msgr2=0x7fd7900fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 --2- 172.21.15.103:0/3696738218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd790100b50 0x7fd7900fc9b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00b4a0 tx=0x7fd78c00b960 comp rx=0 tx=0).stop 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd796c5f640 1 -- 172.21.15.103:0/3696738218 reap_dead start 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 shutdown_connections 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 >> 172.21.15.103:0/3696738218 conn(0x7fd7900fc500 msgr2=0x7fd790069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 shutdown_connections 2024-08-19T01:34:10.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:10.313+0000 7fd7986c2640 1 -- 172.21.15.103:0/3696738218 wait complete. 2024-08-19T01:34:11.010 INFO:teuthology.orchestra.run.smithi103.stdout:107374182459 2024-08-19T01:34:11.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-19T01:34:12.015 INFO:teuthology.orchestra.run.smithi103.stdout:141733920822 2024-08-19T01:34:12.016 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-19T01:34:12.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:11 smithi190 ceph-mon[22032]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 11 B/s, 1 objects/s recovering 2024-08-19T01:34:12.201 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:11 smithi103 ceph-mon[18436]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 11 B/s, 1 objects/s recovering 2024-08-19T01:34:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:11 smithi146 ceph-mon[22098]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 11 B/s, 1 objects/s recovering 2024-08-19T01:34:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 -- 172.21.15.103:0/2692272625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c10bed0 msgr2=0x7f5b6c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 --2- 172.21.15.103:0/2692272625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c10bed0 0x7f5b6c10e340 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009ca0 tx=0x7f5b6402a950 comp rx=0 tx=0).stop 2024-08-19T01:34:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 -- 172.21.15.103:0/2692272625 shutdown_connections 2024-08-19T01:34:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 --2- 172.21.15.103:0/2692272625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c10bed0 0x7f5b6c10e340 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 --2- 172.21.15.103:0/2692272625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b6c103410 0x7f5b6c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 --2- 172.21.15.103:0/2692272625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b6c102af0 0x7f5b6c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 -- 172.21.15.103:0/2692272625 >> 172.21.15.103:0/2692272625 conn(0x7f5b6c0fc4d0 msgr2=0x7f5b6c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 -- 172.21.15.103:0/2692272625 shutdown_connections 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.281+0000 7f5b733e3640 1 -- 172.21.15.103:0/2692272625 wait complete. 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 Processor -- start 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 -- start start 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c102af0 0x7f5b6c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b6c103410 0x7f5b6c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b6c10bed0 0x7f5b6c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c111b10 con 0x7f5b6c102af0 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c111990 con 0x7f5b6c103410 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b733e3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c111c90 con 0x7f5b6c10bed0 2024-08-19T01:34:12.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b7117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c102af0 0x7f5b6c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b7117f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c102af0 0x7f5b6c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40952/0 (socket says 172.21.15.103:40952) 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.282+0000 7f5b7117f640 1 -- 172.21.15.103:0/2259772079 learned_addr learned my addr 172.21.15.103:0/2259772079 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b71980640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b6c10bed0 0x7f5b6c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7097e640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b6c103410 0x7f5b6c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7117f640 1 -- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b6c10bed0 msgr2=0x7f5b6c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7117f640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b6c10bed0 0x7f5b6c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7117f640 1 -- 172.21.15.103:0/2259772079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b6c103410 msgr2=0x7f5b6c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7117f640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b6c103410 0x7f5b6c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7117f640 1 -- 172.21.15.103:0/2259772079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c19d000 con 0x7f5b6c102af0 2024-08-19T01:34:12.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7097e640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b6c103410 0x7f5b6c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b7117f640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c102af0 0x7f5b6c198050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00d950 tx=0x7f5b5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c017030 con 0x7f5b6c102af0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b5c004020 con 0x7f5b6c102af0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c19d2f0 con 0x7f5b6c102af0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c0047a0 con 0x7f5b6c102af0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.283+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a4bd0 con 0x7f5b6c102af0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.284+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b6c0626b0 con 0x7f5b6c102af0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.285+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5b5c0041d0 con 0x7f5b6c102af0 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b627fc640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b580a2c10 0x7f5b580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b7097e640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b580a2c10 0x7f5b580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5b5c0cf3c0 con 0x7f5b6c102af0 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b627fc640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] conn(0x7f5b580ac740 0x7f5b580aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 --> [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b5c01de50 con 0x7f5b580ac740 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5b5c0cf810 con 0x7f5b6c102af0 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b7097e640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b580a2c10 0x7f5b580a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5b68005fd0 tx=0x7f5b680094d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.286+0000 7f5b71980640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] conn(0x7f5b580ac740 0x7f5b580aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.287+0000 7f5b71980640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] conn(0x7f5b580ac740 0x7f5b580aebe0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:12.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.288+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== osd.11 v2:172.21.15.190:6824/2863234594 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b5c01de50 con 0x7f5b580ac740 2024-08-19T01:34:12.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.301+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 --> [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b6c062b30 con 0x7f5b580ac740 2024-08-19T01:34:12.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b627fc640 1 -- 172.21.15.103:0/2259772079 <== osd.11 v2:172.21.15.190:6824/2863234594 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5b6c062b30 con 0x7f5b580ac740 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] conn(0x7f5b580ac740 msgr2=0x7f5b580aebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] conn(0x7f5b580ac740 0x7f5b580aebe0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b580a2c10 msgr2=0x7f5b580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b580a2c10 0x7f5b580a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5b68005fd0 tx=0x7f5b680094d0 comp rx=0 tx=0).stop 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c102af0 msgr2=0x7f5b6c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 --2- 172.21.15.103:0/2259772079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b6c102af0 0x7f5b6c198050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00d950 tx=0x7f5b5c00de10 comp rx=0 tx=0).stop 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b71980640 1 -- 172.21.15.103:0/2259772079 reap_dead start 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 shutdown_connections 2024-08-19T01:34:12.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 >> 172.21.15.103:0/2259772079 conn(0x7f5b6c0fc4d0 msgr2=0x7f5b6c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:12.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 shutdown_connections 2024-08-19T01:34:12.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:12.302+0000 7f5b733e3640 1 -- 172.21.15.103:0/2259772079 wait complete. 2024-08-19T01:34:13.025 INFO:teuthology.orchestra.run.smithi103.stdout:335007449112 2024-08-19T01:34:13.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-19T01:34:13.161 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.163 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 -- 172.21.15.103:0/3789651396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 msgr2=0x7fd95c107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3789651396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c107c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009cd0 tx=0x7fd94c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 -- 172.21.15.103:0/3789651396 shutdown_connections 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3789651396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c102240 0x7fd95c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3789651396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95c1018a0 0x7fd95c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3789651396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c107c80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 -- 172.21.15.103:0/3789651396 >> 172.21.15.103:0/3789651396 conn(0x7fd95c0fd5e0 msgr2=0x7fd95c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 -- 172.21.15.103:0/3789651396 shutdown_connections 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.782+0000 7fd960fa6640 1 -- 172.21.15.103:0/3789651396 wait complete. 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 Processor -- start 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 -- start start 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c1018a0 0x7fd95c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95c102240 0x7fd95c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c111b50 con 0x7fd95c1078a0 2024-08-19T01:34:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c1119d0 con 0x7fd95c1018a0 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd960fa6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd95c111cd0 con 0x7fd95c102240 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40968/0 (socket says 172.21.15.103:40968) 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 -- 172.21.15.103:0/3896869985 learned_addr learned my addr 172.21.15.103:0/3896869985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95a59c640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c1018a0 0x7fd95c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd959d9b640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95c102240 0x7fd95c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 -- 172.21.15.103:0/3896869985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95c102240 msgr2=0x7fd95c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95c102240 0x7fd95c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 -- 172.21.15.103:0/3896869985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c1018a0 msgr2=0x7fd95c19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c1018a0 0x7fd95c19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 -- 172.21.15.103:0/3896869985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c19f1d0 con 0x7fd95c1078a0 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95a59c640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c1018a0 0x7fd95c19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd959d9b640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95c102240 0x7fd95c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd95ad9d640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c19eaf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c970 tx=0x7fd95400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.783+0000 7fd9537fe640 1 -- 172.21.15.103:0/3896869985 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954007bb0 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.784+0000 7fd9537fe640 1 -- 172.21.15.103:0/3896869985 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd954007d50 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.784+0000 7fd9537fe640 1 -- 172.21.15.103:0/3896869985 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954018700 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.784+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95c19f4c0 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.784+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95c1a6da0 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.785+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95c101d00 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.786+0000 7fd9537fe640 1 -- 172.21.15.103:0/3896869985 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd95400fa20 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.786+0000 7fd9537fe640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd9480a2c10 0x7fd9480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.786+0000 7fd95a59c640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd9480a2c10 0x7fd9480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.786+0000 7fd9537fe640 1 -- 172.21.15.103:0/3896869985 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd9540cf280 con 0x7fd95c1078a0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.787+0000 7fd95a59c640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd9480a2c10 0x7fd9480a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006010 tx=0x7fd94c009800 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:13.790+0000 7fd9537fe640 1 -- 172.21.15.103:0/3896869985 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd954014030 con 0x7fd95c1078a0 2024-08-19T01:34:13.820 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.822 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.822 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.824 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.827 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.845 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:13.845 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:14.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.015+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd95c1a6fd0 con 0x7fd95c1078a0 2024-08-19T01:34:14.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.016+0000 7fd9537fe640 1 -- 172.21.15.103:0/3896869985 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd95409c0b0 con 0x7fd95c1078a0 2024-08-19T01:34:14.017 INFO:teuthology.orchestra.run.smithi103.stdout:55834574917 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd9480a2c10 msgr2=0x7fd9480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd9480a2c10 0x7fd9480a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006010 tx=0x7fd94c009800 comp rx=0 tx=0).stop 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 msgr2=0x7fd95c19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c19eaf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c970 tx=0x7fd95400ce30 comp rx=0 tx=0).stop 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 shutdown_connections 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd9480a2c10 0x7fd9480a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd95c1078a0 0x7fd95c19eaf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95c102240 0x7fd95c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 --2- 172.21.15.103:0/3896869985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd95c1018a0 0x7fd95c19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:14.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 >> 172.21.15.103:0/3896869985 conn(0x7fd95c0fd5e0 msgr2=0x7fd95c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:14.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 shutdown_connections 2024-08-19T01:34:14.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:14.018+0000 7fd960fa6640 1 -- 172.21.15.103:0/3896869985 wait complete. 2024-08-19T01:34:14.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:13 smithi190 ceph-mon[22032]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-08-19T01:34:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:13 smithi146 ceph-mon[22098]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-08-19T01:34:14.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:13 smithi103 ceph-mon[18436]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-08-19T01:34:15.029 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:14 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3896869985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-19T01:34:15.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:14 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3896869985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-19T01:34:15.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:14 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3896869985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 -- 172.21.15.103:0/2852467006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840101150 msgr2=0x7f184010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 --2- 172.21.15.103:0/2852467006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840101150 0x7f184010d680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1838009cd0 tx=0x7f183802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 -- 172.21.15.103:0/2852467006 shutdown_connections 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 --2- 172.21.15.103:0/2852467006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840101150 0x7f184010d680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 --2- 172.21.15.103:0/2852467006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18401007b0 0x7f1840100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 --2- 172.21.15.103:0/2852467006 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18401067b0 0x7f1840106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 -- 172.21.15.103:0/2852467006 >> 172.21.15.103:0/2852467006 conn(0x7f18400fc4d0 msgr2=0x7f18400fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 -- 172.21.15.103:0/2852467006 shutdown_connections 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 -- 172.21.15.103:0/2852467006 wait complete. 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 Processor -- start 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.202+0000 7f1848498640 1 -- start start 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18401007b0 0x7f1840198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1840101150 0x7f1840199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18401067b0 0x7f18401a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1840110a60 con 0x7f18401067b0 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f18401108e0 con 0x7f1840101150 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1840110be0 con 0x7f18401007b0 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18401067b0 0x7f18401a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18401067b0 0x7f18401a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40998/0 (socket says 172.21.15.103:40998) 2024-08-19T01:34:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 -- 172.21.15.103:0/1686315713 learned_addr learned my addr 172.21.15.103:0/1686315713 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846234640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18401007b0 0x7f1840198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 -- 172.21.15.103:0/1686315713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18401007b0 msgr2=0x7f1840198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18401007b0 0x7f1840198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 -- 172.21.15.103:0/1686315713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1840101150 msgr2=0x7f1840199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1845a33640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1840101150 0x7f1840199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1840101150 0x7f1840199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 -- 172.21.15.103:0/1686315713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18401a44b0 con 0x7f18401067b0 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846234640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18401007b0 0x7f1840198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1846a35640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18401067b0 0x7f18401a3f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1838005cc0 tx=0x7f1838003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f18337fe640 1 -- 172.21.15.103:0/1686315713 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838004450 con 0x7f18401067b0 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18401a4680 con 0x7f18401067b0 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f18337fe640 1 -- 172.21.15.103:0/1686315713 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1838031c90 con 0x7f18401067b0 2024-08-19T01:34:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f18337fe640 1 -- 172.21.15.103:0/1686315713 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183803b400 con 0x7f18401067b0 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.203+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18401a4b60 con 0x7f18401067b0 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.204+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1840100c10 con 0x7f18401067b0 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.205+0000 7f18337fe640 1 -- 172.21.15.103:0/1686315713 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1838004680 con 0x7f18401067b0 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.206+0000 7f18337fe640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f182c0a2c90 0x7f182c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.206+0000 7f1846234640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f182c0a2c90 0x7f182c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.206+0000 7f18337fe640 1 -- 172.21.15.103:0/1686315713 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f183802e030 con 0x7f18401067b0 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.207+0000 7f1846234640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f182c0a2c90 0x7f182c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f183c005e00 tx=0x7f183c005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:15.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.210+0000 7f18337fe640 1 -- 172.21.15.103:0/1686315713 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f18380bd200 con 0x7f18401067b0 2024-08-19T01:34:15.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.387+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f184002d5e0 con 0x7f18401067b0 2024-08-19T01:34:15.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7f18337fe640 1 -- 172.21.15.103:0/1686315713 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f18380098b0 con 0x7f18401067b0 2024-08-19T01:34:15.389 INFO:teuthology.orchestra.run.smithi103.stdout:244813135909 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f182c0a2c90 msgr2=0x7f182c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f182c0a2c90 0x7f182c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f183c005e00 tx=0x7f183c005c70 comp rx=0 tx=0).stop 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18401067b0 msgr2=0x7f18401a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18401067b0 0x7f18401a3f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1838005cc0 tx=0x7f1838003c20 comp rx=0 tx=0).stop 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 shutdown_connections 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f182c0a2c90 0x7f182c0a5150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18401067b0 0x7f18401a3f70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1840101150 0x7f1840199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 --2- 172.21.15.103:0/1686315713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f18401007b0 0x7f1840198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 >> 172.21.15.103:0/1686315713 conn(0x7f18400fc4d0 msgr2=0x7f18400fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:15.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 shutdown_connections 2024-08-19T01:34:15.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7f1848498640 1 -- 172.21.15.103:0/1686315713 wait complete. 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 -- 172.21.15.103:0/1020570727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0107fd0 msgr2=0x7efcc010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 --2- 172.21.15.103:0/1020570727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0107fd0 0x7efcc010e970 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009cd0 tx=0x7efcb802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 -- 172.21.15.103:0/1020570727 shutdown_connections 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 --2- 172.21.15.103:0/1020570727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0107fd0 0x7efcc010e970 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 --2- 172.21.15.103:0/1020570727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc0103410 0x7efcc01078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 --2- 172.21.15.103:0/1020570727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc0102a60 0x7efcc0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 -- 172.21.15.103:0/1020570727 >> 172.21.15.103:0/1020570727 conn(0x7efcc00fc500 msgr2=0x7efcc00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 -- 172.21.15.103:0/1020570727 shutdown_connections 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.388+0000 7efcc4c56640 1 -- 172.21.15.103:0/1020570727 wait complete. 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 Processor -- start 2024-08-19T01:34:15.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 -- start start 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc0102a60 0x7efcc01006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0103410 0x7efcc0100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc0107fd0 0x7efcc0101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0111c80 con 0x7efcc0103410 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0111b00 con 0x7efcc0107fd0 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcc4c56640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0111e00 con 0x7efcc0102a60 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0103410 0x7efcc0100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0103410 0x7efcc0100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41012/0 (socket says 172.21.15.103:41012) 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 -- 172.21.15.103:0/2570981170 learned_addr learned my addr 172.21.15.103:0/2570981170 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbe59c640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc0102a60 0x7efcc01006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbed9d640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc0107fd0 0x7efcc0101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 -- 172.21.15.103:0/2570981170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc0102a60 msgr2=0x7efcc01006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc0102a60 0x7efcc01006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 -- 172.21.15.103:0/2570981170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc0107fd0 msgr2=0x7efcc0101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc0107fd0 0x7efcc0101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 -- 172.21.15.103:0/2570981170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc01a65e0 con 0x7efcc0103410 2024-08-19T01:34:15.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbed9d640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc0107fd0 0x7efcc0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:15.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.389+0000 7efcbdd9b640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0103410 0x7efcc0100c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efca800d950 tx=0x7efca800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:15.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.390+0000 7efcb77fe640 1 -- 172.21.15.103:0/2570981170 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8017070 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.390+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc01a68d0 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.390+0000 7efcb77fe640 1 -- 172.21.15.103:0/2570981170 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efca8009c30 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.390+0000 7efcb77fe640 1 -- 172.21.15.103:0/2570981170 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80055e0 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.390+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01a6e60 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.391+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc0102e40 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.392+0000 7efcb77fe640 1 -- 172.21.15.103:0/2570981170 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efca8004020 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.393+0000 7efcb77fe640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcac0a2c10 0x7efcac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.393+0000 7efcbe59c640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcac0a2c10 0x7efcac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.393+0000 7efcb77fe640 1 -- 172.21.15.103:0/2570981170 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efca80ceda0 con 0x7efcc0103410 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.393+0000 7efcbe59c640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcac0a2c10 0x7efcac0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb000abe0 tx=0x7efcb000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:15.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.397+0000 7efcb77fe640 1 -- 172.21.15.103:0/2570981170 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efca8020020 con 0x7efcc0103410 2024-08-19T01:34:15.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.568+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efcc0109fe0 con 0x7efcc0103410 2024-08-19T01:34:15.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.569+0000 7efcb77fe640 1 -- 172.21.15.103:0/2570981170 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efca809bbd0 con 0x7efcc0103410 2024-08-19T01:34:15.570 INFO:teuthology.orchestra.run.smithi103.stdout:107374182460 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcac0a2c10 msgr2=0x7efcac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcac0a2c10 0x7efcac0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb000abe0 tx=0x7efcb000a2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0103410 msgr2=0x7efcc0100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0103410 0x7efcc0100c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efca800d950 tx=0x7efca800dd50 comp rx=0 tx=0).stop 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 shutdown_connections 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcac0a2c10 0x7efcac0a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcc0107fd0 0x7efcc0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcc0103410 0x7efcc0100c00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 --2- 172.21.15.103:0/2570981170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc0102a60 0x7efcc01006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:15.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 >> 172.21.15.103:0/2570981170 conn(0x7efcc00fc500 msgr2=0x7efcc0105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:15.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 shutdown_connections 2024-08-19T01:34:15.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:15.571+0000 7efcc4c56640 1 -- 172.21.15.103:0/2570981170 wait complete. 2024-08-19T01:34:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:15 smithi190 ceph-mon[22032]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-08-19T01:34:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:15 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1686315713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-19T01:34:16.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:15 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2570981170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-19T01:34:16.157 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:15 smithi103 ceph-mon[18436]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-08-19T01:34:16.157 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:15 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1686315713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-19T01:34:16.157 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:15 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2570981170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-19T01:34:16.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:15 smithi146 ceph-mon[22098]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-08-19T01:34:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:15 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1686315713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-19T01:34:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:15 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2570981170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-19T01:34:16.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- 172.21.15.103:0/2636257290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 msgr2=0x7fc02810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 --2- 172.21.15.103:0/2636257290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc02810d680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc020009cd0 tx=0x7fc02002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- 172.21.15.103:0/2636257290 shutdown_connections 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 --2- 172.21.15.103:0/2636257290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc02810d680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 --2- 172.21.15.103:0/2636257290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0281007b0 0x7fc028100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 --2- 172.21.15.103:0/2636257290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0281067b0 0x7fc028106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- 172.21.15.103:0/2636257290 >> 172.21.15.103:0/2636257290 conn(0x7fc0280fc4d0 msgr2=0x7fc0280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- 172.21.15.103:0/2636257290 shutdown_connections 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- 172.21.15.103:0/2636257290 wait complete. 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 Processor -- start 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- start start 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0281007b0 0x7fc0281980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc028198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0281067b0 0x7fc02819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc028111b80 con 0x7fc028101150 2024-08-19T01:34:16.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc028111a00 con 0x7fc0281007b0 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.933+0000 7fc02efdb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc028111d00 con 0x7fc0281067b0 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc028198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc028198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41026/0 (socket says 172.21.15.103:41026) 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 -- 172.21.15.103:0/1573330666 learned_addr learned my addr 172.21.15.103:0/1573330666 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc02cd77640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0281007b0 0x7fc0281980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc02d578640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0281067b0 0x7fc02819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 -- 172.21.15.103:0/1573330666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0281067b0 msgr2=0x7fc02819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0281067b0 0x7fc02819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 -- 172.21.15.103:0/1573330666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0281007b0 msgr2=0x7fc0281980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0281007b0 0x7fc0281980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 -- 172.21.15.103:0/1573330666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02819d090 con 0x7fc028101150 2024-08-19T01:34:16.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc02cd77640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0281007b0 0x7fc0281980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc027fff640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc028198620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b4a0 tx=0x7fc01400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc025ffb640 1 -- 172.21.15.103:0/1573330666 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014014070 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc025ffb640 1 -- 172.21.15.103:0/1573330666 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc014010ac0 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc025ffb640 1 -- 172.21.15.103:0/1573330666 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014017c30 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.934+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02819d380 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.935+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0281a4c60 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.936+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc028100c10 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.937+0000 7fc025ffb640 1 -- 172.21.15.103:0/1573330666 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc014017070 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.937+0000 7fc025ffb640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0180a2c10 0x7fc0180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.937+0000 7fc025ffb640 1 -- 172.21.15.103:0/1573330666 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc0140ce3d0 con 0x7fc028101150 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.937+0000 7fc02cd77640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0180a2c10 0x7fc0180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.938+0000 7fc02cd77640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0180a2c10 0x7fc0180a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00bfd0 tx=0x7fc01c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:16.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:16.942+0000 7fc025ffb640 1 -- 172.21.15.103:0/1573330666 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc01409acc0 con 0x7fc028101150 2024-08-19T01:34:17.111 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:16 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:34:17.113 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574915 got 55834574917 for osd.1 2024-08-19T01:34:17.113 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.115+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc02802d5e0 con 0x7fc028101150 2024-08-19T01:34:17.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.116+0000 7fc025ffb640 1 -- 172.21.15.103:0/1573330666 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc01409efc0 con 0x7fc028101150 2024-08-19T01:34:17.116 INFO:teuthology.orchestra.run.smithi103.stdout:193273528367 2024-08-19T01:34:17.117 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:16 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:34:17.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.117+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0180a2c10 msgr2=0x7fc0180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:17.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.117+0000 7fc02efdb640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0180a2c10 0x7fc0180a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00bfd0 tx=0x7fc01c00b390 comp rx=0 tx=0).stop 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.117+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 msgr2=0x7fc028198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc028198620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc01400b4a0 tx=0x7fc01400b960 comp rx=0 tx=0).stop 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 shutdown_connections 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0180a2c10 0x7fc0180a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0281067b0 0x7fc02819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc028101150 0x7fc028198620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 --2- 172.21.15.103:0/1573330666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc0281007b0 0x7fc0281980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 >> 172.21.15.103:0/1573330666 conn(0x7fc0280fc4d0 msgr2=0x7fc0280fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 shutdown_connections 2024-08-19T01:34:17.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.118+0000 7fc02efdb640 1 -- 172.21.15.103:0/1573330666 wait complete. 2024-08-19T01:34:17.122 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:16 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:34:17.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 -- 172.21.15.103:0/2763032118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201018b0 msgr2=0x7f8e20101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:17.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 --2- 172.21.15.103:0/2763032118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201018b0 0x7f8e20101d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009930 tx=0x7f8e1002a420 comp rx=0 tx=0).stop 2024-08-19T01:34:17.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 -- 172.21.15.103:0/2763032118 shutdown_connections 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 --2- 172.21.15.103:0/2763032118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e20102250 0x7f8e2010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 --2- 172.21.15.103:0/2763032118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201018b0 0x7f8e20101d10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 --2- 172.21.15.103:0/2763032118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e201078b0 0x7f8e20107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 -- 172.21.15.103:0/2763032118 >> 172.21.15.103:0/2763032118 conn(0x7f8e200fd5f0 msgr2=0x7f8e200ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 -- 172.21.15.103:0/2763032118 shutdown_connections 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.267+0000 7f8e2864f640 1 -- 172.21.15.103:0/2763032118 wait complete. 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 Processor -- start 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 -- start start 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e201018b0 0x7f8e20074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e20102250 0x7f8e20074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201078b0 0x7f8e20074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20111b80 con 0x7f8e201078b0 2024-08-19T01:34:17.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20111a00 con 0x7f8e201018b0 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e2864f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20111d00 con 0x7f8e20102250 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e26bec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201078b0 0x7f8e20074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e26bec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201078b0 0x7f8e20074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41042/0 (socket says 172.21.15.103:41042) 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e26bec640 1 -- 172.21.15.103:0/3151169860 learned_addr learned my addr 172.21.15.103:0/3151169860 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.268+0000 7f8e25bea640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e20102250 0x7f8e20074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e263eb640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e201018b0 0x7f8e20074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e26bec640 1 -- 172.21.15.103:0/3151169860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e20102250 msgr2=0x7f8e20074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e26bec640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e20102250 0x7f8e20074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e26bec640 1 -- 172.21.15.103:0/3151169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e201018b0 msgr2=0x7f8e20074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e26bec640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e201018b0 0x7f8e20074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e26bec640 1 -- 172.21.15.103:0/3151169860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e201a6600 con 0x7f8e201078b0 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e26bec640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201078b0 0x7f8e20074ca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b970 tx=0x7f8e1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:17.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e177fe640 1 -- 172.21.15.103:0/3151169860 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18019070 con 0x7f8e201078b0 2024-08-19T01:34:17.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e201a68f0 con 0x7f8e201078b0 2024-08-19T01:34:17.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e177fe640 1 -- 172.21.15.103:0/3151169860 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e18014440 con 0x7f8e201078b0 2024-08-19T01:34:17.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e177fe640 1 -- 172.21.15.103:0/3151169860 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e180075d0 con 0x7f8e201078b0 2024-08-19T01:34:17.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.269+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e201a6e50 con 0x7f8e201078b0 2024-08-19T01:34:17.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.270+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e20101d10 con 0x7f8e201078b0 2024-08-19T01:34:17.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.272+0000 7f8e177fe640 1 -- 172.21.15.103:0/3151169860 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8e1801c070 con 0x7f8e201078b0 2024-08-19T01:34:17.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.272+0000 7f8e177fe640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e0c0a2c10 0x7f8e0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:17.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.272+0000 7f8e263eb640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e0c0a2c10 0x7f8e0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:17.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.272+0000 7f8e177fe640 1 -- 172.21.15.103:0/3151169860 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8e180ce620 con 0x7f8e201078b0 2024-08-19T01:34:17.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.273+0000 7f8e263eb640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e0c0a2c10 0x7f8e0c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0058c0 tx=0x7f8e1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:17.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.276+0000 7f8e177fe640 1 -- 172.21.15.103:0/3151169860 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8e1809af10 con 0x7f8e201078b0 2024-08-19T01:34:17.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.450+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8e20109e40 con 0x7f8e201078b0 2024-08-19T01:34:17.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.450+0000 7f8e177fe640 1 -- 172.21.15.103:0/3151169860 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e1809f210 con 0x7f8e201078b0 2024-08-19T01:34:17.451 INFO:teuthology.orchestra.run.smithi103.stdout:214748364842 2024-08-19T01:34:17.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e0c0a2c10 msgr2=0x7f8e0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e0c0a2c10 0x7f8e0c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0058c0 tx=0x7f8e1c00a2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201078b0 msgr2=0x7f8e20074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201078b0 0x7f8e20074ca0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b970 tx=0x7f8e1800be30 comp rx=0 tx=0).stop 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 shutdown_connections 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8e0c0a2c10 0x7f8e0c0a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e201078b0 0x7f8e20074ca0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e20102250 0x7f8e20074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 --2- 172.21.15.103:0/3151169860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e201018b0 0x7f8e20074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 >> 172.21.15.103:0/3151169860 conn(0x7f8e200fd5f0 msgr2=0x7f8e200fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 shutdown_connections 2024-08-19T01:34:17.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:17.453+0000 7f8e2864f640 1 -- 172.21.15.103:0/3151169860 wait complete. 2024-08-19T01:34:17.849 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135906 got 244813135909 for osd.8 2024-08-19T01:34:17.850 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:18.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:17 smithi190 ceph-mon[22032]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-08-19T01:34:18.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:17 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1573330666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-19T01:34:18.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:17 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3151169860' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-19T01:34:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:17 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:17 smithi146 ceph-mon[22098]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-08-19T01:34:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:17 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1573330666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-19T01:34:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:17 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3151169860' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-19T01:34:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:17 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.273 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:17 smithi103 ceph-mon[18436]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-08-19T01:34:18.273 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:17 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1573330666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-19T01:34:18.273 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:17 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3151169860' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-19T01:34:18.273 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.273 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.274 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:18.274 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:17 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:19.383 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182459 got 107374182460 for osd.3 2024-08-19T01:34:19.383 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:19.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 -- 172.21.15.103:0/1918646366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc100b50 msgr2=0x7f25bc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:19.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 --2- 172.21.15.103:0/1918646366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc100b50 0x7f25bc100f30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009ca0 tx=0x7f25ac02a950 comp rx=0 tx=0).stop 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 -- 172.21.15.103:0/1918646366 shutdown_connections 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 --2- 172.21.15.103:0/1918646366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc102c20 0x7f25bc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 --2- 172.21.15.103:0/1918646366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc1022a0 0x7f25bc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 --2- 172.21.15.103:0/1918646366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc100b50 0x7f25bc100f30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 -- 172.21.15.103:0/1918646366 >> 172.21.15.103:0/1918646366 conn(0x7f25bc0fc500 msgr2=0x7f25bc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 -- 172.21.15.103:0/1918646366 shutdown_connections 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.770+0000 7f25c14a3640 1 -- 172.21.15.103:0/1918646366 wait complete. 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 Processor -- start 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 -- start start 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100b50 0x7f25bc199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc1022a0 0x7f25bc199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc102c20 0x7f25bc1a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:19.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc110c00 con 0x7f25bc1022a0 2024-08-19T01:34:19.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc110a80 con 0x7f25bc100b50 2024-08-19T01:34:19.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.771+0000 7f25c14a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc110d80 con 0x7f25bc102c20 2024-08-19T01:34:19.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc1022a0 0x7f25bc199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:19.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc1022a0 0x7f25bc199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41066/0 (socket says 172.21.15.103:41066) 2024-08-19T01:34:19.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 -- 172.21.15.103:0/56108842 learned_addr learned my addr 172.21.15.103:0/56108842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:19.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25bb7fe640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc102c20 0x7f25bc1a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:19.718 INFO:teuthology.orchestra.run.smithi103.stdout:279172874273 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25baffd640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100b50 0x7f25bc199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 -- 172.21.15.103:0/56108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc102c20 msgr2=0x7f25bc1a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc102c20 0x7f25bc1a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 -- 172.21.15.103:0/56108842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100b50 msgr2=0x7f25bc199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100b50 0x7f25bc199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 -- 172.21.15.103:0/56108842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc1a46f0 con 0x7f25bc1022a0 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25baffd640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100b50 0x7f25bc199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25ba7fc640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc1022a0 0x7f25bc199670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b4a0 tx=0x7f25b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25a3fff640 1 -- 172.21.15.103:0/56108842 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0014070 con 0x7f25bc1022a0 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25a3fff640 1 -- 172.21.15.103:0/56108842 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25b0010ac0 con 0x7f25bc1022a0 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc1a48c0 con 0x7f25bc1022a0 2024-08-19T01:34:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25a3fff640 1 -- 172.21.15.103:0/56108842 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0017c30 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.772+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc1a4e00 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.773+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25bc100fb0 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.774+0000 7f25a3fff640 1 -- 172.21.15.103:0/56108842 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f25b0017070 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.775+0000 7f25a3fff640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25a40a2c10 0x7f25a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.775+0000 7f25baffd640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25a40a2c10 0x7f25a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.775+0000 7f25a3fff640 1 -- 172.21.15.103:0/56108842 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f25b00ce760 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.776+0000 7f25baffd640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25a40a2c10 0x7f25a40a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005cc0 tx=0x7f25ac005c30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.779+0000 7f25a3fff640 1 -- 172.21.15.103:0/56108842 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f25b009b6a0 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.953+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f25bc108dc0 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.953+0000 7f25a3fff640 1 -- 172.21.15.103:0/56108842 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f25b009afa0 con 0x7f25bc1022a0 2024-08-19T01:34:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25a40a2c10 msgr2=0x7f25a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25a40a2c10 0x7f25a40a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005cc0 tx=0x7f25ac005c30 comp rx=0 tx=0).stop 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc1022a0 msgr2=0x7f25bc199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc1022a0 0x7f25bc199670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b4a0 tx=0x7f25b000b960 comp rx=0 tx=0).stop 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 shutdown_connections 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25a40a2c10 0x7f25a40a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25bc102c20 0x7f25bc1a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25bc1022a0 0x7f25bc199670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 --2- 172.21.15.103:0/56108842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25bc100b50 0x7f25bc199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 >> 172.21.15.103:0/56108842 conn(0x7f25bc0fc500 msgr2=0x7f25bc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 shutdown_connections 2024-08-19T01:34:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:18.955+0000 7f25c14a3640 1 -- 172.21.15.103:0/56108842 wait complete. 2024-08-19T01:34:20.019 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:19 smithi103 ceph-mon[18436]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:20.020 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:19 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/56108842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-19T01:34:20.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:19 smithi190 ceph-mon[22032]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:20.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:19 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/56108842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-19T01:34:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:19 smithi146 ceph-mon[22098]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:19 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/56108842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-19T01:34:20.377 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874271 got 279172874273 for osd.9 2024-08-19T01:34:20.377 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:20.831 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364839 got 214748364842 for osd.7 2024-08-19T01:34:20.831 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:21.248 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:21.249 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:21.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 -- 172.21.15.103:0/1483708627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8102ae0 msgr2=0x7f8ee8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1483708627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8102ae0 0x7f8ee8102ec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009930 tx=0x7f8ed802a420 comp rx=0 tx=0).stop 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 -- 172.21.15.103:0/1483708627 shutdown_connections 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1483708627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ee8108010 0x7f8ee810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1483708627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee8103490 0x7f8ee81078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1483708627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8102ae0 0x7f8ee8102ec0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 -- 172.21.15.103:0/1483708627 >> 172.21.15.103:0/1483708627 conn(0x7f8ee80fc500 msgr2=0x7f8ee80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 -- 172.21.15.103:0/1483708627 shutdown_connections 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.552+0000 7f8eed67f640 1 -- 172.21.15.103:0/1483708627 wait complete. 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 Processor -- start 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 -- start start 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ee8102ae0 0x7f8ee8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee8103490 0x7f8ee8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8108010 0x7f8ee819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8111c90 con 0x7f8ee8108010 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8111b10 con 0x7f8ee8102ae0 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8eed67f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8111e10 con 0x7f8ee8103490 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8108010 0x7f8ee819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8108010 0x7f8ee819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41096/0 (socket says 172.21.15.103:41096) 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8ee77fe640 1 -- 172.21.15.103:0/1874004414 learned_addr learned my addr 172.21.15.103:0/1874004414 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.553+0000 7f8ee6ffd640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ee8102ae0 0x7f8ee8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ee67fc640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee8103490 0x7f8ee8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ee77fe640 1 -- 172.21.15.103:0/1874004414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee8103490 msgr2=0x7f8ee8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ee77fe640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee8103490 0x7f8ee8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ee77fe640 1 -- 172.21.15.103:0/1874004414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ee8102ae0 msgr2=0x7f8ee8198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ee77fe640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ee8102ae0 0x7f8ee8198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ee77fe640 1 -- 172.21.15.103:0/1874004414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee819d200 con 0x7f8ee8108010 2024-08-19T01:34:21.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ee77fe640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8108010 0x7f8ee819cb20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000cad0 tx=0x7f8ee000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ecffff640 1 -- 172.21.15.103:0/1874004414 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000ae20 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee819d4f0 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ecffff640 1 -- 172.21.15.103:0/1874004414 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ee0005020 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8ecffff640 1 -- 172.21.15.103:0/1874004414 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000f6b0 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.554+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a4dd0 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.555+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee806a500 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.556+0000 7f8ecffff640 1 -- 172.21.15.103:0/1874004414 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8ee00051d0 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.557+0000 7f8ecffff640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ed00a2c10 0x7f8ed00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.557+0000 7f8ee6ffd640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ed00a2c10 0x7f8ed00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.557+0000 7f8ecffff640 1 -- 172.21.15.103:0/1874004414 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8ee00cf220 con 0x7f8ee8108010 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.557+0000 7f8ee6ffd640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ed00a2c10 0x7f8ed00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009900 tx=0x7f8ed80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.561+0000 7f8ecffff640 1 -- 172.21.15.103:0/1874004414 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8ee0014030 con 0x7f8ee8108010 2024-08-19T01:34:21.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.426+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3288767407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 msgr2=0x7f9f5410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.426+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3288767407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f5410e7b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009cd0 tx=0x7f9f4c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.426+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3288767407 shutdown_connections 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.426+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3288767407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f5410e7b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.426+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3288767407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f541018e0 0x7f9f54101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.426+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3288767407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f541078e0 0x7f9f54107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.426+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3288767407 >> 172.21.15.103:0/3288767407 conn(0x7f9f540fd600 msgr2=0x7f9f540ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3288767407 shutdown_connections 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3288767407 wait complete. 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 Processor -- start 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 -- start start 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f541018e0 0x7f9f54198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f54198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f541078e0 0x7f9f5419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54111c30 con 0x7f9f54102280 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54111ab0 con 0x7f9f541078e0 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f5c72e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54111db0 con 0x7f9f541018e0 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.427+0000 7f9f59cc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f54198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f54198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41084/0 (socket says 172.21.15.103:41084) 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 -- 172.21.15.103:0/3416433424 learned_addr learned my addr 172.21.15.103:0/3416433424 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f5accb640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f541078e0 0x7f9f5419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f5a4ca640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f541018e0 0x7f9f54198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.684 INFO:teuthology.orchestra.run.smithi103.stdout:163208757300 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 -- 172.21.15.103:0/3416433424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f541018e0 msgr2=0x7f9f54198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f541018e0 0x7f9f54198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 -- 172.21.15.103:0/3416433424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f541078e0 msgr2=0x7f9f5419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f541078e0 0x7f9f5419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 -- 172.21.15.103:0/3416433424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5419d1e0 con 0x7f9f54102280 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f59cc9640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f54198770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400b4a0 tx=0x7f9f4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f4b7fe640 1 -- 172.21.15.103:0/3416433424 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44014070 con 0x7f9f54102280 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5419d4d0 con 0x7f9f54102280 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f4b7fe640 1 -- 172.21.15.103:0/3416433424 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f44010ac0 con 0x7f9f54102280 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f4b7fe640 1 -- 172.21.15.103:0/3416433424 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44017c30 con 0x7f9f54102280 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.428+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f541a4db0 con 0x7f9f54102280 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.429+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f54101dc0 con 0x7f9f54102280 2024-08-19T01:34:21.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.430+0000 7f9f4b7fe640 1 -- 172.21.15.103:0/3416433424 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9f44017070 con 0x7f9f54102280 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.431+0000 7f9f4b7fe640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9f400a2c10 0x7f9f400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.431+0000 7f9f4b7fe640 1 -- 172.21.15.103:0/3416433424 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9f440ce950 con 0x7f9f54102280 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.431+0000 7f9f5a4ca640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9f400a2c10 0x7f9f400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.432+0000 7f9f5a4ca640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9f400a2c10 0x7f9f400a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009890 tx=0x7f9f50008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.435+0000 7f9f4b7fe640 1 -- 172.21.15.103:0/3416433424 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9f4409b830 con 0x7f9f54102280 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.610+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9f54199590 con 0x7f9f54102280 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.610+0000 7f9f4b7fe640 1 -- 172.21.15.103:0/3416433424 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9f54199590 con 0x7f9f54102280 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.612+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9f400a2c10 msgr2=0x7f9f400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.612+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9f400a2c10 0x7f9f400a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009890 tx=0x7f9f50008040 comp rx=0 tx=0).stop 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.612+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 msgr2=0x7f9f54198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.612+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f54198770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400b4a0 tx=0x7f9f4400b960 comp rx=0 tx=0).stop 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 shutdown_connections 2024-08-19T01:34:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9f400a2c10 0x7f9f400a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f541078e0 0x7f9f5419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54102280 0x7f9f54198770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 --2- 172.21.15.103:0/3416433424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f541018e0 0x7f9f54198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 >> 172.21.15.103:0/3416433424 conn(0x7f9f540fd600 msgr2=0x7f9f540fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 shutdown_connections 2024-08-19T01:34:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.613+0000 7f9f5c72e640 1 -- 172.21.15.103:0/3416433424 wait complete. 2024-08-19T01:34:21.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.740+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8ee8109ff0 con 0x7f8ee8108010 2024-08-19T01:34:21.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.740+0000 7f8ecffff640 1 -- 172.21.15.103:0/1874004414 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8ee009bff0 con 0x7f8ee8108010 2024-08-19T01:34:21.741 INFO:teuthology.orchestra.run.smithi103.stdout:141733920825 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ed00a2c10 msgr2=0x7f8ed00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ed00a2c10 0x7f8ed00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009900 tx=0x7f8ed80058d0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8108010 msgr2=0x7f8ee819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8108010 0x7f8ee819cb20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000cad0 tx=0x7f8ee000cf90 comp rx=0 tx=0).stop 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 shutdown_connections 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ed00a2c10 0x7f8ed00a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ee8108010 0x7f8ee819cb20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee8103490 0x7f8ee8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 --2- 172.21.15.103:0/1874004414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ee8102ae0 0x7f8ee8198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:21.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 >> 172.21.15.103:0/1874004414 conn(0x7f8ee80fc500 msgr2=0x7f8ee8103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:21.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 shutdown_connections 2024-08-19T01:34:21.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:21.743+0000 7f8eed67f640 1 -- 172.21.15.103:0/1874004414 wait complete. 2024-08-19T01:34:21.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:21 smithi103 ceph-mon[18436]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-19T01:34:21.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:21 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3416433424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-19T01:34:21.933 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:21 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1874004414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-19T01:34:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:21 smithi190 ceph-mon[22032]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-19T01:34:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:21 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3416433424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-19T01:34:22.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:21 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1874004414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-19T01:34:22.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:21 smithi146 ceph-mon[22098]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-08-19T01:34:22.275 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:21 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3416433424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-19T01:34:22.275 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:21 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1874004414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-19T01:34:22.740 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757297 got 163208757300 for osd.5 2024-08-19T01:34:22.740 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:23.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528365 got 193273528367 for osd.6 2024-08-19T01:34:23.476 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2376086922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81067b0 msgr2=0x7fbcf8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2376086922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81067b0 0x7fbcf8106b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009930 tx=0x7fbce802a420 comp rx=0 tx=0).stop 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2376086922 shutdown_connections 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2376086922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcf8101150 0x7fbcf810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2376086922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf81007b0 0x7fbcf8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2376086922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81067b0 0x7fbcf8106b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2376086922 >> 172.21.15.103:0/2376086922 conn(0x7fbcf80fc4d0 msgr2=0x7fbcf80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.670+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2376086922 shutdown_connections 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2376086922 wait complete. 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 Processor -- start 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 -- start start 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81007b0 0x7fbcf81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf8101150 0x7fbcf8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcf81067b0 0x7fbcf819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8111b80 con 0x7fbcf81007b0 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8111a00 con 0x7fbcf81067b0 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.671+0000 7fbcfe69a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf8111d00 con 0x7fbcf8101150 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81007b0 0x7fbcf81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81007b0 0x7fbcf81980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45748/0 (socket says 172.21.15.103:45748) 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 -- 172.21.15.103:0/2619049449 learned_addr learned my addr 172.21.15.103:0/2619049449 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf77fe640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf8101150 0x7fbcf8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcfcc37640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcf81067b0 0x7fbcf819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 -- 172.21.15.103:0/2619049449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf8101150 msgr2=0x7fbcf8198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf8101150 0x7fbcf8198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 -- 172.21.15.103:0/2619049449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcf81067b0 msgr2=0x7fbcf819c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcf81067b0 0x7fbcf819c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 -- 172.21.15.103:0/2619049449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf819d070 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcfcc37640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcf81067b0 0x7fbcf819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf7fff640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81007b0 0x7fbcf81980c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbce8005940 tx=0x7fbce80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf57fa640 1 -- 172.21.15.103:0/2619049449 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce803f030 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf819d300 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf57fa640 1 -- 172.21.15.103:0/2619049449 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce8003cb0 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcf57fa640 1 -- 172.21.15.103:0/2619049449 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8035070 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.672+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf81a4c40 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.673+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf8101ef0 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.674+0000 7fbcf57fa640 1 -- 172.21.15.103:0/2619049449 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbce8042030 con 0x7fbcf81007b0 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.675+0000 7fbcf57fa640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbce00a2c10 0x7fbce00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.675+0000 7fbcf77fe640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbce00a2c10 0x7fbce00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.675+0000 7fbcf57fa640 1 -- 172.21.15.103:0/2619049449 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbce80ef780 con 0x7fbcf81007b0 2024-08-19T01:34:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.675+0000 7fbcf77fe640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbce00a2c10 0x7fbce00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80febf0 tx=0x7fbcec00a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.679+0000 7fbcf57fa640 1 -- 172.21.15.103:0/2619049449 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbce80bc5b0 con 0x7fbcf81007b0 2024-08-19T01:34:23.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.853+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbcf8100c10 con 0x7fbcf81007b0 2024-08-19T01:34:23.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.853+0000 7fbcf57fa640 1 -- 172.21.15.103:0/2619049449 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbce80bbeb0 con 0x7fbcf81007b0 2024-08-19T01:34:23.854 INFO:teuthology.orchestra.run.smithi103.stdout:77309411395 2024-08-19T01:34:23.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.855+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbce00a2c10 msgr2=0x7fbce00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.855+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbce00a2c10 0x7fbce00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80febf0 tx=0x7fbcec00a4c0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.855+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81007b0 msgr2=0x7fbcf81980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.855+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81007b0 0x7fbcf81980c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbce8005940 tx=0x7fbce80042f0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.855+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 shutdown_connections 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.856+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbce00a2c10 0x7fbce00a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.856+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbcf81067b0 0x7fbcf819c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.856+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf8101150 0x7fbcf8198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.856+0000 7fbcfe69a640 1 --2- 172.21.15.103:0/2619049449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcf81007b0 0x7fbcf81980c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.856+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 >> 172.21.15.103:0/2619049449 conn(0x7fbcf80fc4d0 msgr2=0x7fbcf80fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.856+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 shutdown_connections 2024-08-19T01:34:23.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.856+0000 7fbcfe69a640 1 -- 172.21.15.103:0/2619049449 wait complete. 2024-08-19T01:34:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.890+0000 7f7066ea5640 1 -- 172.21.15.103:0/3625030161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060107fd0 msgr2=0x7f706010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.890+0000 7f7066ea5640 1 --2- 172.21.15.103:0/3625030161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060107fd0 0x7f706010e970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7058009cd0 tx=0x7f705802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.890+0000 7f7066ea5640 1 -- 172.21.15.103:0/3625030161 shutdown_connections 2024-08-19T01:34:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.890+0000 7f7066ea5640 1 --2- 172.21.15.103:0/3625030161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060107fd0 0x7f706010e970 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.890+0000 7f7066ea5640 1 --2- 172.21.15.103:0/3625030161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7060103410 0x7f70601078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.890+0000 7f7066ea5640 1 --2- 172.21.15.103:0/3625030161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7060102a60 0x7f7060102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.890+0000 7f7066ea5640 1 -- 172.21.15.103:0/3625030161 >> 172.21.15.103:0/3625030161 conn(0x7f70600fc500 msgr2=0x7f70600fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.891+0000 7f7066ea5640 1 -- 172.21.15.103:0/3625030161 shutdown_connections 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.891+0000 7f7066ea5640 1 -- 172.21.15.103:0/3625030161 wait complete. 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.891+0000 7f7066ea5640 1 Processor -- start 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.891+0000 7f7066ea5640 1 -- start start 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.891+0000 7f7066ea5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060102a60 0x7f7060198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7066ea5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7060103410 0x7f7060198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7066ea5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7060107fd0 0x7f706019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7066ea5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7060111cd0 con 0x7f7060102a60 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7066ea5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7060111b50 con 0x7f7060103410 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7066ea5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7060111e50 con 0x7f7060107fd0 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060102a60 0x7f7060198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060102a60 0x7f7060198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45772/0 (socket says 172.21.15.103:45772) 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 -- 172.21.15.103:0/1078685775 learned_addr learned my addr 172.21.15.103:0/1078685775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7065442640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7060107fd0 0x7f706019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f705ffff640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7060103410 0x7f7060198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 -- 172.21.15.103:0/1078685775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7060107fd0 msgr2=0x7f706019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7060107fd0 0x7f706019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 -- 172.21.15.103:0/1078685775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7060103410 msgr2=0x7f7060198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7060103410 0x7f7060198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 -- 172.21.15.103:0/1078685775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706019d0f0 con 0x7f7060102a60 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7064c41640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060102a60 0x7f7060198140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f705400c920 tx=0x7f705400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f705dffb640 1 -- 172.21.15.103:0/1078685775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054007c70 con 0x7f7060102a60 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f705dffb640 1 -- 172.21.15.103:0/1078685775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7054007e10 con 0x7f7060102a60 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f705dffb640 1 -- 172.21.15.103:0/1078685775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054018820 con 0x7f7060102a60 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.892+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706019d3e0 con 0x7f7060102a60 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.893+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70601a4cc0 con 0x7f7060102a60 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.893+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7060102e40 con 0x7f7060102a60 2024-08-19T01:34:23.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.894+0000 7f705dffb640 1 -- 172.21.15.103:0/1078685775 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f70540189c0 con 0x7f7060102a60 2024-08-19T01:34:23.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.895+0000 7f705dffb640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f70500a2c90 0x7f70500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:23.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.895+0000 7f705ffff640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f70500a2c90 0x7f70500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:23.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.895+0000 7f705dffb640 1 -- 172.21.15.103:0/1078685775 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f70540cf340 con 0x7f7060102a60 2024-08-19T01:34:23.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.896+0000 7f705ffff640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f70500a2c90 0x7f70500a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f704c005fd0 tx=0x7f704c00b510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:23.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:23.899+0000 7f705dffb640 1 -- 172.21.15.103:0/1078685775 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7054014030 con 0x7f7060102a60 2024-08-19T01:34:24.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.076+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7060109fe0 con 0x7f7060102a60 2024-08-19T01:34:24.077 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:23 smithi103 ceph-mon[18436]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-08-19T01:34:24.077 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:23 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2619049449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-19T01:34:24.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.076+0000 7f705dffb640 1 -- 172.21.15.103:0/1078685775 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f705409c170 con 0x7f7060102a60 2024-08-19T01:34:24.078 INFO:teuthology.orchestra.run.smithi103.stdout:300647710750 2024-08-19T01:34:24.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f70500a2c90 msgr2=0x7f70500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:24.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f70500a2c90 0x7f70500a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f704c005fd0 tx=0x7f704c00b510 comp rx=0 tx=0).stop 2024-08-19T01:34:24.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060102a60 msgr2=0x7f7060198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:24.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060102a60 0x7f7060198140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f705400c920 tx=0x7f705400cde0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 shutdown_connections 2024-08-19T01:34:24.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f70500a2c90 0x7f70500a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7060107fd0 0x7f706019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7060103410 0x7f7060198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 --2- 172.21.15.103:0/1078685775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7060102a60 0x7f7060198140 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:24.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 >> 172.21.15.103:0/1078685775 conn(0x7f70600fc500 msgr2=0x7f7060105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:24.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 shutdown_connections 2024-08-19T01:34:24.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:24.078+0000 7f7066ea5640 1 -- 172.21.15.103:0/1078685775 wait complete. 2024-08-19T01:34:24.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:23 smithi190 ceph-mon[22032]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-08-19T01:34:24.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:23 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2619049449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-19T01:34:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:23 smithi146 ceph-mon[22098]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-08-19T01:34:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:23 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2619049449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-19T01:34:24.901 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710745 got 300647710750 for osd.10 2024-08-19T01:34:24.901 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:25.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 -- 172.21.15.103:0/298398615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1018b0 msgr2=0x7f727c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:25.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 --2- 172.21.15.103:0/298398615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1018b0 0x7f727c101d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7264009930 tx=0x7f726402a420 comp rx=0 tx=0).stop 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 -- 172.21.15.103:0/298398615 shutdown_connections 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 --2- 172.21.15.103:0/298398615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f727c102250 0x7f727c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 --2- 172.21.15.103:0/298398615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1018b0 0x7f727c101d10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 --2- 172.21.15.103:0/298398615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f727c1078b0 0x7f727c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 -- 172.21.15.103:0/298398615 >> 172.21.15.103:0/298398615 conn(0x7f727c0fd5f0 msgr2=0x7f727c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 -- 172.21.15.103:0/298398615 shutdown_connections 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.098+0000 7f728082b640 1 -- 172.21.15.103:0/298398615 wait complete. 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 Processor -- start 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 -- start start 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f727c1018b0 0x7f727c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f727c102250 0x7f727c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:25.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1078b0 0x7f727c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f727c111b60 con 0x7f727c1078b0 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1119e0 con 0x7f727c1018b0 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f728082b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f727c111ce0 con 0x7f727c102250 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f727ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1078b0 0x7f727c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f727ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1078b0 0x7f727c19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45792/0 (socket says 172.21.15.103:45792) 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.099+0000 7f727ad9d640 1 -- 172.21.15.103:0/2305481444 learned_addr learned my addr 172.21.15.103:0/2305481444 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f727a59c640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f727c1018b0 0x7f727c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f727ad9d640 1 -- 172.21.15.103:0/2305481444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f727c102250 msgr2=0x7f727c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f727ad9d640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f727c102250 0x7f727c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f727ad9d640 1 -- 172.21.15.103:0/2305481444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f727c1018b0 msgr2=0x7f727c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f727ad9d640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f727c1018b0 0x7f727c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f727ad9d640 1 -- 172.21.15.103:0/2305481444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c19d050 con 0x7f727c1078b0 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f727ad9d640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1078b0 0x7f727c19c970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f727400cce0 tx=0x7f727400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:25.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f72737fe640 1 -- 172.21.15.103:0/2305481444 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727400f450 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c19d340 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f72737fe640 1 -- 172.21.15.103:0/2305481444 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727400faa0 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f72737fe640 1 -- 172.21.15.103:0/2305481444 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274018b70 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.100+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a4c20 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.101+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c101d10 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.102+0000 7f72737fe640 1 -- 172.21.15.103:0/2305481444 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7274018d10 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.103+0000 7f72737fe640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72680a2c10 0x7f72680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.103+0000 7f727a59c640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72680a2c10 0x7f72680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.103+0000 7f72737fe640 1 -- 172.21.15.103:0/2305481444 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f72740cf740 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.103+0000 7f727a59c640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72680a2c10 0x7f72680a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f726c005d90 tx=0x7f726c005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:25.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.107+0000 7f72737fe640 1 -- 172.21.15.103:0/2305481444 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7274014030 con 0x7f727c1078b0 2024-08-19T01:34:25.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:24 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1078685775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-19T01:34:25.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:24 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1078685775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-19T01:34:25.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:24 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1078685775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-19T01:34:25.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.275+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f727c109e80 con 0x7f727c1078b0 2024-08-19T01:34:25.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.275+0000 7f72737fe640 1 -- 172.21.15.103:0/2305481444 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f727409c510 con 0x7f727c1078b0 2024-08-19T01:34:25.276 INFO:teuthology.orchestra.run.smithi103.stdout:335007449114 2024-08-19T01:34:25.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72680a2c10 msgr2=0x7f72680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:25.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72680a2c10 0x7f72680a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f726c005d90 tx=0x7f726c005d20 comp rx=0 tx=0).stop 2024-08-19T01:34:25.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1078b0 msgr2=0x7f727c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1078b0 0x7f727c19c970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f727400cce0 tx=0x7f727400a620 comp rx=0 tx=0).stop 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 shutdown_connections 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72680a2c10 0x7f72680a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f727c1078b0 0x7f727c19c970 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f727c102250 0x7f727c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 --2- 172.21.15.103:0/2305481444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f727c1018b0 0x7f727c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.277+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 >> 172.21.15.103:0/2305481444 conn(0x7f727c0fd5f0 msgr2=0x7f727c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.278+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 shutdown_connections 2024-08-19T01:34:25.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:25.278+0000 7f728082b640 1 -- 172.21.15.103:0/2305481444 wait complete. 2024-08-19T01:34:25.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411390 got 77309411395 for osd.2 2024-08-19T01:34:25.977 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:26.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:25 smithi190 ceph-mon[22032]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:26.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:25 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2305481444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-19T01:34:26.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:25 smithi146 ceph-mon[22098]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:26.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:25 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2305481444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-19T01:34:26.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:25 smithi103 ceph-mon[18436]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:25 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2305481444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-19T01:34:26.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449112 got 335007449114 for osd.11 2024-08-19T01:34:26.718 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:27.515 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920822 got 141733920825 for osd.4 2024-08-19T01:34:27.515 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 -- 172.21.15.103:0/193049101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 msgr2=0x7f2f54102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/193049101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f54102eb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009900 tx=0x7f2f4802a480 comp rx=0 tx=0).stop 2024-08-19T01:34:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 -- 172.21.15.103:0/193049101 shutdown_connections 2024-08-19T01:34:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/193049101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5410beb0 0x7f2f5410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/193049101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f541033f0 0x7f2f5410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/193049101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f54102eb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 -- 172.21.15.103:0/193049101 >> 172.21.15.103:0/193049101 conn(0x7f2f540fc4d0 msgr2=0x7f2f540fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 -- 172.21.15.103:0/193049101 shutdown_connections 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.709+0000 7f2f5b09c640 1 -- 172.21.15.103:0/193049101 wait complete. 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 Processor -- start 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 -- start start 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f541980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f541033f0 0x7f2f541985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5410beb0 0x7f2f5419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54111b40 con 0x7f2f54102ad0 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541119c0 con 0x7f2f5410beb0 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f5b09c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54111cc0 con 0x7f2f541033f0 2024-08-19T01:34:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f58e38640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f541980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f58e38640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f541980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45814/0 (socket says 172.21.15.103:45814) 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f58e38640 1 -- 172.21.15.103:0/3541964760 learned_addr learned my addr 172.21.15.103:0/3541964760 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f53fff640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f541033f0 0x7f2f541985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.710+0000 7f2f59639640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5410beb0 0x7f2f5419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f58e38640 1 -- 172.21.15.103:0/3541964760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f541033f0 msgr2=0x7f2f541985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f58e38640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f541033f0 0x7f2f541985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f58e38640 1 -- 172.21.15.103:0/3541964760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5410beb0 msgr2=0x7f2f5419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f58e38640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5410beb0 0x7f2f5419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f58e38640 1 -- 172.21.15.103:0/3541964760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5419d050 con 0x7f2f54102ad0 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f53fff640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f541033f0 0x7f2f541985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f59639640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5410beb0 0x7f2f5419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f58e38640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f541980a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f48005940 tx=0x7f2f48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f51ffb640 1 -- 172.21.15.103:0/3541964760 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4803f030 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5419d2e0 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f51ffb640 1 -- 172.21.15.103:0/3541964760 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f48003aa0 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f51ffb640 1 -- 172.21.15.103:0/3541964760 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48035070 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.711+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541a4c20 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.712+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5406ad40 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.713+0000 7f2f51ffb640 1 -- 172.21.15.103:0/3541964760 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2f48042030 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.714+0000 7f2f51ffb640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f440a2c10 0x7f2f440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.714+0000 7f2f53fff640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f440a2c10 0x7f2f440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.714+0000 7f2f51ffb640 1 -- 172.21.15.103:0/3541964760 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f2f480efa40 con 0x7f2f54102ad0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.714+0000 7f2f53fff640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f440a2c10 0x7f2f440a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f540feb40 tx=0x7f2f4000b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.718+0000 7f2f51ffb640 1 -- 172.21.15.103:0/3541964760 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2f480bc870 con 0x7f2f54102ad0 2024-08-19T01:34:27.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.898+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2f54102eb0 con 0x7f2f54102ad0 2024-08-19T01:34:27.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.898+0000 7f2f51ffb640 1 -- 172.21.15.103:0/3541964760 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f480bc170 con 0x7f2f54102ad0 2024-08-19T01:34:27.900 INFO:teuthology.orchestra.run.smithi103.stdout:34359738444 2024-08-19T01:34:27.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f440a2c10 msgr2=0x7f2f440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:27.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f440a2c10 0x7f2f440a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f540feb40 tx=0x7f2f4000b5f0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 msgr2=0x7f2f541980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:27.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f541980a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f48005940 tx=0x7f2f48004270 comp rx=0 tx=0).stop 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 shutdown_connections 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f2f440a2c10 0x7f2f440a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5410beb0 0x7f2f5419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f541033f0 0x7f2f541985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 --2- 172.21.15.103:0/3541964760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f54102ad0 0x7f2f541980a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 >> 172.21.15.103:0/3541964760 conn(0x7f2f540fc4d0 msgr2=0x7f2f540fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 shutdown_connections 2024-08-19T01:34:27.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:27.900+0000 7f2f5b09c640 1 -- 172.21.15.103:0/3541964760 wait complete. 2024-08-19T01:34:28.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:27 smithi190 ceph-mon[22032]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:27 smithi146 ceph-mon[22098]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:28.219 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:27 smithi103 ceph-mon[18436]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:29.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:28 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3541964760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-19T01:34:29.190 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:28 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3541964760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-19T01:34:29.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:28 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3541964760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-19T01:34:29.220 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738440 got 34359738444 for osd.0 2024-08-19T01:34:29.220 DEBUG:teuthology.parallel:result is None 2024-08-19T01:34:29.221 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-19T01:34:29.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:34:29.494 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:30.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 -- 172.21.15.103:0/1051558213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8181033f0 msgr2=0x7ff81810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/1051558213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8181033f0 0x7ff81810b8e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff808009900 tx=0x7ff80802a480 comp rx=0 tx=0).stop 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 -- 172.21.15.103:0/1051558213 shutdown_connections 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/1051558213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff81810beb0 0x7ff81810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/1051558213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8181033f0 0x7ff81810b8e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/1051558213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff818102ad0 0x7ff818102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 -- 172.21.15.103:0/1051558213 >> 172.21.15.103:0/1051558213 conn(0x7ff8180fc4d0 msgr2=0x7ff8180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 -- 172.21.15.103:0/1051558213 shutdown_connections 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.017+0000 7ff81f5f9640 1 -- 172.21.15.103:0/1051558213 wait complete. 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 Processor -- start 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 -- start start 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818102ad0 0x7ff818198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8181033f0 0x7ff8181985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff81810beb0 0x7ff81819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff818111b40 con 0x7ff81810beb0 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181119c0 con 0x7ff8181033f0 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81f5f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff818111cc0 con 0x7ff818102ad0 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81db96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff81810beb0 0x7ff81819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81db96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff81810beb0 0x7ff81819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45828/0 (socket says 172.21.15.103:45828) 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81db96640 1 -- 172.21.15.103:0/3832496291 learned_addr learned my addr 172.21.15.103:0/3832496291 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81cb94640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8181033f0 0x7ff8181985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.018+0000 7ff81d395640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818102ad0 0x7ff818198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81db96640 1 -- 172.21.15.103:0/3832496291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818102ad0 msgr2=0x7ff818198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81db96640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818102ad0 0x7ff818198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81db96640 1 -- 172.21.15.103:0/3832496291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8181033f0 msgr2=0x7ff8181985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81db96640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8181033f0 0x7ff8181985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81db96640 1 -- 172.21.15.103:0/3832496291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81819d030 con 0x7ff81810beb0 2024-08-19T01:34:30.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81db96640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff81810beb0 0x7ff81819c950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff81000eb10 tx=0x7ff81000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff80e7fc640 1 -- 172.21.15.103:0/3832496291 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810004b20 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff80e7fc640 1 -- 172.21.15.103:0/3832496291 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff810004cc0 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81819d320 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff80e7fc640 1 -- 172.21.15.103:0/3832496291 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810010690 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.019+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a4c00 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.020+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff818102eb0 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.021+0000 7ff80e7fc640 1 -- 172.21.15.103:0/3832496291 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff810005020 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.022+0000 7ff80e7fc640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8040a2c10 0x7ff8040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.022+0000 7ff81d395640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8040a2c10 0x7ff8040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.022+0000 7ff80e7fc640 1 -- 172.21.15.103:0/3832496291 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff8100cf6b0 con 0x7ff81810beb0 2024-08-19T01:34:30.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.022+0000 7ff81d395640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8040a2c10 0x7ff8040a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff814009bd0 tx=0x7ff814005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:30.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.026+0000 7ff80e7fc640 1 -- 172.21.15.103:0/3832496291 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff81009c4e0 con 0x7ff81810beb0 2024-08-19T01:34:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:29 smithi146 ceph-mon[22098]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:29 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:29 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.213+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff818062360 con 0x7ff8040a2c10 2024-08-19T01:34:30.215 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:29 smithi103 ceph-mon[18436]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:30.215 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:29 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:30.216 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:29 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:30.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.219+0000 7ff80e7fc640 1 -- 172.21.15.103:0/3832496291 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258988 (secure 0 0 0) 0x7ff818062360 con 0x7ff8040a2c10 2024-08-19T01:34:30.224 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:30.225 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8040a2c10 msgr2=0x7ff8040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8040a2c10 0x7ff8040a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff814009bd0 tx=0x7ff814005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff81810beb0 msgr2=0x7ff81819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff81810beb0 0x7ff81819c950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff81000eb10 tx=0x7ff81000c2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 shutdown_connections 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8040a2c10 0x7ff8040a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff81810beb0 0x7ff81819c950 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8181033f0 0x7ff8181985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 --2- 172.21.15.103:0/3832496291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff818102ad0 0x7ff818198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:30.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 >> 172.21.15.103:0/3832496291 conn(0x7ff8180fc4d0 msgr2=0x7ff8180fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:30.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.222+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 shutdown_connections 2024-08-19T01:34:30.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:30.223+0000 7ff81f5f9640 1 -- 172.21.15.103:0/3832496291 wait complete. 2024-08-19T01:34:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:29 smithi190 ceph-mon[22032]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:29 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:30.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:29 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:30.979 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-08-19T01:34:28.786865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658810,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":681,"num_read_kb":460,"num_write":427,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364928,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576896,"statfs":{"total":1151940427776,"available":1151566741504,"internally_reserved":0,"allocated":11833344,"data_stored":7409778,"data_compressed":112947,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003182"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880448+0000","last_change":"2024-08-19T01:33:23.202035+0000","last_active":"2024-08-19T01:34:01.880448+0000","last_peered":"2024-08-19T01:34:01.880448+0000","last_clean":"2024-08-19T01:34:01.880448+0000","last_became_active":"2024-08-19T01:33:23.201418+0000","last_became_peered":"2024-08-19T01:33:23.201418+0000","last_unstale":"2024-08-19T01:34:01.880448+0000","last_undegraded":"2024-08-19T01:34:01.880448+0000","last_fullsized":"2024-08-19T01:34:01.880448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:54:57.927424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882924+0000","last_change":"2024-08-19T01:33:23.202047+0000","last_active":"2024-08-19T01:34:01.882924+0000","last_peered":"2024-08-19T01:34:01.882924+0000","last_clean":"2024-08-19T01:34:01.882924+0000","last_became_active":"2024-08-19T01:33:23.201762+0000","last_became_peered":"2024-08-19T01:33:23.201762+0000","last_unstale":"2024-08-19T01:34:01.882924+0000","last_undegraded":"2024-08-19T01:34:01.882924+0000","last_fullsized":"2024-08-19T01:34:01.882924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:23:57.252284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882773+0000","last_change":"2024-08-19T01:33:23.202876+0000","last_active":"2024-08-19T01:34:01.882773+0000","last_peered":"2024-08-19T01:34:01.882773+0000","last_clean":"2024-08-19T01:34:01.882773+0000","last_became_active":"2024-08-19T01:33:23.202528+0000","last_became_peered":"2024-08-19T01:33:23.202528+0000","last_unstale":"2024-08-19T01:34:01.882773+0000","last_undegraded":"2024-08-19T01:34:01.882773+0000","last_fullsized":"2024-08-19T01:34:01.882773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:20.514390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878534+0000","last_change":"2024-08-19T01:33:23.199443+0000","last_active":"2024-08-19T01:34:01.878534+0000","last_peered":"2024-08-19T01:34:01.878534+0000","last_clean":"2024-08-19T01:34:01.878534+0000","last_became_active":"2024-08-19T01:33:23.198344+0000","last_became_peered":"2024-08-19T01:33:23.198344+0000","last_unstale":"2024-08-19T01:34:01.878534+0000","last_undegraded":"2024-08-19T01:34:01.878534+0000","last_fullsized":"2024-08-19T01:34:01.878534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:43:43.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883556+0000","last_change":"2024-08-19T01:33:23.191185+0000","last_active":"2024-08-19T01:34:01.883556+0000","last_peered":"2024-08-19T01:34:01.883556+0000","last_clean":"2024-08-19T01:34:01.883556+0000","last_became_active":"2024-08-19T01:33:23.190892+0000","last_became_peered":"2024-08-19T01:33:23.190892+0000","last_unstale":"2024-08-19T01:34:01.883556+0000","last_undegraded":"2024-08-19T01:34:01.883556+0000","last_fullsized":"2024-08-19T01:34:01.883556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:28.384609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880419+0000","last_change":"2024-08-19T01:33:23.193036+0000","last_active":"2024-08-19T01:34:01.880419+0000","last_peered":"2024-08-19T01:34:01.880419+0000","last_clean":"2024-08-19T01:34:01.880419+0000","last_became_active":"2024-08-19T01:33:23.192686+0000","last_became_peered":"2024-08-19T01:33:23.192686+0000","last_unstale":"2024-08-19T01:34:01.880419+0000","last_undegraded":"2024-08-19T01:34:01.880419+0000","last_fullsized":"2024-08-19T01:34:01.880419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:46.235112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880846+0000","last_change":"2024-08-19T01:33:23.197636+0000","last_active":"2024-08-19T01:34:01.880846+0000","last_peered":"2024-08-19T01:34:01.880846+0000","last_clean":"2024-08-19T01:34:01.880846+0000","last_became_active":"2024-08-19T01:33:23.197323+0000","last_became_peered":"2024-08-19T01:33:23.197323+0000","last_unstale":"2024-08-19T01:34:01.880846+0000","last_undegraded":"2024-08-19T01:34:01.880846+0000","last_fullsized":"2024-08-19T01:34:01.880846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:46:36.168786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877768+0000","last_change":"2024-08-19T01:33:23.193717+0000","last_active":"2024-08-19T01:34:01.877768+0000","last_peered":"2024-08-19T01:34:01.877768+0000","last_clean":"2024-08-19T01:34:01.877768+0000","last_became_active":"2024-08-19T01:33:23.193376+0000","last_became_peered":"2024-08-19T01:33:23.193376+0000","last_unstale":"2024-08-19T01:34:01.877768+0000","last_undegraded":"2024-08-19T01:34:01.877768+0000","last_fullsized":"2024-08-19T01:34:01.877768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:16.290081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048946+0000","last_change":"2024-08-19T01:33:23.197956+0000","last_active":"2024-08-19T01:34:02.048946+0000","last_peered":"2024-08-19T01:34:02.048946+0000","last_clean":"2024-08-19T01:34:02.048946+0000","last_became_active":"2024-08-19T01:33:23.197667+0000","last_became_peered":"2024-08-19T01:33:23.197667+0000","last_unstale":"2024-08-19T01:34:02.048946+0000","last_undegraded":"2024-08-19T01:34:02.048946+0000","last_fullsized":"2024-08-19T01:34:02.048946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:16.841046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881099+0000","last_change":"2024-08-19T01:33:23.194569+0000","last_active":"2024-08-19T01:34:01.881099+0000","last_peered":"2024-08-19T01:34:01.881099+0000","last_clean":"2024-08-19T01:34:01.881099+0000","last_became_active":"2024-08-19T01:33:23.194223+0000","last_became_peered":"2024-08-19T01:33:23.194223+0000","last_unstale":"2024-08-19T01:34:01.881099+0000","last_undegraded":"2024-08-19T01:34:01.881099+0000","last_fullsized":"2024-08-19T01:34:01.881099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:14:30.184738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881455+0000","last_change":"2024-08-19T01:33:23.202396+0000","last_active":"2024-08-19T01:34:01.881455+0000","last_peered":"2024-08-19T01:34:01.881455+0000","last_clean":"2024-08-19T01:34:01.881455+0000","last_became_active":"2024-08-19T01:33:23.202063+0000","last_became_peered":"2024-08-19T01:33:23.202063+0000","last_unstale":"2024-08-19T01:34:01.881455+0000","last_undegraded":"2024-08-19T01:34:01.881455+0000","last_fullsized":"2024-08-19T01:34:01.881455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:57.513639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878563+0000","last_change":"2024-08-19T01:33:23.200610+0000","last_active":"2024-08-19T01:34:01.878563+0000","last_peered":"2024-08-19T01:34:01.878563+0000","last_clean":"2024-08-19T01:34:01.878563+0000","last_became_active":"2024-08-19T01:33:23.200136+0000","last_became_peered":"2024-08-19T01:33:23.200136+0000","last_unstale":"2024-08-19T01:34:01.878563+0000","last_undegraded":"2024-08-19T01:34:01.878563+0000","last_fullsized":"2024-08-19T01:34:01.878563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:47:39.252702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929133+0000","last_change":"2024-08-19T01:33:23.196062+0000","last_active":"2024-08-19T01:34:01.929133+0000","last_peered":"2024-08-19T01:34:01.929133+0000","last_clean":"2024-08-19T01:34:01.929133+0000","last_became_active":"2024-08-19T01:33:23.195577+0000","last_became_peered":"2024-08-19T01:33:23.195577+0000","last_unstale":"2024-08-19T01:34:01.929133+0000","last_undegraded":"2024-08-19T01:34:01.929133+0000","last_fullsized":"2024-08-19T01:34:01.929133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:46:09.111323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882617+0000","last_change":"2024-08-19T01:33:23.201664+0000","last_active":"2024-08-19T01:34:01.882617+0000","last_peered":"2024-08-19T01:34:01.882617+0000","last_clean":"2024-08-19T01:34:01.882617+0000","last_became_active":"2024-08-19T01:33:23.200241+0000","last_became_peered":"2024-08-19T01:33:23.200241+0000","last_unstale":"2024-08-19T01:34:01.882617+0000","last_undegraded":"2024-08-19T01:34:01.882617+0000","last_fullsized":"2024-08-19T01:34:01.882617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:17.199776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882446+0000","last_change":"2024-08-19T01:33:23.198102+0000","last_active":"2024-08-19T01:34:01.882446+0000","last_peered":"2024-08-19T01:34:01.882446+0000","last_clean":"2024-08-19T01:34:01.882446+0000","last_became_active":"2024-08-19T01:33:23.197778+0000","last_became_peered":"2024-08-19T01:33:23.197778+0000","last_unstale":"2024-08-19T01:34:01.882446+0000","last_undegraded":"2024-08-19T01:34:01.882446+0000","last_fullsized":"2024-08-19T01:34:01.882446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:03.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048327+0000","last_change":"2024-08-19T01:33:23.194581+0000","last_active":"2024-08-19T01:34:02.048327+0000","last_peered":"2024-08-19T01:34:02.048327+0000","last_clean":"2024-08-19T01:34:02.048327+0000","last_became_active":"2024-08-19T01:33:23.194256+0000","last_became_peered":"2024-08-19T01:33:23.194256+0000","last_unstale":"2024-08-19T01:34:02.048327+0000","last_undegraded":"2024-08-19T01:34:02.048327+0000","last_fullsized":"2024-08-19T01:34:02.048327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:17:20.954190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880474+0000","last_change":"2024-08-19T01:33:23.201242+0000","last_active":"2024-08-19T01:34:01.880474+0000","last_peered":"2024-08-19T01:34:01.880474+0000","last_clean":"2024-08-19T01:34:01.880474+0000","last_became_active":"2024-08-19T01:33:23.200835+0000","last_became_peered":"2024-08-19T01:33:23.200835+0000","last_unstale":"2024-08-19T01:34:01.880474+0000","last_undegraded":"2024-08-19T01:34:01.880474+0000","last_fullsized":"2024-08-19T01:34:01.880474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:35:54.312266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880057+0000","last_change":"2024-08-19T01:33:23.187728+0000","last_active":"2024-08-19T01:34:01.880057+0000","last_peered":"2024-08-19T01:34:01.880057+0000","last_clean":"2024-08-19T01:34:01.880057+0000","last_became_active":"2024-08-19T01:33:23.187352+0000","last_became_peered":"2024-08-19T01:33:23.187352+0000","last_unstale":"2024-08-19T01:34:01.880057+0000","last_undegraded":"2024-08-19T01:34:01.880057+0000","last_fullsized":"2024-08-19T01:34:01.880057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:06:03.041328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883303+0000","last_change":"2024-08-19T01:33:23.193146+0000","last_active":"2024-08-19T01:34:01.883303+0000","last_peered":"2024-08-19T01:34:01.883303+0000","last_clean":"2024-08-19T01:34:01.883303+0000","last_became_active":"2024-08-19T01:33:23.192810+0000","last_became_peered":"2024-08-19T01:33:23.192810+0000","last_unstale":"2024-08-19T01:34:01.883303+0000","last_undegraded":"2024-08-19T01:34:01.883303+0000","last_fullsized":"2024-08-19T01:34:01.883303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:12.044175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880515+0000","last_change":"2024-08-19T01:33:23.194635+0000","last_active":"2024-08-19T01:34:01.880515+0000","last_peered":"2024-08-19T01:34:01.880515+0000","last_clean":"2024-08-19T01:34:01.880515+0000","last_became_active":"2024-08-19T01:33:23.194308+0000","last_became_peered":"2024-08-19T01:33:23.194308+0000","last_unstale":"2024-08-19T01:34:01.880515+0000","last_undegraded":"2024-08-19T01:34:01.880515+0000","last_fullsized":"2024-08-19T01:34:01.880515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:12.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929187+0000","last_change":"2024-08-19T01:33:23.193108+0000","last_active":"2024-08-19T01:34:01.929187+0000","last_peered":"2024-08-19T01:34:01.929187+0000","last_clean":"2024-08-19T01:34:01.929187+0000","last_became_active":"2024-08-19T01:33:23.192830+0000","last_became_peered":"2024-08-19T01:33:23.192830+0000","last_unstale":"2024-08-19T01:34:01.929187+0000","last_undegraded":"2024-08-19T01:34:01.929187+0000","last_fullsized":"2024-08-19T01:34:01.929187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:33.355498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048496+0000","last_change":"2024-08-19T01:33:23.195789+0000","last_active":"2024-08-19T01:34:02.048496+0000","last_peered":"2024-08-19T01:34:02.048496+0000","last_clean":"2024-08-19T01:34:02.048496+0000","last_became_active":"2024-08-19T01:33:23.195064+0000","last_became_peered":"2024-08-19T01:33:23.195064+0000","last_unstale":"2024-08-19T01:34:02.048496+0000","last_undegraded":"2024-08-19T01:34:02.048496+0000","last_fullsized":"2024-08-19T01:34:02.048496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:05:33.848953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877781+0000","last_change":"2024-08-19T01:33:23.200968+0000","last_active":"2024-08-19T01:34:01.877781+0000","last_peered":"2024-08-19T01:34:01.877781+0000","last_clean":"2024-08-19T01:34:01.877781+0000","last_became_active":"2024-08-19T01:33:23.200606+0000","last_became_peered":"2024-08-19T01:33:23.200606+0000","last_unstale":"2024-08-19T01:34:01.877781+0000","last_undegraded":"2024-08-19T01:34:01.877781+0000","last_fullsized":"2024-08-19T01:34:01.877781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:44.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878995+0000","last_change":"2024-08-19T01:33:23.191204+0000","last_active":"2024-08-19T01:34:01.878995+0000","last_peered":"2024-08-19T01:34:01.878995+0000","last_clean":"2024-08-19T01:34:01.878995+0000","last_became_active":"2024-08-19T01:33:23.190709+0000","last_became_peered":"2024-08-19T01:33:23.190709+0000","last_unstale":"2024-08-19T01:34:01.878995+0000","last_undegraded":"2024-08-19T01:34:01.878995+0000","last_fullsized":"2024-08-19T01:34:01.878995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:56:10.988967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.885389+0000","last_change":"2024-08-19T01:34:01.885389+0000","last_active":"2024-08-19T01:34:01.885389+0000","last_peered":"2024-08-19T01:34:01.885389+0000","last_clean":"2024-08-19T01:34:01.885389+0000","last_became_active":"2024-08-19T01:34:01.885100+0000","last_became_peered":"2024-08-19T01:34:01.885100+0000","last_unstale":"2024-08-19T01:34:01.885389+0000","last_undegraded":"2024-08-19T01:34:01.885389+0000","last_fullsized":"2024-08-19T01:34:01.885389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:21.616876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882484+0000","last_change":"2024-08-19T01:33:23.187309+0000","last_active":"2024-08-19T01:34:01.882484+0000","last_peered":"2024-08-19T01:34:01.882484+0000","last_clean":"2024-08-19T01:34:01.882484+0000","last_became_active":"2024-08-19T01:33:23.186976+0000","last_became_peered":"2024-08-19T01:33:23.186976+0000","last_unstale":"2024-08-19T01:34:01.882484+0000","last_undegraded":"2024-08-19T01:34:01.882484+0000","last_fullsized":"2024-08-19T01:34:01.882484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:46:02.852000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880176+0000","last_change":"2024-08-19T01:33:23.190292+0000","last_active":"2024-08-19T01:34:01.880176+0000","last_peered":"2024-08-19T01:34:01.880176+0000","last_clean":"2024-08-19T01:34:01.880176+0000","last_became_active":"2024-08-19T01:33:23.189969+0000","last_became_peered":"2024-08-19T01:33:23.189969+0000","last_unstale":"2024-08-19T01:34:01.880176+0000","last_undegraded":"2024-08-19T01:34:01.880176+0000","last_fullsized":"2024-08-19T01:34:01.880176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:35:51.388150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929083+0000","last_change":"2024-08-19T01:33:23.192148+0000","last_active":"2024-08-19T01:34:01.929083+0000","last_peered":"2024-08-19T01:34:01.929083+0000","last_clean":"2024-08-19T01:34:01.929083+0000","last_became_active":"2024-08-19T01:33:23.191845+0000","last_became_peered":"2024-08-19T01:33:23.191845+0000","last_unstale":"2024-08-19T01:34:01.929083+0000","last_undegraded":"2024-08-19T01:34:01.929083+0000","last_fullsized":"2024-08-19T01:34:01.929083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:51.308393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882840+0000","last_change":"2024-08-19T01:33:23.195033+0000","last_active":"2024-08-19T01:34:01.882840+0000","last_peered":"2024-08-19T01:34:01.882840+0000","last_clean":"2024-08-19T01:34:01.882840+0000","last_became_active":"2024-08-19T01:33:23.194745+0000","last_became_peered":"2024-08-19T01:33:23.194745+0000","last_unstale":"2024-08-19T01:34:01.882840+0000","last_undegraded":"2024-08-19T01:34:01.882840+0000","last_fullsized":"2024-08-19T01:34:01.882840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:33:40.748311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882344+0000","last_change":"2024-08-19T01:33:23.195817+0000","last_active":"2024-08-19T01:34:01.882344+0000","last_peered":"2024-08-19T01:34:01.882344+0000","last_clean":"2024-08-19T01:34:01.882344+0000","last_became_active":"2024-08-19T01:33:23.194688+0000","last_became_peered":"2024-08-19T01:33:23.194688+0000","last_unstale":"2024-08-19T01:34:01.882344+0000","last_undegraded":"2024-08-19T01:34:01.882344+0000","last_fullsized":"2024-08-19T01:34:01.882344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:13:04.913921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883174+0000","last_change":"2024-08-19T01:33:23.187604+0000","last_active":"2024-08-19T01:34:01.883174+0000","last_peered":"2024-08-19T01:34:01.883174+0000","last_clean":"2024-08-19T01:34:01.883174+0000","last_became_active":"2024-08-19T01:33:23.187242+0000","last_became_peered":"2024-08-19T01:33:23.187242+0000","last_unstale":"2024-08-19T01:34:01.883174+0000","last_undegraded":"2024-08-19T01:34:01.883174+0000","last_fullsized":"2024-08-19T01:34:01.883174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:28.592525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879987+0000","last_change":"2024-08-19T01:33:23.191737+0000","last_active":"2024-08-19T01:34:01.879987+0000","last_peered":"2024-08-19T01:34:01.879987+0000","last_clean":"2024-08-19T01:34:01.879987+0000","last_became_active":"2024-08-19T01:33:23.191358+0000","last_became_peered":"2024-08-19T01:33:23.191358+0000","last_unstale":"2024-08-19T01:34:01.879987+0000","last_undegraded":"2024-08-19T01:34:01.879987+0000","last_fullsized":"2024-08-19T01:34:01.879987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:28:14.694157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883084+0000","last_change":"2024-08-19T01:33:23.193110+0000","last_active":"2024-08-19T01:34:01.883084+0000","last_peered":"2024-08-19T01:34:01.883084+0000","last_clean":"2024-08-19T01:34:01.883084+0000","last_became_active":"2024-08-19T01:33:23.192789+0000","last_became_peered":"2024-08-19T01:33:23.192789+0000","last_unstale":"2024-08-19T01:34:01.883084+0000","last_undegraded":"2024-08-19T01:34:01.883084+0000","last_fullsized":"2024-08-19T01:34:01.883084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:45:48.141403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":84,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.049980+0000","last_change":"2024-08-19T01:33:21.173518+0000","last_active":"2024-08-19T01:34:26.049980+0000","last_peered":"2024-08-19T01:34:26.049980+0000","last_clean":"2024-08-19T01:34:26.049980+0000","last_became_active":"2024-08-19T01:33:21.172288+0000","last_became_peered":"2024-08-19T01:33:21.172288+0000","last_unstale":"2024-08-19T01:34:26.049980+0000","last_undegraded":"2024-08-19T01:34:26.049980+0000","last_fullsized":"2024-08-19T01:34:26.049980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:59:42.864724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879305+0000","last_change":"2024-08-19T01:33:23.190160+0000","last_active":"2024-08-19T01:34:01.879305+0000","last_peered":"2024-08-19T01:34:01.879305+0000","last_clean":"2024-08-19T01:34:01.879305+0000","last_became_active":"2024-08-19T01:33:23.189802+0000","last_became_peered":"2024-08-19T01:33:23.189802+0000","last_unstale":"2024-08-19T01:34:01.879305+0000","last_undegraded":"2024-08-19T01:34:01.879305+0000","last_fullsized":"2024-08-19T01:34:01.879305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:53:10.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880543+0000","last_change":"2024-08-19T01:33:21.167443+0000","last_active":"2024-08-19T01:34:01.880543+0000","last_peered":"2024-08-19T01:34:01.880543+0000","last_clean":"2024-08-19T01:34:01.880543+0000","last_became_active":"2024-08-19T01:33:21.167054+0000","last_became_peered":"2024-08-19T01:33:21.167054+0000","last_unstale":"2024-08-19T01:34:01.880543+0000","last_undegraded":"2024-08-19T01:34:01.880543+0000","last_fullsized":"2024-08-19T01:34:01.880543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:00:38.358807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928860+0000","last_change":"2024-08-19T01:33:21.163097+0000","last_active":"2024-08-19T01:34:01.928860+0000","last_peered":"2024-08-19T01:34:01.928860+0000","last_clean":"2024-08-19T01:34:01.928860+0000","last_became_active":"2024-08-19T01:33:21.162802+0000","last_became_peered":"2024-08-19T01:33:21.162802+0000","last_unstale":"2024-08-19T01:34:01.928860+0000","last_undegraded":"2024-08-19T01:34:01.928860+0000","last_fullsized":"2024-08-19T01:34:01.928860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:15:41.244104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877814+0000","last_change":"2024-08-19T01:33:23.200975+0000","last_active":"2024-08-19T01:34:01.877814+0000","last_peered":"2024-08-19T01:34:01.877814+0000","last_clean":"2024-08-19T01:34:01.877814+0000","last_became_active":"2024-08-19T01:33:23.200667+0000","last_became_peered":"2024-08-19T01:33:23.200667+0000","last_unstale":"2024-08-19T01:34:01.877814+0000","last_undegraded":"2024-08-19T01:34:01.877814+0000","last_fullsized":"2024-08-19T01:34:01.877814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:35.105521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879336+0000","last_change":"2024-08-19T01:33:21.171252+0000","last_active":"2024-08-19T01:34:01.879336+0000","last_peered":"2024-08-19T01:34:01.879336+0000","last_clean":"2024-08-19T01:34:01.879336+0000","last_became_active":"2024-08-19T01:33:21.170944+0000","last_became_peered":"2024-08-19T01:33:21.170944+0000","last_unstale":"2024-08-19T01:34:01.879336+0000","last_undegraded":"2024-08-19T01:34:01.879336+0000","last_fullsized":"2024-08-19T01:34:01.879336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:10:15.869452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879832+0000","last_change":"2024-08-19T01:33:23.191931+0000","last_active":"2024-08-19T01:34:01.879832+0000","last_peered":"2024-08-19T01:34:01.879832+0000","last_clean":"2024-08-19T01:34:01.879832+0000","last_became_active":"2024-08-19T01:33:23.191601+0000","last_became_peered":"2024-08-19T01:33:23.191601+0000","last_unstale":"2024-08-19T01:34:01.879832+0000","last_undegraded":"2024-08-19T01:34:01.879832+0000","last_fullsized":"2024-08-19T01:34:01.879832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:52.874546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":130,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.049223+0000","last_change":"2024-08-19T01:33:21.170714+0000","last_active":"2024-08-19T01:34:26.049223+0000","last_peered":"2024-08-19T01:34:26.049223+0000","last_clean":"2024-08-19T01:34:26.049223+0000","last_became_active":"2024-08-19T01:33:21.170379+0000","last_became_peered":"2024-08-19T01:33:21.170379+0000","last_unstale":"2024-08-19T01:34:26.049223+0000","last_undegraded":"2024-08-19T01:34:26.049223+0000","last_fullsized":"2024-08-19T01:34:26.049223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:27:27.417634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048807+0000","last_change":"2024-08-19T01:33:23.189680+0000","last_active":"2024-08-19T01:34:02.048807+0000","last_peered":"2024-08-19T01:34:02.048807+0000","last_clean":"2024-08-19T01:34:02.048807+0000","last_became_active":"2024-08-19T01:33:23.189029+0000","last_became_peered":"2024-08-19T01:33:23.189029+0000","last_unstale":"2024-08-19T01:34:02.048807+0000","last_undegraded":"2024-08-19T01:34:02.048807+0000","last_fullsized":"2024-08-19T01:34:02.048807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:39.667675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881530+0000","last_change":"2024-08-19T01:33:21.169233+0000","last_active":"2024-08-19T01:34:01.881530+0000","last_peered":"2024-08-19T01:34:01.881530+0000","last_clean":"2024-08-19T01:34:01.881530+0000","last_became_active":"2024-08-19T01:33:21.168761+0000","last_became_peered":"2024-08-19T01:33:21.168761+0000","last_unstale":"2024-08-19T01:34:01.881530+0000","last_undegraded":"2024-08-19T01:34:01.881530+0000","last_fullsized":"2024-08-19T01:34:01.881530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:31:56.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883126+0000","last_change":"2024-08-19T01:33:23.194250+0000","last_active":"2024-08-19T01:34:01.883126+0000","last_peered":"2024-08-19T01:34:01.883126+0000","last_clean":"2024-08-19T01:34:01.883126+0000","last_became_active":"2024-08-19T01:33:23.193939+0000","last_became_peered":"2024-08-19T01:33:23.193939+0000","last_unstale":"2024-08-19T01:34:01.883126+0000","last_undegraded":"2024-08-19T01:34:01.883126+0000","last_fullsized":"2024-08-19T01:34:01.883126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:19:09.703701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879370+0000","last_change":"2024-08-19T01:33:21.287684+0000","last_active":"2024-08-19T01:34:01.879370+0000","last_peered":"2024-08-19T01:34:01.879370+0000","last_clean":"2024-08-19T01:34:01.879370+0000","last_became_active":"2024-08-19T01:33:21.287252+0000","last_became_peered":"2024-08-19T01:33:21.287252+0000","last_unstale":"2024-08-19T01:34:01.879370+0000","last_undegraded":"2024-08-19T01:34:01.879370+0000","last_fullsized":"2024-08-19T01:34:01.879370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:08.801345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048572+0000","last_change":"2024-08-19T01:33:23.195077+0000","last_active":"2024-08-19T01:34:02.048572+0000","last_peered":"2024-08-19T01:34:02.048572+0000","last_clean":"2024-08-19T01:34:02.048572+0000","last_became_active":"2024-08-19T01:33:23.194753+0000","last_became_peered":"2024-08-19T01:33:23.194753+0000","last_unstale":"2024-08-19T01:34:02.048572+0000","last_undegraded":"2024-08-19T01:34:02.048572+0000","last_fullsized":"2024-08-19T01:34:02.048572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:12:09.186261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'30","reported_seq":164,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.049168+0000","last_change":"2024-08-19T01:33:21.168845+0000","last_active":"2024-08-19T01:34:26.049168+0000","last_peered":"2024-08-19T01:34:26.049168+0000","last_clean":"2024-08-19T01:34:26.049168+0000","last_became_active":"2024-08-19T01:33:21.168479+0000","last_became_peered":"2024-08-19T01:33:21.168479+0000","last_unstale":"2024-08-19T01:34:26.049168+0000","last_undegraded":"2024-08-19T01:34:26.049168+0000","last_fullsized":"2024-08-19T01:34:26.049168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:59:50.296329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60616,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":114,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883678+0000","last_change":"2024-08-19T01:33:23.192252+0000","last_active":"2024-08-19T01:34:01.883678+0000","last_peered":"2024-08-19T01:34:01.883678+0000","last_clean":"2024-08-19T01:34:01.883678+0000","last_became_active":"2024-08-19T01:33:23.191934+0000","last_became_peered":"2024-08-19T01:33:23.191934+0000","last_unstale":"2024-08-19T01:34:01.883678+0000","last_undegraded":"2024-08-19T01:34:01.883678+0000","last_fullsized":"2024-08-19T01:34:01.883678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:31.134825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882306+0000","last_change":"2024-08-19T01:33:21.168541+0000","last_active":"2024-08-19T01:34:01.882306+0000","last_peered":"2024-08-19T01:34:01.882306+0000","last_clean":"2024-08-19T01:34:01.882306+0000","last_became_active":"2024-08-19T01:33:21.168205+0000","last_became_peered":"2024-08-19T01:33:21.168205+0000","last_unstale":"2024-08-19T01:34:01.882306+0000","last_undegraded":"2024-08-19T01:34:01.882306+0000","last_fullsized":"2024-08-19T01:34:01.882306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:19:52.078947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928897+0000","last_change":"2024-08-19T01:33:23.193013+0000","last_active":"2024-08-19T01:34:01.928897+0000","last_peered":"2024-08-19T01:34:01.928897+0000","last_clean":"2024-08-19T01:34:01.928897+0000","last_became_active":"2024-08-19T01:33:23.192728+0000","last_became_peered":"2024-08-19T01:33:23.192728+0000","last_unstale":"2024-08-19T01:34:01.928897+0000","last_undegraded":"2024-08-19T01:34:01.928897+0000","last_fullsized":"2024-08-19T01:34:01.928897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:23:17.587527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879101+0000","last_change":"2024-08-19T01:33:21.170118+0000","last_active":"2024-08-19T01:34:01.879101+0000","last_peered":"2024-08-19T01:34:01.879101+0000","last_clean":"2024-08-19T01:34:01.879101+0000","last_became_active":"2024-08-19T01:33:21.169757+0000","last_became_peered":"2024-08-19T01:33:21.169757+0000","last_unstale":"2024-08-19T01:34:01.879101+0000","last_undegraded":"2024-08-19T01:34:01.879101+0000","last_fullsized":"2024-08-19T01:34:01.879101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:09:26.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929944+0000","last_change":"2024-08-19T01:34:01.929944+0000","last_active":"2024-08-19T01:34:01.929944+0000","last_peered":"2024-08-19T01:34:01.929944+0000","last_clean":"2024-08-19T01:34:01.929944+0000","last_became_active":"2024-08-19T01:34:01.929679+0000","last_became_peered":"2024-08-19T01:34:01.929679+0000","last_unstale":"2024-08-19T01:34:01.929944+0000","last_undegraded":"2024-08-19T01:34:01.929944+0000","last_fullsized":"2024-08-19T01:34:01.929944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:25:24.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":81,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:23.060187+0000","last_change":"2024-08-19T01:33:21.165204+0000","last_active":"2024-08-19T01:34:23.060187+0000","last_peered":"2024-08-19T01:34:23.060187+0000","last_clean":"2024-08-19T01:34:23.060187+0000","last_became_active":"2024-08-19T01:33:21.164826+0000","last_became_peered":"2024-08-19T01:33:21.164826+0000","last_unstale":"2024-08-19T01:34:23.060187+0000","last_undegraded":"2024-08-19T01:34:23.060187+0000","last_fullsized":"2024-08-19T01:34:23.060187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:45:23.212064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880574+0000","last_change":"2024-08-19T01:33:23.198176+0000","last_active":"2024-08-19T01:34:01.880574+0000","last_peered":"2024-08-19T01:34:01.880574+0000","last_clean":"2024-08-19T01:34:01.880574+0000","last_became_active":"2024-08-19T01:33:23.196187+0000","last_became_peered":"2024-08-19T01:33:23.196187+0000","last_unstale":"2024-08-19T01:34:01.880574+0000","last_undegraded":"2024-08-19T01:34:01.880574+0000","last_fullsized":"2024-08-19T01:34:01.880574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:53:05.943467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.079412+0000","last_change":"2024-08-19T01:34:02.079028+0000","last_active":"2024-08-19T01:34:02.079412+0000","last_peered":"2024-08-19T01:34:02.079412+0000","last_clean":"2024-08-19T01:34:02.079412+0000","last_became_active":"2024-08-19T01:34:02.049890+0000","last_became_peered":"2024-08-19T01:34:02.049890+0000","last_unstale":"2024-08-19T01:34:02.079412+0000","last_undegraded":"2024-08-19T01:34:02.079412+0000","last_fullsized":"2024-08-19T01:34:02.079412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:58.595161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879105+0000","last_change":"2024-08-19T01:33:23.201104+0000","last_active":"2024-08-19T01:34:01.879105+0000","last_peered":"2024-08-19T01:34:01.879105+0000","last_clean":"2024-08-19T01:34:01.879105+0000","last_became_active":"2024-08-19T01:33:23.200809+0000","last_became_peered":"2024-08-19T01:33:23.200809+0000","last_unstale":"2024-08-19T01:34:01.879105+0000","last_undegraded":"2024-08-19T01:34:01.879105+0000","last_fullsized":"2024-08-19T01:34:01.879105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:54.901412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883975+0000","last_change":"2024-08-19T01:33:21.172423+0000","last_active":"2024-08-19T01:34:01.883975+0000","last_peered":"2024-08-19T01:34:01.883975+0000","last_clean":"2024-08-19T01:34:01.883975+0000","last_became_active":"2024-08-19T01:33:21.171950+0000","last_became_peered":"2024-08-19T01:33:21.171950+0000","last_unstale":"2024-08-19T01:34:01.883975+0000","last_undegraded":"2024-08-19T01:34:01.883975+0000","last_fullsized":"2024-08-19T01:34:01.883975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:44:11.446875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880314+0000","last_change":"2024-08-19T01:33:23.192933+0000","last_active":"2024-08-19T01:34:01.880314+0000","last_peered":"2024-08-19T01:34:01.880314+0000","last_clean":"2024-08-19T01:34:01.880314+0000","last_became_active":"2024-08-19T01:33:23.192601+0000","last_became_peered":"2024-08-19T01:33:23.192601+0000","last_unstale":"2024-08-19T01:34:01.880314+0000","last_undegraded":"2024-08-19T01:34:01.880314+0000","last_fullsized":"2024-08-19T01:34:01.880314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:09.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929233+0000","last_change":"2024-08-19T01:33:21.168225+0000","last_active":"2024-08-19T01:34:01.929233+0000","last_peered":"2024-08-19T01:34:01.929233+0000","last_clean":"2024-08-19T01:34:01.929233+0000","last_became_active":"2024-08-19T01:33:21.167881+0000","last_became_peered":"2024-08-19T01:33:21.167881+0000","last_unstale":"2024-08-19T01:34:01.929233+0000","last_undegraded":"2024-08-19T01:34:01.929233+0000","last_fullsized":"2024-08-19T01:34:01.929233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:53:38.037036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881493+0000","last_change":"2024-08-19T01:33:23.194079+0000","last_active":"2024-08-19T01:34:01.881493+0000","last_peered":"2024-08-19T01:34:01.881493+0000","last_clean":"2024-08-19T01:34:01.881493+0000","last_became_active":"2024-08-19T01:33:23.193749+0000","last_became_peered":"2024-08-19T01:33:23.193749+0000","last_unstale":"2024-08-19T01:34:01.881493+0000","last_undegraded":"2024-08-19T01:34:01.881493+0000","last_fullsized":"2024-08-19T01:34:01.881493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:53:41.102163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879491+0000","last_change":"2024-08-19T01:33:21.168916+0000","last_active":"2024-08-19T01:34:01.879491+0000","last_peered":"2024-08-19T01:34:01.879491+0000","last_clean":"2024-08-19T01:34:01.879491+0000","last_became_active":"2024-08-19T01:33:21.168618+0000","last_became_peered":"2024-08-19T01:33:21.168618+0000","last_unstale":"2024-08-19T01:34:01.879491+0000","last_undegraded":"2024-08-19T01:34:01.879491+0000","last_fullsized":"2024-08-19T01:34:01.879491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:45:11.567779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881292+0000","last_change":"2024-08-19T01:33:23.195546+0000","last_active":"2024-08-19T01:34:01.881292+0000","last_peered":"2024-08-19T01:34:01.881292+0000","last_clean":"2024-08-19T01:34:01.881292+0000","last_became_active":"2024-08-19T01:33:23.195221+0000","last_became_peered":"2024-08-19T01:33:23.195221+0000","last_unstale":"2024-08-19T01:34:01.881292+0000","last_undegraded":"2024-08-19T01:34:01.881292+0000","last_fullsized":"2024-08-19T01:34:01.881292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:46:10.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:23.045383+0000","last_change":"2024-08-19T01:33:21.287946+0000","last_active":"2024-08-19T01:34:23.045383+0000","last_peered":"2024-08-19T01:34:23.045383+0000","last_clean":"2024-08-19T01:34:23.045383+0000","last_became_active":"2024-08-19T01:33:21.287594+0000","last_became_peered":"2024-08-19T01:33:21.287594+0000","last_unstale":"2024-08-19T01:34:23.045383+0000","last_undegraded":"2024-08-19T01:34:23.045383+0000","last_fullsized":"2024-08-19T01:34:23.045383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:10:06.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881482+0000","last_change":"2024-08-19T01:33:23.200914+0000","last_active":"2024-08-19T01:34:01.881482+0000","last_peered":"2024-08-19T01:34:01.881482+0000","last_clean":"2024-08-19T01:34:01.881482+0000","last_became_active":"2024-08-19T01:33:23.200611+0000","last_became_peered":"2024-08-19T01:33:23.200611+0000","last_unstale":"2024-08-19T01:34:01.881482+0000","last_undegraded":"2024-08-19T01:34:01.881482+0000","last_fullsized":"2024-08-19T01:34:01.881482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:22:00.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880857+0000","last_change":"2024-08-19T01:32:09.926979+0000","last_active":"2024-08-19T01:34:01.880857+0000","last_peered":"2024-08-19T01:34:01.880857+0000","last_clean":"2024-08-19T01:34:01.880857+0000","last_became_active":"2024-08-19T01:32:09.926560+0000","last_became_peered":"2024-08-19T01:32:09.926560+0000","last_unstale":"2024-08-19T01:34:01.880857+0000","last_undegraded":"2024-08-19T01:34:01.880857+0000","last_fullsized":"2024-08-19T01:34:01.880857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_clean_scrub_stamp":"2024-08-19T01:29:05.356703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:12:23.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878743+0000","last_change":"2024-08-19T01:33:21.165099+0000","last_active":"2024-08-19T01:34:01.878743+0000","last_peered":"2024-08-19T01:34:01.878743+0000","last_clean":"2024-08-19T01:34:01.878743+0000","last_became_active":"2024-08-19T01:33:21.164756+0000","last_became_peered":"2024-08-19T01:33:21.164756+0000","last_unstale":"2024-08-19T01:34:01.878743+0000","last_undegraded":"2024-08-19T01:34:01.878743+0000","last_fullsized":"2024-08-19T01:34:01.878743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:11:54.480263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877931+0000","last_change":"2024-08-19T01:33:23.200185+0000","last_active":"2024-08-19T01:34:01.877931+0000","last_peered":"2024-08-19T01:34:01.877931+0000","last_clean":"2024-08-19T01:34:01.877931+0000","last_became_active":"2024-08-19T01:33:23.199885+0000","last_became_peered":"2024-08-19T01:33:23.199885+0000","last_unstale":"2024-08-19T01:34:01.877931+0000","last_undegraded":"2024-08-19T01:34:01.877931+0000","last_fullsized":"2024-08-19T01:34:01.877931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:17.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880754+0000","last_change":"2024-08-19T01:33:21.172587+0000","last_active":"2024-08-19T01:34:01.880754+0000","last_peered":"2024-08-19T01:34:01.880754+0000","last_clean":"2024-08-19T01:34:01.880754+0000","last_became_active":"2024-08-19T01:33:21.172233+0000","last_became_peered":"2024-08-19T01:33:21.172233+0000","last_unstale":"2024-08-19T01:34:01.880754+0000","last_undegraded":"2024-08-19T01:34:01.880754+0000","last_fullsized":"2024-08-19T01:34:01.880754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:26.690403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928855+0000","last_change":"2024-08-19T01:33:23.197272+0000","last_active":"2024-08-19T01:34:01.928855+0000","last_peered":"2024-08-19T01:34:01.928855+0000","last_clean":"2024-08-19T01:34:01.928855+0000","last_became_active":"2024-08-19T01:33:23.196869+0000","last_became_peered":"2024-08-19T01:33:23.196869+0000","last_unstale":"2024-08-19T01:34:01.928855+0000","last_undegraded":"2024-08-19T01:34:01.928855+0000","last_fullsized":"2024-08-19T01:34:01.928855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:30.020184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:24.060832+0000","last_change":"2024-08-19T01:33:21.287366+0000","last_active":"2024-08-19T01:34:24.060832+0000","last_peered":"2024-08-19T01:34:24.060832+0000","last_clean":"2024-08-19T01:34:24.060832+0000","last_became_active":"2024-08-19T01:33:21.287044+0000","last_became_peered":"2024-08-19T01:33:21.287044+0000","last_unstale":"2024-08-19T01:34:24.060832+0000","last_undegraded":"2024-08-19T01:34:24.060832+0000","last_fullsized":"2024-08-19T01:34:24.060832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:19:04.194284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.885371+0000","last_change":"2024-08-19T01:34:01.885001+0000","last_active":"2024-08-19T01:34:01.885371+0000","last_peered":"2024-08-19T01:34:01.885371+0000","last_clean":"2024-08-19T01:34:01.885371+0000","last_became_active":"2024-08-19T01:34:01.884680+0000","last_became_peered":"2024-08-19T01:34:01.884680+0000","last_unstale":"2024-08-19T01:34:01.885371+0000","last_undegraded":"2024-08-19T01:34:01.885371+0000","last_fullsized":"2024-08-19T01:34:01.885371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:08.251298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":83,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.063456+0000","last_change":"2024-08-19T01:33:21.164639+0000","last_active":"2024-08-19T01:34:26.063456+0000","last_peered":"2024-08-19T01:34:26.063456+0000","last_clean":"2024-08-19T01:34:26.063456+0000","last_became_active":"2024-08-19T01:33:21.164330+0000","last_became_peered":"2024-08-19T01:33:21.164330+0000","last_unstale":"2024-08-19T01:34:26.063456+0000","last_undegraded":"2024-08-19T01:34:26.063456+0000","last_fullsized":"2024-08-19T01:34:26.063456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:26:04.035130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883666+0000","last_change":"2024-08-19T01:34:01.883666+0000","last_active":"2024-08-19T01:34:01.883666+0000","last_peered":"2024-08-19T01:34:01.883666+0000","last_clean":"2024-08-19T01:34:01.883666+0000","last_became_active":"2024-08-19T01:34:01.883264+0000","last_became_peered":"2024-08-19T01:34:01.883264+0000","last_unstale":"2024-08-19T01:34:01.883666+0000","last_undegraded":"2024-08-19T01:34:01.883666+0000","last_fullsized":"2024-08-19T01:34:01.883666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:41:52.266751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879931+0000","last_change":"2024-08-19T01:33:21.287906+0000","last_active":"2024-08-19T01:34:01.879931+0000","last_peered":"2024-08-19T01:34:01.879931+0000","last_clean":"2024-08-19T01:34:01.879931+0000","last_became_active":"2024-08-19T01:33:21.287597+0000","last_became_peered":"2024-08-19T01:33:21.287597+0000","last_unstale":"2024-08-19T01:34:01.879931+0000","last_undegraded":"2024-08-19T01:34:01.879931+0000","last_fullsized":"2024-08-19T01:34:01.879931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:30.605948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880677+0000","last_change":"2024-08-19T01:33:23.191026+0000","last_active":"2024-08-19T01:34:01.880677+0000","last_peered":"2024-08-19T01:34:01.880677+0000","last_clean":"2024-08-19T01:34:01.880677+0000","last_became_active":"2024-08-19T01:33:23.190382+0000","last_became_peered":"2024-08-19T01:33:23.190382+0000","last_unstale":"2024-08-19T01:34:01.880677+0000","last_undegraded":"2024-08-19T01:34:01.880677+0000","last_fullsized":"2024-08-19T01:34:01.880677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:33:45.065441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883898+0000","last_change":"2024-08-19T01:33:21.288008+0000","last_active":"2024-08-19T01:34:01.883898+0000","last_peered":"2024-08-19T01:34:01.883898+0000","last_clean":"2024-08-19T01:34:01.883898+0000","last_became_active":"2024-08-19T01:33:21.287613+0000","last_became_peered":"2024-08-19T01:33:21.287613+0000","last_unstale":"2024-08-19T01:34:01.883898+0000","last_undegraded":"2024-08-19T01:34:01.883898+0000","last_fullsized":"2024-08-19T01:34:01.883898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:35:18.169564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929091+0000","last_change":"2024-08-19T01:33:23.197201+0000","last_active":"2024-08-19T01:34:01.929091+0000","last_peered":"2024-08-19T01:34:01.929091+0000","last_clean":"2024-08-19T01:34:01.929091+0000","last_became_active":"2024-08-19T01:33:23.196946+0000","last_became_peered":"2024-08-19T01:33:23.196946+0000","last_unstale":"2024-08-19T01:34:01.929091+0000","last_undegraded":"2024-08-19T01:34:01.929091+0000","last_fullsized":"2024-08-19T01:34:01.929091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:14:31.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880120+0000","last_change":"2024-08-19T01:33:21.167109+0000","last_active":"2024-08-19T01:34:01.880120+0000","last_peered":"2024-08-19T01:34:01.880120+0000","last_clean":"2024-08-19T01:34:01.880120+0000","last_became_active":"2024-08-19T01:33:21.166788+0000","last_became_peered":"2024-08-19T01:33:21.166788+0000","last_unstale":"2024-08-19T01:34:01.880120+0000","last_undegraded":"2024-08-19T01:34:01.880120+0000","last_fullsized":"2024-08-19T01:34:01.880120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:44.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878126+0000","last_change":"2024-08-19T01:33:23.199699+0000","last_active":"2024-08-19T01:34:01.878126+0000","last_peered":"2024-08-19T01:34:01.878126+0000","last_clean":"2024-08-19T01:34:01.878126+0000","last_became_active":"2024-08-19T01:33:23.199417+0000","last_became_peered":"2024-08-19T01:33:23.199417+0000","last_unstale":"2024-08-19T01:34:01.878126+0000","last_undegraded":"2024-08-19T01:34:01.878126+0000","last_fullsized":"2024-08-19T01:34:01.878126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:36.519135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878888+0000","last_change":"2024-08-19T01:33:21.164177+0000","last_active":"2024-08-19T01:34:01.878888+0000","last_peered":"2024-08-19T01:34:01.878888+0000","last_clean":"2024-08-19T01:34:01.878888+0000","last_became_active":"2024-08-19T01:33:21.163832+0000","last_became_peered":"2024-08-19T01:33:21.163832+0000","last_unstale":"2024-08-19T01:34:01.878888+0000","last_undegraded":"2024-08-19T01:34:01.878888+0000","last_fullsized":"2024-08-19T01:34:01.878888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:31.907677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882973+0000","last_change":"2024-08-19T01:33:23.195102+0000","last_active":"2024-08-19T01:34:01.882973+0000","last_peered":"2024-08-19T01:34:01.882973+0000","last_clean":"2024-08-19T01:34:01.882973+0000","last_became_active":"2024-08-19T01:33:23.194802+0000","last_became_peered":"2024-08-19T01:33:23.194802+0000","last_unstale":"2024-08-19T01:34:01.882973+0000","last_undegraded":"2024-08-19T01:34:01.882973+0000","last_fullsized":"2024-08-19T01:34:01.882973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:33.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048628+0000","last_change":"2024-08-19T01:33:21.287933+0000","last_active":"2024-08-19T01:34:02.048628+0000","last_peered":"2024-08-19T01:34:02.048628+0000","last_clean":"2024-08-19T01:34:02.048628+0000","last_became_active":"2024-08-19T01:33:21.287568+0000","last_became_peered":"2024-08-19T01:33:21.287568+0000","last_unstale":"2024-08-19T01:34:02.048628+0000","last_undegraded":"2024-08-19T01:34:02.048628+0000","last_fullsized":"2024-08-19T01:34:02.048628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:23.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883283+0000","last_change":"2024-08-19T01:33:23.194320+0000","last_active":"2024-08-19T01:34:01.883283+0000","last_peered":"2024-08-19T01:34:01.883283+0000","last_clean":"2024-08-19T01:34:01.883283+0000","last_became_active":"2024-08-19T01:33:23.194002+0000","last_became_peered":"2024-08-19T01:33:23.194002+0000","last_unstale":"2024-08-19T01:34:01.883283+0000","last_undegraded":"2024-08-19T01:34:01.883283+0000","last_fullsized":"2024-08-19T01:34:01.883283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:59.198997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877938+0000","last_change":"2024-08-19T01:33:21.165421+0000","last_active":"2024-08-19T01:34:01.877938+0000","last_peered":"2024-08-19T01:34:01.877938+0000","last_clean":"2024-08-19T01:34:01.877938+0000","last_became_active":"2024-08-19T01:33:21.165013+0000","last_became_peered":"2024-08-19T01:33:21.165013+0000","last_unstale":"2024-08-19T01:34:01.877938+0000","last_undegraded":"2024-08-19T01:34:01.877938+0000","last_fullsized":"2024-08-19T01:34:01.877938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:07:18.546099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880098+0000","last_change":"2024-08-19T01:33:23.193544+0000","last_active":"2024-08-19T01:34:01.880098+0000","last_peered":"2024-08-19T01:34:01.880098+0000","last_clean":"2024-08-19T01:34:01.880098+0000","last_became_active":"2024-08-19T01:33:23.193213+0000","last_became_peered":"2024-08-19T01:33:23.193213+0000","last_unstale":"2024-08-19T01:34:01.880098+0000","last_undegraded":"2024-08-19T01:34:01.880098+0000","last_fullsized":"2024-08-19T01:34:01.880098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:20:54.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.890071+0000","last_change":"2024-08-19T01:34:01.889627+0000","last_active":"2024-08-19T01:34:01.890071+0000","last_peered":"2024-08-19T01:34:01.890071+0000","last_clean":"2024-08-19T01:34:01.890071+0000","last_became_active":"2024-08-19T01:34:01.885181+0000","last_became_peered":"2024-08-19T01:34:01.885181+0000","last_unstale":"2024-08-19T01:34:01.890071+0000","last_undegraded":"2024-08-19T01:34:01.890071+0000","last_fullsized":"2024-08-19T01:34:01.890071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:51.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878113+0000","last_change":"2024-08-19T01:33:23.196266+0000","last_active":"2024-08-19T01:34:01.878113+0000","last_peered":"2024-08-19T01:34:01.878113+0000","last_clean":"2024-08-19T01:34:01.878113+0000","last_became_active":"2024-08-19T01:33:23.194268+0000","last_became_peered":"2024-08-19T01:33:23.194268+0000","last_unstale":"2024-08-19T01:34:01.878113+0000","last_undegraded":"2024-08-19T01:34:01.878113+0000","last_fullsized":"2024-08-19T01:34:01.878113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:15:38.628746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.890210+0000","last_change":"2024-08-19T01:34:01.890210+0000","last_active":"2024-08-19T01:34:01.890210+0000","last_peered":"2024-08-19T01:34:01.890210+0000","last_clean":"2024-08-19T01:34:01.890210+0000","last_became_active":"2024-08-19T01:34:01.885955+0000","last_became_peered":"2024-08-19T01:34:01.885955+0000","last_unstale":"2024-08-19T01:34:01.890210+0000","last_undegraded":"2024-08-19T01:34:01.890210+0000","last_fullsized":"2024-08-19T01:34:01.890210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.176530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881137+0000","last_change":"2024-08-19T01:33:23.202534+0000","last_active":"2024-08-19T01:34:01.881137+0000","last_peered":"2024-08-19T01:34:01.881137+0000","last_clean":"2024-08-19T01:34:01.881137+0000","last_became_active":"2024-08-19T01:33:23.202191+0000","last_became_peered":"2024-08-19T01:33:23.202191+0000","last_unstale":"2024-08-19T01:34:01.881137+0000","last_undegraded":"2024-08-19T01:34:01.881137+0000","last_fullsized":"2024-08-19T01:34:01.881137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:03.478905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879469+0000","last_change":"2024-08-19T01:33:23.201136+0000","last_active":"2024-08-19T01:34:01.879469+0000","last_peered":"2024-08-19T01:34:01.879469+0000","last_clean":"2024-08-19T01:34:01.879469+0000","last_became_active":"2024-08-19T01:33:23.200864+0000","last_became_peered":"2024-08-19T01:33:23.200864+0000","last_unstale":"2024-08-19T01:34:01.879469+0000","last_undegraded":"2024-08-19T01:34:01.879469+0000","last_fullsized":"2024-08-19T01:34:01.879469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:32:42.091880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883696+0000","last_change":"2024-08-19T01:33:21.172149+0000","last_active":"2024-08-19T01:34:01.883696+0000","last_peered":"2024-08-19T01:34:01.883696+0000","last_clean":"2024-08-19T01:34:01.883696+0000","last_became_active":"2024-08-19T01:33:21.171833+0000","last_became_peered":"2024-08-19T01:33:21.171833+0000","last_unstale":"2024-08-19T01:34:01.883696+0000","last_undegraded":"2024-08-19T01:34:01.883696+0000","last_fullsized":"2024-08-19T01:34:01.883696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:25:11.097212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048466+0000","last_change":"2024-08-19T01:33:23.193675+0000","last_active":"2024-08-19T01:34:02.048466+0000","last_peered":"2024-08-19T01:34:02.048466+0000","last_clean":"2024-08-19T01:34:02.048466+0000","last_became_active":"2024-08-19T01:33:23.188719+0000","last_became_peered":"2024-08-19T01:33:23.188719+0000","last_unstale":"2024-08-19T01:34:02.048466+0000","last_undegraded":"2024-08-19T01:34:02.048466+0000","last_fullsized":"2024-08-19T01:34:02.048466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:54:12.698211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883781+0000","last_change":"2024-08-19T01:33:21.172675+0000","last_active":"2024-08-19T01:34:01.883781+0000","last_peered":"2024-08-19T01:34:01.883781+0000","last_clean":"2024-08-19T01:34:01.883781+0000","last_became_active":"2024-08-19T01:33:21.172320+0000","last_became_peered":"2024-08-19T01:33:21.172320+0000","last_unstale":"2024-08-19T01:34:01.883781+0000","last_undegraded":"2024-08-19T01:34:01.883781+0000","last_fullsized":"2024-08-19T01:34:01.883781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:29:41.511220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883123+0000","last_change":"2024-08-19T01:33:23.203113+0000","last_active":"2024-08-19T01:34:01.883123+0000","last_peered":"2024-08-19T01:34:01.883123+0000","last_clean":"2024-08-19T01:34:01.883123+0000","last_became_active":"2024-08-19T01:33:23.202807+0000","last_became_peered":"2024-08-19T01:33:23.202807+0000","last_unstale":"2024-08-19T01:34:01.883123+0000","last_undegraded":"2024-08-19T01:34:01.883123+0000","last_fullsized":"2024-08-19T01:34:01.883123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:24:21.570929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878197+0000","last_change":"2024-08-19T01:33:21.287382+0000","last_active":"2024-08-19T01:34:01.878197+0000","last_peered":"2024-08-19T01:34:01.878197+0000","last_clean":"2024-08-19T01:34:01.878197+0000","last_became_active":"2024-08-19T01:33:21.287040+0000","last_became_peered":"2024-08-19T01:33:21.287040+0000","last_unstale":"2024-08-19T01:34:01.878197+0000","last_undegraded":"2024-08-19T01:34:01.878197+0000","last_fullsized":"2024-08-19T01:34:01.878197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:01:49.973969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878016+0000","last_change":"2024-08-19T01:33:23.197869+0000","last_active":"2024-08-19T01:34:01.878016+0000","last_peered":"2024-08-19T01:34:01.878016+0000","last_clean":"2024-08-19T01:34:01.878016+0000","last_became_active":"2024-08-19T01:33:23.197529+0000","last_became_peered":"2024-08-19T01:33:23.197529+0000","last_unstale":"2024-08-19T01:34:01.878016+0000","last_undegraded":"2024-08-19T01:34:01.878016+0000","last_fullsized":"2024-08-19T01:34:01.878016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:14.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":86,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:23.059287+0000","last_change":"2024-08-19T01:33:21.172180+0000","last_active":"2024-08-19T01:34:23.059287+0000","last_peered":"2024-08-19T01:34:23.059287+0000","last_clean":"2024-08-19T01:34:23.059287+0000","last_became_active":"2024-08-19T01:33:21.171895+0000","last_became_peered":"2024-08-19T01:33:21.171895+0000","last_unstale":"2024-08-19T01:34:23.059287+0000","last_undegraded":"2024-08-19T01:34:23.059287+0000","last_fullsized":"2024-08-19T01:34:23.059287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:51:38.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68208,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":555,"num_read_kb":351,"num_write":244,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":204624,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":454083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":1106006,"data_compressed":37649,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":453901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":454167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":1047576,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":454008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1046474,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":453869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":453443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":515645,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":515649,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":454957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62870,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62866,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62742,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:34:30.984 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-19T01:34:31.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:30 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:34:31.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:30 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:34:31.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:30 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:31.236 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:31.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:30 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:34:31.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:30 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:34:31.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:30 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:30 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:34:31.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:30 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:34:31.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:30 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:34:31.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.751+0000 7f83d2536640 1 -- 172.21.15.103:0/3433095296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc100b50 msgr2=0x7f83cc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.751+0000 7f83d2536640 1 --2- 172.21.15.103:0/3433095296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc100b50 0x7f83cc100f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009900 tx=0x7f83bc02a480 comp rx=0 tx=0).stop 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.751+0000 7f83d2536640 1 -- 172.21.15.103:0/3433095296 shutdown_connections 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.751+0000 7f83d2536640 1 --2- 172.21.15.103:0/3433095296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83cc102c20 0x7f83cc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.751+0000 7f83d2536640 1 --2- 172.21.15.103:0/3433095296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83cc1022a0 0x7f83cc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.751+0000 7f83d2536640 1 --2- 172.21.15.103:0/3433095296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc100b50 0x7f83cc100f30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.751+0000 7f83d2536640 1 -- 172.21.15.103:0/3433095296 >> 172.21.15.103:0/3433095296 conn(0x7f83cc0fc500 msgr2=0x7f83cc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.752+0000 7f83d2536640 1 -- 172.21.15.103:0/3433095296 shutdown_connections 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.752+0000 7f83d2536640 1 -- 172.21.15.103:0/3433095296 wait complete. 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.752+0000 7f83d2536640 1 Processor -- start 2024-08-19T01:34:31.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.752+0000 7f83d2536640 1 -- start start 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83cc100b50 0x7f83cc198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83cc1022a0 0x7f83cc199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc102c20 0x7f83cc1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc110b20 con 0x7f83cc102c20 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1109a0 con 0x7f83cc100b50 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc110ca0 con 0x7f83cc1022a0 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc102c20 0x7f83cc1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc102c20 0x7f83cc1a3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45836/0 (socket says 172.21.15.103:45836) 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 -- 172.21.15.103:0/2562096385 learned_addr learned my addr 172.21.15.103:0/2562096385 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83cbfff640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83cc100b50 0x7f83cc198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83cb7fe640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83cc1022a0 0x7f83cc199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 -- 172.21.15.103:0/2562096385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83cc1022a0 msgr2=0x7f83cc199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83cc1022a0 0x7f83cc199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 -- 172.21.15.103:0/2562096385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83cc100b50 msgr2=0x7f83cc198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83cc100b50 0x7f83cc198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 -- 172.21.15.103:0/2562096385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc1a4510 con 0x7f83cc102c20 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83cbfff640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83cc100b50 0x7f83cc198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83cb7fe640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83cc1022a0 0x7f83cc199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d0ad3640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc102c20 0x7f83cc1a3fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f83c400cad0 tx=0x7f83c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83c97fa640 1 -- 172.21.15.103:0/2562096385 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c400ae20 con 0x7f83cc102c20 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc1a46e0 con 0x7f83cc102c20 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83c97fa640 1 -- 172.21.15.103:0/2562096385 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c4005020 con 0x7f83cc102c20 2024-08-19T01:34:31.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83c97fa640 1 -- 172.21.15.103:0/2562096385 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c400f6b0 con 0x7f83cc102c20 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.753+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1a4bf0 con 0x7f83cc102c20 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.755+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc100f30 con 0x7f83cc102c20 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.756+0000 7f83c97fa640 1 -- 172.21.15.103:0/2562096385 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f83c40051d0 con 0x7f83cc102c20 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.756+0000 7f83c97fa640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f83b80a2c10 0x7f83b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.756+0000 7f83cbfff640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f83b80a2c10 0x7f83b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.757+0000 7f83c97fa640 1 -- 172.21.15.103:0/2562096385 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f83c40cf270 con 0x7f83cc102c20 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.757+0000 7f83cbfff640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f83b80a2c10 0x7f83b80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83bc000c00 tx=0x7f83bc032d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:31.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.760+0000 7f83c97fa640 1 -- 172.21.15.103:0/2562096385 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f83c4014030 con 0x7f83cc102c20 2024-08-19T01:34:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.925+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83cc062360 con 0x7f83b80a2c10 2024-08-19T01:34:31.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:31 smithi103 ceph-mon[18436]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:31.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:31 smithi103 ceph-mon[18436]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:31.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.932+0000 7f83c97fa640 1 -- 172.21.15.103:0/2562096385 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258988 (secure 0 0 0) 0x7f83cc062360 con 0x7f83b80a2c10 2024-08-19T01:34:31.936 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:31.937 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T01:34:31.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f83b80a2c10 msgr2=0x7f83b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f83b80a2c10 0x7f83b80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83bc000c00 tx=0x7f83bc032d50 comp rx=0 tx=0).stop 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc102c20 msgr2=0x7f83cc1a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc102c20 0x7f83cc1a3fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f83c400cad0 tx=0x7f83c400cf90 comp rx=0 tx=0).stop 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 shutdown_connections 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f83b80a2c10 0x7f83b80a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83cc102c20 0x7f83cc1a3fd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83cc1022a0 0x7f83cc199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 --2- 172.21.15.103:0/2562096385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f83cc100b50 0x7f83cc198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 >> 172.21.15.103:0/2562096385 conn(0x7f83cc0fc500 msgr2=0x7f83cc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 shutdown_connections 2024-08-19T01:34:31.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:31.935+0000 7f83d2536640 1 -- 172.21.15.103:0/2562096385 wait complete. 2024-08-19T01:34:32.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:31 smithi146 ceph-mon[22098]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:32.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:31 smithi146 ceph-mon[22098]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:32.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:31 smithi190 ceph-mon[22032]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:32.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:31 smithi190 ceph-mon[22032]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:32.696 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-08-19T01:34:30.787350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658810,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":696,"num_read_kb":465,"num_write":427,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364928,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576896,"statfs":{"total":1151940427776,"available":1151566741504,"internally_reserved":0,"allocated":11833344,"data_stored":7409778,"data_compressed":112947,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003272"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880448+0000","last_change":"2024-08-19T01:33:23.202035+0000","last_active":"2024-08-19T01:34:01.880448+0000","last_peered":"2024-08-19T01:34:01.880448+0000","last_clean":"2024-08-19T01:34:01.880448+0000","last_became_active":"2024-08-19T01:33:23.201418+0000","last_became_peered":"2024-08-19T01:33:23.201418+0000","last_unstale":"2024-08-19T01:34:01.880448+0000","last_undegraded":"2024-08-19T01:34:01.880448+0000","last_fullsized":"2024-08-19T01:34:01.880448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:54:57.927424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882924+0000","last_change":"2024-08-19T01:33:23.202047+0000","last_active":"2024-08-19T01:34:01.882924+0000","last_peered":"2024-08-19T01:34:01.882924+0000","last_clean":"2024-08-19T01:34:01.882924+0000","last_became_active":"2024-08-19T01:33:23.201762+0000","last_became_peered":"2024-08-19T01:33:23.201762+0000","last_unstale":"2024-08-19T01:34:01.882924+0000","last_undegraded":"2024-08-19T01:34:01.882924+0000","last_fullsized":"2024-08-19T01:34:01.882924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:23:57.252284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882773+0000","last_change":"2024-08-19T01:33:23.202876+0000","last_active":"2024-08-19T01:34:01.882773+0000","last_peered":"2024-08-19T01:34:01.882773+0000","last_clean":"2024-08-19T01:34:01.882773+0000","last_became_active":"2024-08-19T01:33:23.202528+0000","last_became_peered":"2024-08-19T01:33:23.202528+0000","last_unstale":"2024-08-19T01:34:01.882773+0000","last_undegraded":"2024-08-19T01:34:01.882773+0000","last_fullsized":"2024-08-19T01:34:01.882773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:20.514390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878534+0000","last_change":"2024-08-19T01:33:23.199443+0000","last_active":"2024-08-19T01:34:01.878534+0000","last_peered":"2024-08-19T01:34:01.878534+0000","last_clean":"2024-08-19T01:34:01.878534+0000","last_became_active":"2024-08-19T01:33:23.198344+0000","last_became_peered":"2024-08-19T01:33:23.198344+0000","last_unstale":"2024-08-19T01:34:01.878534+0000","last_undegraded":"2024-08-19T01:34:01.878534+0000","last_fullsized":"2024-08-19T01:34:01.878534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:43:43.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883556+0000","last_change":"2024-08-19T01:33:23.191185+0000","last_active":"2024-08-19T01:34:01.883556+0000","last_peered":"2024-08-19T01:34:01.883556+0000","last_clean":"2024-08-19T01:34:01.883556+0000","last_became_active":"2024-08-19T01:33:23.190892+0000","last_became_peered":"2024-08-19T01:33:23.190892+0000","last_unstale":"2024-08-19T01:34:01.883556+0000","last_undegraded":"2024-08-19T01:34:01.883556+0000","last_fullsized":"2024-08-19T01:34:01.883556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:28.384609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880419+0000","last_change":"2024-08-19T01:33:23.193036+0000","last_active":"2024-08-19T01:34:01.880419+0000","last_peered":"2024-08-19T01:34:01.880419+0000","last_clean":"2024-08-19T01:34:01.880419+0000","last_became_active":"2024-08-19T01:33:23.192686+0000","last_became_peered":"2024-08-19T01:33:23.192686+0000","last_unstale":"2024-08-19T01:34:01.880419+0000","last_undegraded":"2024-08-19T01:34:01.880419+0000","last_fullsized":"2024-08-19T01:34:01.880419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:46.235112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880846+0000","last_change":"2024-08-19T01:33:23.197636+0000","last_active":"2024-08-19T01:34:01.880846+0000","last_peered":"2024-08-19T01:34:01.880846+0000","last_clean":"2024-08-19T01:34:01.880846+0000","last_became_active":"2024-08-19T01:33:23.197323+0000","last_became_peered":"2024-08-19T01:33:23.197323+0000","last_unstale":"2024-08-19T01:34:01.880846+0000","last_undegraded":"2024-08-19T01:34:01.880846+0000","last_fullsized":"2024-08-19T01:34:01.880846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:46:36.168786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877768+0000","last_change":"2024-08-19T01:33:23.193717+0000","last_active":"2024-08-19T01:34:01.877768+0000","last_peered":"2024-08-19T01:34:01.877768+0000","last_clean":"2024-08-19T01:34:01.877768+0000","last_became_active":"2024-08-19T01:33:23.193376+0000","last_became_peered":"2024-08-19T01:33:23.193376+0000","last_unstale":"2024-08-19T01:34:01.877768+0000","last_undegraded":"2024-08-19T01:34:01.877768+0000","last_fullsized":"2024-08-19T01:34:01.877768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:16.290081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048946+0000","last_change":"2024-08-19T01:33:23.197956+0000","last_active":"2024-08-19T01:34:02.048946+0000","last_peered":"2024-08-19T01:34:02.048946+0000","last_clean":"2024-08-19T01:34:02.048946+0000","last_became_active":"2024-08-19T01:33:23.197667+0000","last_became_peered":"2024-08-19T01:33:23.197667+0000","last_unstale":"2024-08-19T01:34:02.048946+0000","last_undegraded":"2024-08-19T01:34:02.048946+0000","last_fullsized":"2024-08-19T01:34:02.048946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:16.841046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881099+0000","last_change":"2024-08-19T01:33:23.194569+0000","last_active":"2024-08-19T01:34:01.881099+0000","last_peered":"2024-08-19T01:34:01.881099+0000","last_clean":"2024-08-19T01:34:01.881099+0000","last_became_active":"2024-08-19T01:33:23.194223+0000","last_became_peered":"2024-08-19T01:33:23.194223+0000","last_unstale":"2024-08-19T01:34:01.881099+0000","last_undegraded":"2024-08-19T01:34:01.881099+0000","last_fullsized":"2024-08-19T01:34:01.881099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:14:30.184738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881455+0000","last_change":"2024-08-19T01:33:23.202396+0000","last_active":"2024-08-19T01:34:01.881455+0000","last_peered":"2024-08-19T01:34:01.881455+0000","last_clean":"2024-08-19T01:34:01.881455+0000","last_became_active":"2024-08-19T01:33:23.202063+0000","last_became_peered":"2024-08-19T01:33:23.202063+0000","last_unstale":"2024-08-19T01:34:01.881455+0000","last_undegraded":"2024-08-19T01:34:01.881455+0000","last_fullsized":"2024-08-19T01:34:01.881455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:57.513639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878563+0000","last_change":"2024-08-19T01:33:23.200610+0000","last_active":"2024-08-19T01:34:01.878563+0000","last_peered":"2024-08-19T01:34:01.878563+0000","last_clean":"2024-08-19T01:34:01.878563+0000","last_became_active":"2024-08-19T01:33:23.200136+0000","last_became_peered":"2024-08-19T01:33:23.200136+0000","last_unstale":"2024-08-19T01:34:01.878563+0000","last_undegraded":"2024-08-19T01:34:01.878563+0000","last_fullsized":"2024-08-19T01:34:01.878563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:47:39.252702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929133+0000","last_change":"2024-08-19T01:33:23.196062+0000","last_active":"2024-08-19T01:34:01.929133+0000","last_peered":"2024-08-19T01:34:01.929133+0000","last_clean":"2024-08-19T01:34:01.929133+0000","last_became_active":"2024-08-19T01:33:23.195577+0000","last_became_peered":"2024-08-19T01:33:23.195577+0000","last_unstale":"2024-08-19T01:34:01.929133+0000","last_undegraded":"2024-08-19T01:34:01.929133+0000","last_fullsized":"2024-08-19T01:34:01.929133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:46:09.111323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882617+0000","last_change":"2024-08-19T01:33:23.201664+0000","last_active":"2024-08-19T01:34:01.882617+0000","last_peered":"2024-08-19T01:34:01.882617+0000","last_clean":"2024-08-19T01:34:01.882617+0000","last_became_active":"2024-08-19T01:33:23.200241+0000","last_became_peered":"2024-08-19T01:33:23.200241+0000","last_unstale":"2024-08-19T01:34:01.882617+0000","last_undegraded":"2024-08-19T01:34:01.882617+0000","last_fullsized":"2024-08-19T01:34:01.882617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:17.199776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882446+0000","last_change":"2024-08-19T01:33:23.198102+0000","last_active":"2024-08-19T01:34:01.882446+0000","last_peered":"2024-08-19T01:34:01.882446+0000","last_clean":"2024-08-19T01:34:01.882446+0000","last_became_active":"2024-08-19T01:33:23.197778+0000","last_became_peered":"2024-08-19T01:33:23.197778+0000","last_unstale":"2024-08-19T01:34:01.882446+0000","last_undegraded":"2024-08-19T01:34:01.882446+0000","last_fullsized":"2024-08-19T01:34:01.882446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:03.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048327+0000","last_change":"2024-08-19T01:33:23.194581+0000","last_active":"2024-08-19T01:34:02.048327+0000","last_peered":"2024-08-19T01:34:02.048327+0000","last_clean":"2024-08-19T01:34:02.048327+0000","last_became_active":"2024-08-19T01:33:23.194256+0000","last_became_peered":"2024-08-19T01:33:23.194256+0000","last_unstale":"2024-08-19T01:34:02.048327+0000","last_undegraded":"2024-08-19T01:34:02.048327+0000","last_fullsized":"2024-08-19T01:34:02.048327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:17:20.954190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880474+0000","last_change":"2024-08-19T01:33:23.201242+0000","last_active":"2024-08-19T01:34:01.880474+0000","last_peered":"2024-08-19T01:34:01.880474+0000","last_clean":"2024-08-19T01:34:01.880474+0000","last_became_active":"2024-08-19T01:33:23.200835+0000","last_became_peered":"2024-08-19T01:33:23.200835+0000","last_unstale":"2024-08-19T01:34:01.880474+0000","last_undegraded":"2024-08-19T01:34:01.880474+0000","last_fullsized":"2024-08-19T01:34:01.880474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:35:54.312266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880057+0000","last_change":"2024-08-19T01:33:23.187728+0000","last_active":"2024-08-19T01:34:01.880057+0000","last_peered":"2024-08-19T01:34:01.880057+0000","last_clean":"2024-08-19T01:34:01.880057+0000","last_became_active":"2024-08-19T01:33:23.187352+0000","last_became_peered":"2024-08-19T01:33:23.187352+0000","last_unstale":"2024-08-19T01:34:01.880057+0000","last_undegraded":"2024-08-19T01:34:01.880057+0000","last_fullsized":"2024-08-19T01:34:01.880057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:06:03.041328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883303+0000","last_change":"2024-08-19T01:33:23.193146+0000","last_active":"2024-08-19T01:34:01.883303+0000","last_peered":"2024-08-19T01:34:01.883303+0000","last_clean":"2024-08-19T01:34:01.883303+0000","last_became_active":"2024-08-19T01:33:23.192810+0000","last_became_peered":"2024-08-19T01:33:23.192810+0000","last_unstale":"2024-08-19T01:34:01.883303+0000","last_undegraded":"2024-08-19T01:34:01.883303+0000","last_fullsized":"2024-08-19T01:34:01.883303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:12.044175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880515+0000","last_change":"2024-08-19T01:33:23.194635+0000","last_active":"2024-08-19T01:34:01.880515+0000","last_peered":"2024-08-19T01:34:01.880515+0000","last_clean":"2024-08-19T01:34:01.880515+0000","last_became_active":"2024-08-19T01:33:23.194308+0000","last_became_peered":"2024-08-19T01:33:23.194308+0000","last_unstale":"2024-08-19T01:34:01.880515+0000","last_undegraded":"2024-08-19T01:34:01.880515+0000","last_fullsized":"2024-08-19T01:34:01.880515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:12.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929187+0000","last_change":"2024-08-19T01:33:23.193108+0000","last_active":"2024-08-19T01:34:01.929187+0000","last_peered":"2024-08-19T01:34:01.929187+0000","last_clean":"2024-08-19T01:34:01.929187+0000","last_became_active":"2024-08-19T01:33:23.192830+0000","last_became_peered":"2024-08-19T01:33:23.192830+0000","last_unstale":"2024-08-19T01:34:01.929187+0000","last_undegraded":"2024-08-19T01:34:01.929187+0000","last_fullsized":"2024-08-19T01:34:01.929187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:33.355498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048496+0000","last_change":"2024-08-19T01:33:23.195789+0000","last_active":"2024-08-19T01:34:02.048496+0000","last_peered":"2024-08-19T01:34:02.048496+0000","last_clean":"2024-08-19T01:34:02.048496+0000","last_became_active":"2024-08-19T01:33:23.195064+0000","last_became_peered":"2024-08-19T01:33:23.195064+0000","last_unstale":"2024-08-19T01:34:02.048496+0000","last_undegraded":"2024-08-19T01:34:02.048496+0000","last_fullsized":"2024-08-19T01:34:02.048496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:05:33.848953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877781+0000","last_change":"2024-08-19T01:33:23.200968+0000","last_active":"2024-08-19T01:34:01.877781+0000","last_peered":"2024-08-19T01:34:01.877781+0000","last_clean":"2024-08-19T01:34:01.877781+0000","last_became_active":"2024-08-19T01:33:23.200606+0000","last_became_peered":"2024-08-19T01:33:23.200606+0000","last_unstale":"2024-08-19T01:34:01.877781+0000","last_undegraded":"2024-08-19T01:34:01.877781+0000","last_fullsized":"2024-08-19T01:34:01.877781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:44.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878995+0000","last_change":"2024-08-19T01:33:23.191204+0000","last_active":"2024-08-19T01:34:01.878995+0000","last_peered":"2024-08-19T01:34:01.878995+0000","last_clean":"2024-08-19T01:34:01.878995+0000","last_became_active":"2024-08-19T01:33:23.190709+0000","last_became_peered":"2024-08-19T01:33:23.190709+0000","last_unstale":"2024-08-19T01:34:01.878995+0000","last_undegraded":"2024-08-19T01:34:01.878995+0000","last_fullsized":"2024-08-19T01:34:01.878995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:56:10.988967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.885389+0000","last_change":"2024-08-19T01:34:01.885389+0000","last_active":"2024-08-19T01:34:01.885389+0000","last_peered":"2024-08-19T01:34:01.885389+0000","last_clean":"2024-08-19T01:34:01.885389+0000","last_became_active":"2024-08-19T01:34:01.885100+0000","last_became_peered":"2024-08-19T01:34:01.885100+0000","last_unstale":"2024-08-19T01:34:01.885389+0000","last_undegraded":"2024-08-19T01:34:01.885389+0000","last_fullsized":"2024-08-19T01:34:01.885389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:21.616876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882484+0000","last_change":"2024-08-19T01:33:23.187309+0000","last_active":"2024-08-19T01:34:01.882484+0000","last_peered":"2024-08-19T01:34:01.882484+0000","last_clean":"2024-08-19T01:34:01.882484+0000","last_became_active":"2024-08-19T01:33:23.186976+0000","last_became_peered":"2024-08-19T01:33:23.186976+0000","last_unstale":"2024-08-19T01:34:01.882484+0000","last_undegraded":"2024-08-19T01:34:01.882484+0000","last_fullsized":"2024-08-19T01:34:01.882484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:46:02.852000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880176+0000","last_change":"2024-08-19T01:33:23.190292+0000","last_active":"2024-08-19T01:34:01.880176+0000","last_peered":"2024-08-19T01:34:01.880176+0000","last_clean":"2024-08-19T01:34:01.880176+0000","last_became_active":"2024-08-19T01:33:23.189969+0000","last_became_peered":"2024-08-19T01:33:23.189969+0000","last_unstale":"2024-08-19T01:34:01.880176+0000","last_undegraded":"2024-08-19T01:34:01.880176+0000","last_fullsized":"2024-08-19T01:34:01.880176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:35:51.388150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929083+0000","last_change":"2024-08-19T01:33:23.192148+0000","last_active":"2024-08-19T01:34:01.929083+0000","last_peered":"2024-08-19T01:34:01.929083+0000","last_clean":"2024-08-19T01:34:01.929083+0000","last_became_active":"2024-08-19T01:33:23.191845+0000","last_became_peered":"2024-08-19T01:33:23.191845+0000","last_unstale":"2024-08-19T01:34:01.929083+0000","last_undegraded":"2024-08-19T01:34:01.929083+0000","last_fullsized":"2024-08-19T01:34:01.929083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:51.308393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882840+0000","last_change":"2024-08-19T01:33:23.195033+0000","last_active":"2024-08-19T01:34:01.882840+0000","last_peered":"2024-08-19T01:34:01.882840+0000","last_clean":"2024-08-19T01:34:01.882840+0000","last_became_active":"2024-08-19T01:33:23.194745+0000","last_became_peered":"2024-08-19T01:33:23.194745+0000","last_unstale":"2024-08-19T01:34:01.882840+0000","last_undegraded":"2024-08-19T01:34:01.882840+0000","last_fullsized":"2024-08-19T01:34:01.882840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:33:40.748311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882344+0000","last_change":"2024-08-19T01:33:23.195817+0000","last_active":"2024-08-19T01:34:01.882344+0000","last_peered":"2024-08-19T01:34:01.882344+0000","last_clean":"2024-08-19T01:34:01.882344+0000","last_became_active":"2024-08-19T01:33:23.194688+0000","last_became_peered":"2024-08-19T01:33:23.194688+0000","last_unstale":"2024-08-19T01:34:01.882344+0000","last_undegraded":"2024-08-19T01:34:01.882344+0000","last_fullsized":"2024-08-19T01:34:01.882344+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:13:04.913921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883174+0000","last_change":"2024-08-19T01:33:23.187604+0000","last_active":"2024-08-19T01:34:01.883174+0000","last_peered":"2024-08-19T01:34:01.883174+0000","last_clean":"2024-08-19T01:34:01.883174+0000","last_became_active":"2024-08-19T01:33:23.187242+0000","last_became_peered":"2024-08-19T01:33:23.187242+0000","last_unstale":"2024-08-19T01:34:01.883174+0000","last_undegraded":"2024-08-19T01:34:01.883174+0000","last_fullsized":"2024-08-19T01:34:01.883174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:28.592525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879987+0000","last_change":"2024-08-19T01:33:23.191737+0000","last_active":"2024-08-19T01:34:01.879987+0000","last_peered":"2024-08-19T01:34:01.879987+0000","last_clean":"2024-08-19T01:34:01.879987+0000","last_became_active":"2024-08-19T01:33:23.191358+0000","last_became_peered":"2024-08-19T01:33:23.191358+0000","last_unstale":"2024-08-19T01:34:01.879987+0000","last_undegraded":"2024-08-19T01:34:01.879987+0000","last_fullsized":"2024-08-19T01:34:01.879987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:28:14.694157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883084+0000","last_change":"2024-08-19T01:33:23.193110+0000","last_active":"2024-08-19T01:34:01.883084+0000","last_peered":"2024-08-19T01:34:01.883084+0000","last_clean":"2024-08-19T01:34:01.883084+0000","last_became_active":"2024-08-19T01:33:23.192789+0000","last_became_peered":"2024-08-19T01:33:23.192789+0000","last_unstale":"2024-08-19T01:34:01.883084+0000","last_undegraded":"2024-08-19T01:34:01.883084+0000","last_fullsized":"2024-08-19T01:34:01.883084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:45:48.141403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":84,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.049980+0000","last_change":"2024-08-19T01:33:21.173518+0000","last_active":"2024-08-19T01:34:26.049980+0000","last_peered":"2024-08-19T01:34:26.049980+0000","last_clean":"2024-08-19T01:34:26.049980+0000","last_became_active":"2024-08-19T01:33:21.172288+0000","last_became_peered":"2024-08-19T01:33:21.172288+0000","last_unstale":"2024-08-19T01:34:26.049980+0000","last_undegraded":"2024-08-19T01:34:26.049980+0000","last_fullsized":"2024-08-19T01:34:26.049980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:59:42.864724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879305+0000","last_change":"2024-08-19T01:33:23.190160+0000","last_active":"2024-08-19T01:34:01.879305+0000","last_peered":"2024-08-19T01:34:01.879305+0000","last_clean":"2024-08-19T01:34:01.879305+0000","last_became_active":"2024-08-19T01:33:23.189802+0000","last_became_peered":"2024-08-19T01:33:23.189802+0000","last_unstale":"2024-08-19T01:34:01.879305+0000","last_undegraded":"2024-08-19T01:34:01.879305+0000","last_fullsized":"2024-08-19T01:34:01.879305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:53:10.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880543+0000","last_change":"2024-08-19T01:33:21.167443+0000","last_active":"2024-08-19T01:34:01.880543+0000","last_peered":"2024-08-19T01:34:01.880543+0000","last_clean":"2024-08-19T01:34:01.880543+0000","last_became_active":"2024-08-19T01:33:21.167054+0000","last_became_peered":"2024-08-19T01:33:21.167054+0000","last_unstale":"2024-08-19T01:34:01.880543+0000","last_undegraded":"2024-08-19T01:34:01.880543+0000","last_fullsized":"2024-08-19T01:34:01.880543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:00:38.358807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928860+0000","last_change":"2024-08-19T01:33:21.163097+0000","last_active":"2024-08-19T01:34:01.928860+0000","last_peered":"2024-08-19T01:34:01.928860+0000","last_clean":"2024-08-19T01:34:01.928860+0000","last_became_active":"2024-08-19T01:33:21.162802+0000","last_became_peered":"2024-08-19T01:33:21.162802+0000","last_unstale":"2024-08-19T01:34:01.928860+0000","last_undegraded":"2024-08-19T01:34:01.928860+0000","last_fullsized":"2024-08-19T01:34:01.928860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:15:41.244104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877814+0000","last_change":"2024-08-19T01:33:23.200975+0000","last_active":"2024-08-19T01:34:01.877814+0000","last_peered":"2024-08-19T01:34:01.877814+0000","last_clean":"2024-08-19T01:34:01.877814+0000","last_became_active":"2024-08-19T01:33:23.200667+0000","last_became_peered":"2024-08-19T01:33:23.200667+0000","last_unstale":"2024-08-19T01:34:01.877814+0000","last_undegraded":"2024-08-19T01:34:01.877814+0000","last_fullsized":"2024-08-19T01:34:01.877814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:35.105521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879336+0000","last_change":"2024-08-19T01:33:21.171252+0000","last_active":"2024-08-19T01:34:01.879336+0000","last_peered":"2024-08-19T01:34:01.879336+0000","last_clean":"2024-08-19T01:34:01.879336+0000","last_became_active":"2024-08-19T01:33:21.170944+0000","last_became_peered":"2024-08-19T01:33:21.170944+0000","last_unstale":"2024-08-19T01:34:01.879336+0000","last_undegraded":"2024-08-19T01:34:01.879336+0000","last_fullsized":"2024-08-19T01:34:01.879336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:10:15.869452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879832+0000","last_change":"2024-08-19T01:33:23.191931+0000","last_active":"2024-08-19T01:34:01.879832+0000","last_peered":"2024-08-19T01:34:01.879832+0000","last_clean":"2024-08-19T01:34:01.879832+0000","last_became_active":"2024-08-19T01:33:23.191601+0000","last_became_peered":"2024-08-19T01:33:23.191601+0000","last_unstale":"2024-08-19T01:34:01.879832+0000","last_undegraded":"2024-08-19T01:34:01.879832+0000","last_fullsized":"2024-08-19T01:34:01.879832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:52.874546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":130,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.049223+0000","last_change":"2024-08-19T01:33:21.170714+0000","last_active":"2024-08-19T01:34:26.049223+0000","last_peered":"2024-08-19T01:34:26.049223+0000","last_clean":"2024-08-19T01:34:26.049223+0000","last_became_active":"2024-08-19T01:33:21.170379+0000","last_became_peered":"2024-08-19T01:33:21.170379+0000","last_unstale":"2024-08-19T01:34:26.049223+0000","last_undegraded":"2024-08-19T01:34:26.049223+0000","last_fullsized":"2024-08-19T01:34:26.049223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:27:27.417634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048807+0000","last_change":"2024-08-19T01:33:23.189680+0000","last_active":"2024-08-19T01:34:02.048807+0000","last_peered":"2024-08-19T01:34:02.048807+0000","last_clean":"2024-08-19T01:34:02.048807+0000","last_became_active":"2024-08-19T01:33:23.189029+0000","last_became_peered":"2024-08-19T01:33:23.189029+0000","last_unstale":"2024-08-19T01:34:02.048807+0000","last_undegraded":"2024-08-19T01:34:02.048807+0000","last_fullsized":"2024-08-19T01:34:02.048807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:39.667675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881530+0000","last_change":"2024-08-19T01:33:21.169233+0000","last_active":"2024-08-19T01:34:01.881530+0000","last_peered":"2024-08-19T01:34:01.881530+0000","last_clean":"2024-08-19T01:34:01.881530+0000","last_became_active":"2024-08-19T01:33:21.168761+0000","last_became_peered":"2024-08-19T01:33:21.168761+0000","last_unstale":"2024-08-19T01:34:01.881530+0000","last_undegraded":"2024-08-19T01:34:01.881530+0000","last_fullsized":"2024-08-19T01:34:01.881530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:31:56.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883126+0000","last_change":"2024-08-19T01:33:23.194250+0000","last_active":"2024-08-19T01:34:01.883126+0000","last_peered":"2024-08-19T01:34:01.883126+0000","last_clean":"2024-08-19T01:34:01.883126+0000","last_became_active":"2024-08-19T01:33:23.193939+0000","last_became_peered":"2024-08-19T01:33:23.193939+0000","last_unstale":"2024-08-19T01:34:01.883126+0000","last_undegraded":"2024-08-19T01:34:01.883126+0000","last_fullsized":"2024-08-19T01:34:01.883126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:19:09.703701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879370+0000","last_change":"2024-08-19T01:33:21.287684+0000","last_active":"2024-08-19T01:34:01.879370+0000","last_peered":"2024-08-19T01:34:01.879370+0000","last_clean":"2024-08-19T01:34:01.879370+0000","last_became_active":"2024-08-19T01:33:21.287252+0000","last_became_peered":"2024-08-19T01:33:21.287252+0000","last_unstale":"2024-08-19T01:34:01.879370+0000","last_undegraded":"2024-08-19T01:34:01.879370+0000","last_fullsized":"2024-08-19T01:34:01.879370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:08.801345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048572+0000","last_change":"2024-08-19T01:33:23.195077+0000","last_active":"2024-08-19T01:34:02.048572+0000","last_peered":"2024-08-19T01:34:02.048572+0000","last_clean":"2024-08-19T01:34:02.048572+0000","last_became_active":"2024-08-19T01:33:23.194753+0000","last_became_peered":"2024-08-19T01:33:23.194753+0000","last_unstale":"2024-08-19T01:34:02.048572+0000","last_undegraded":"2024-08-19T01:34:02.048572+0000","last_fullsized":"2024-08-19T01:34:02.048572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:12:09.186261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'30","reported_seq":164,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.049168+0000","last_change":"2024-08-19T01:33:21.168845+0000","last_active":"2024-08-19T01:34:26.049168+0000","last_peered":"2024-08-19T01:34:26.049168+0000","last_clean":"2024-08-19T01:34:26.049168+0000","last_became_active":"2024-08-19T01:33:21.168479+0000","last_became_peered":"2024-08-19T01:33:21.168479+0000","last_unstale":"2024-08-19T01:34:26.049168+0000","last_undegraded":"2024-08-19T01:34:26.049168+0000","last_fullsized":"2024-08-19T01:34:26.049168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:59:50.296329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60616,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":114,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883678+0000","last_change":"2024-08-19T01:33:23.192252+0000","last_active":"2024-08-19T01:34:01.883678+0000","last_peered":"2024-08-19T01:34:01.883678+0000","last_clean":"2024-08-19T01:34:01.883678+0000","last_became_active":"2024-08-19T01:33:23.191934+0000","last_became_peered":"2024-08-19T01:33:23.191934+0000","last_unstale":"2024-08-19T01:34:01.883678+0000","last_undegraded":"2024-08-19T01:34:01.883678+0000","last_fullsized":"2024-08-19T01:34:01.883678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:31.134825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882306+0000","last_change":"2024-08-19T01:33:21.168541+0000","last_active":"2024-08-19T01:34:01.882306+0000","last_peered":"2024-08-19T01:34:01.882306+0000","last_clean":"2024-08-19T01:34:01.882306+0000","last_became_active":"2024-08-19T01:33:21.168205+0000","last_became_peered":"2024-08-19T01:33:21.168205+0000","last_unstale":"2024-08-19T01:34:01.882306+0000","last_undegraded":"2024-08-19T01:34:01.882306+0000","last_fullsized":"2024-08-19T01:34:01.882306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:19:52.078947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928897+0000","last_change":"2024-08-19T01:33:23.193013+0000","last_active":"2024-08-19T01:34:01.928897+0000","last_peered":"2024-08-19T01:34:01.928897+0000","last_clean":"2024-08-19T01:34:01.928897+0000","last_became_active":"2024-08-19T01:33:23.192728+0000","last_became_peered":"2024-08-19T01:33:23.192728+0000","last_unstale":"2024-08-19T01:34:01.928897+0000","last_undegraded":"2024-08-19T01:34:01.928897+0000","last_fullsized":"2024-08-19T01:34:01.928897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:23:17.587527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879101+0000","last_change":"2024-08-19T01:33:21.170118+0000","last_active":"2024-08-19T01:34:01.879101+0000","last_peered":"2024-08-19T01:34:01.879101+0000","last_clean":"2024-08-19T01:34:01.879101+0000","last_became_active":"2024-08-19T01:33:21.169757+0000","last_became_peered":"2024-08-19T01:33:21.169757+0000","last_unstale":"2024-08-19T01:34:01.879101+0000","last_undegraded":"2024-08-19T01:34:01.879101+0000","last_fullsized":"2024-08-19T01:34:01.879101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:09:26.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929944+0000","last_change":"2024-08-19T01:34:01.929944+0000","last_active":"2024-08-19T01:34:01.929944+0000","last_peered":"2024-08-19T01:34:01.929944+0000","last_clean":"2024-08-19T01:34:01.929944+0000","last_became_active":"2024-08-19T01:34:01.929679+0000","last_became_peered":"2024-08-19T01:34:01.929679+0000","last_unstale":"2024-08-19T01:34:01.929944+0000","last_undegraded":"2024-08-19T01:34:01.929944+0000","last_fullsized":"2024-08-19T01:34:01.929944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:25:24.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":86,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:28.065960+0000","last_change":"2024-08-19T01:33:21.165204+0000","last_active":"2024-08-19T01:34:28.065960+0000","last_peered":"2024-08-19T01:34:28.065960+0000","last_clean":"2024-08-19T01:34:28.065960+0000","last_became_active":"2024-08-19T01:33:21.164826+0000","last_became_peered":"2024-08-19T01:33:21.164826+0000","last_unstale":"2024-08-19T01:34:28.065960+0000","last_undegraded":"2024-08-19T01:34:28.065960+0000","last_fullsized":"2024-08-19T01:34:28.065960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:45:23.212064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880574+0000","last_change":"2024-08-19T01:33:23.198176+0000","last_active":"2024-08-19T01:34:01.880574+0000","last_peered":"2024-08-19T01:34:01.880574+0000","last_clean":"2024-08-19T01:34:01.880574+0000","last_became_active":"2024-08-19T01:33:23.196187+0000","last_became_peered":"2024-08-19T01:33:23.196187+0000","last_unstale":"2024-08-19T01:34:01.880574+0000","last_undegraded":"2024-08-19T01:34:01.880574+0000","last_fullsized":"2024-08-19T01:34:01.880574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:53:05.943467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.079412+0000","last_change":"2024-08-19T01:34:02.079028+0000","last_active":"2024-08-19T01:34:02.079412+0000","last_peered":"2024-08-19T01:34:02.079412+0000","last_clean":"2024-08-19T01:34:02.079412+0000","last_became_active":"2024-08-19T01:34:02.049890+0000","last_became_peered":"2024-08-19T01:34:02.049890+0000","last_unstale":"2024-08-19T01:34:02.079412+0000","last_undegraded":"2024-08-19T01:34:02.079412+0000","last_fullsized":"2024-08-19T01:34:02.079412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:58.595161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879105+0000","last_change":"2024-08-19T01:33:23.201104+0000","last_active":"2024-08-19T01:34:01.879105+0000","last_peered":"2024-08-19T01:34:01.879105+0000","last_clean":"2024-08-19T01:34:01.879105+0000","last_became_active":"2024-08-19T01:33:23.200809+0000","last_became_peered":"2024-08-19T01:33:23.200809+0000","last_unstale":"2024-08-19T01:34:01.879105+0000","last_undegraded":"2024-08-19T01:34:01.879105+0000","last_fullsized":"2024-08-19T01:34:01.879105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:54.901412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883975+0000","last_change":"2024-08-19T01:33:21.172423+0000","last_active":"2024-08-19T01:34:01.883975+0000","last_peered":"2024-08-19T01:34:01.883975+0000","last_clean":"2024-08-19T01:34:01.883975+0000","last_became_active":"2024-08-19T01:33:21.171950+0000","last_became_peered":"2024-08-19T01:33:21.171950+0000","last_unstale":"2024-08-19T01:34:01.883975+0000","last_undegraded":"2024-08-19T01:34:01.883975+0000","last_fullsized":"2024-08-19T01:34:01.883975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:44:11.446875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880314+0000","last_change":"2024-08-19T01:33:23.192933+0000","last_active":"2024-08-19T01:34:01.880314+0000","last_peered":"2024-08-19T01:34:01.880314+0000","last_clean":"2024-08-19T01:34:01.880314+0000","last_became_active":"2024-08-19T01:33:23.192601+0000","last_became_peered":"2024-08-19T01:33:23.192601+0000","last_unstale":"2024-08-19T01:34:01.880314+0000","last_undegraded":"2024-08-19T01:34:01.880314+0000","last_fullsized":"2024-08-19T01:34:01.880314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:09.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929233+0000","last_change":"2024-08-19T01:33:21.168225+0000","last_active":"2024-08-19T01:34:01.929233+0000","last_peered":"2024-08-19T01:34:01.929233+0000","last_clean":"2024-08-19T01:34:01.929233+0000","last_became_active":"2024-08-19T01:33:21.167881+0000","last_became_peered":"2024-08-19T01:33:21.167881+0000","last_unstale":"2024-08-19T01:34:01.929233+0000","last_undegraded":"2024-08-19T01:34:01.929233+0000","last_fullsized":"2024-08-19T01:34:01.929233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:53:38.037036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881493+0000","last_change":"2024-08-19T01:33:23.194079+0000","last_active":"2024-08-19T01:34:01.881493+0000","last_peered":"2024-08-19T01:34:01.881493+0000","last_clean":"2024-08-19T01:34:01.881493+0000","last_became_active":"2024-08-19T01:33:23.193749+0000","last_became_peered":"2024-08-19T01:33:23.193749+0000","last_unstale":"2024-08-19T01:34:01.881493+0000","last_undegraded":"2024-08-19T01:34:01.881493+0000","last_fullsized":"2024-08-19T01:34:01.881493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:53:41.102163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879491+0000","last_change":"2024-08-19T01:33:21.168916+0000","last_active":"2024-08-19T01:34:01.879491+0000","last_peered":"2024-08-19T01:34:01.879491+0000","last_clean":"2024-08-19T01:34:01.879491+0000","last_became_active":"2024-08-19T01:33:21.168618+0000","last_became_peered":"2024-08-19T01:33:21.168618+0000","last_unstale":"2024-08-19T01:34:01.879491+0000","last_undegraded":"2024-08-19T01:34:01.879491+0000","last_fullsized":"2024-08-19T01:34:01.879491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:45:11.567779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881292+0000","last_change":"2024-08-19T01:33:23.195546+0000","last_active":"2024-08-19T01:34:01.881292+0000","last_peered":"2024-08-19T01:34:01.881292+0000","last_clean":"2024-08-19T01:34:01.881292+0000","last_became_active":"2024-08-19T01:33:23.195221+0000","last_became_peered":"2024-08-19T01:33:23.195221+0000","last_unstale":"2024-08-19T01:34:01.881292+0000","last_undegraded":"2024-08-19T01:34:01.881292+0000","last_fullsized":"2024-08-19T01:34:01.881292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:46:10.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":84,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:28.052921+0000","last_change":"2024-08-19T01:33:21.287946+0000","last_active":"2024-08-19T01:34:28.052921+0000","last_peered":"2024-08-19T01:34:28.052921+0000","last_clean":"2024-08-19T01:34:28.052921+0000","last_became_active":"2024-08-19T01:33:21.287594+0000","last_became_peered":"2024-08-19T01:33:21.287594+0000","last_unstale":"2024-08-19T01:34:28.052921+0000","last_undegraded":"2024-08-19T01:34:28.052921+0000","last_fullsized":"2024-08-19T01:34:28.052921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:10:06.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881482+0000","last_change":"2024-08-19T01:33:23.200914+0000","last_active":"2024-08-19T01:34:01.881482+0000","last_peered":"2024-08-19T01:34:01.881482+0000","last_clean":"2024-08-19T01:34:01.881482+0000","last_became_active":"2024-08-19T01:33:23.200611+0000","last_became_peered":"2024-08-19T01:33:23.200611+0000","last_unstale":"2024-08-19T01:34:01.881482+0000","last_undegraded":"2024-08-19T01:34:01.881482+0000","last_fullsized":"2024-08-19T01:34:01.881482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:22:00.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880857+0000","last_change":"2024-08-19T01:32:09.926979+0000","last_active":"2024-08-19T01:34:01.880857+0000","last_peered":"2024-08-19T01:34:01.880857+0000","last_clean":"2024-08-19T01:34:01.880857+0000","last_became_active":"2024-08-19T01:32:09.926560+0000","last_became_peered":"2024-08-19T01:32:09.926560+0000","last_unstale":"2024-08-19T01:34:01.880857+0000","last_undegraded":"2024-08-19T01:34:01.880857+0000","last_fullsized":"2024-08-19T01:34:01.880857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_clean_scrub_stamp":"2024-08-19T01:29:05.356703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:12:23.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878743+0000","last_change":"2024-08-19T01:33:21.165099+0000","last_active":"2024-08-19T01:34:01.878743+0000","last_peered":"2024-08-19T01:34:01.878743+0000","last_clean":"2024-08-19T01:34:01.878743+0000","last_became_active":"2024-08-19T01:33:21.164756+0000","last_became_peered":"2024-08-19T01:33:21.164756+0000","last_unstale":"2024-08-19T01:34:01.878743+0000","last_undegraded":"2024-08-19T01:34:01.878743+0000","last_fullsized":"2024-08-19T01:34:01.878743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:11:54.480263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877931+0000","last_change":"2024-08-19T01:33:23.200185+0000","last_active":"2024-08-19T01:34:01.877931+0000","last_peered":"2024-08-19T01:34:01.877931+0000","last_clean":"2024-08-19T01:34:01.877931+0000","last_became_active":"2024-08-19T01:33:23.199885+0000","last_became_peered":"2024-08-19T01:33:23.199885+0000","last_unstale":"2024-08-19T01:34:01.877931+0000","last_undegraded":"2024-08-19T01:34:01.877931+0000","last_fullsized":"2024-08-19T01:34:01.877931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:17.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880754+0000","last_change":"2024-08-19T01:33:21.172587+0000","last_active":"2024-08-19T01:34:01.880754+0000","last_peered":"2024-08-19T01:34:01.880754+0000","last_clean":"2024-08-19T01:34:01.880754+0000","last_became_active":"2024-08-19T01:33:21.172233+0000","last_became_peered":"2024-08-19T01:33:21.172233+0000","last_unstale":"2024-08-19T01:34:01.880754+0000","last_undegraded":"2024-08-19T01:34:01.880754+0000","last_fullsized":"2024-08-19T01:34:01.880754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:26.690403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928855+0000","last_change":"2024-08-19T01:33:23.197272+0000","last_active":"2024-08-19T01:34:01.928855+0000","last_peered":"2024-08-19T01:34:01.928855+0000","last_clean":"2024-08-19T01:34:01.928855+0000","last_became_active":"2024-08-19T01:33:23.196869+0000","last_became_peered":"2024-08-19T01:33:23.196869+0000","last_unstale":"2024-08-19T01:34:01.928855+0000","last_undegraded":"2024-08-19T01:34:01.928855+0000","last_fullsized":"2024-08-19T01:34:01.928855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:30.020184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:24.060832+0000","last_change":"2024-08-19T01:33:21.287366+0000","last_active":"2024-08-19T01:34:24.060832+0000","last_peered":"2024-08-19T01:34:24.060832+0000","last_clean":"2024-08-19T01:34:24.060832+0000","last_became_active":"2024-08-19T01:33:21.287044+0000","last_became_peered":"2024-08-19T01:33:21.287044+0000","last_unstale":"2024-08-19T01:34:24.060832+0000","last_undegraded":"2024-08-19T01:34:24.060832+0000","last_fullsized":"2024-08-19T01:34:24.060832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:19:04.194284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.885371+0000","last_change":"2024-08-19T01:34:01.885001+0000","last_active":"2024-08-19T01:34:01.885371+0000","last_peered":"2024-08-19T01:34:01.885371+0000","last_clean":"2024-08-19T01:34:01.885371+0000","last_became_active":"2024-08-19T01:34:01.884680+0000","last_became_peered":"2024-08-19T01:34:01.884680+0000","last_unstale":"2024-08-19T01:34:01.885371+0000","last_undegraded":"2024-08-19T01:34:01.885371+0000","last_fullsized":"2024-08-19T01:34:01.885371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:08.251298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":83,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:26.063456+0000","last_change":"2024-08-19T01:33:21.164639+0000","last_active":"2024-08-19T01:34:26.063456+0000","last_peered":"2024-08-19T01:34:26.063456+0000","last_clean":"2024-08-19T01:34:26.063456+0000","last_became_active":"2024-08-19T01:33:21.164330+0000","last_became_peered":"2024-08-19T01:33:21.164330+0000","last_unstale":"2024-08-19T01:34:26.063456+0000","last_undegraded":"2024-08-19T01:34:26.063456+0000","last_fullsized":"2024-08-19T01:34:26.063456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:26:04.035130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883666+0000","last_change":"2024-08-19T01:34:01.883666+0000","last_active":"2024-08-19T01:34:01.883666+0000","last_peered":"2024-08-19T01:34:01.883666+0000","last_clean":"2024-08-19T01:34:01.883666+0000","last_became_active":"2024-08-19T01:34:01.883264+0000","last_became_peered":"2024-08-19T01:34:01.883264+0000","last_unstale":"2024-08-19T01:34:01.883666+0000","last_undegraded":"2024-08-19T01:34:01.883666+0000","last_fullsized":"2024-08-19T01:34:01.883666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:41:52.266751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879931+0000","last_change":"2024-08-19T01:33:21.287906+0000","last_active":"2024-08-19T01:34:01.879931+0000","last_peered":"2024-08-19T01:34:01.879931+0000","last_clean":"2024-08-19T01:34:01.879931+0000","last_became_active":"2024-08-19T01:33:21.287597+0000","last_became_peered":"2024-08-19T01:33:21.287597+0000","last_unstale":"2024-08-19T01:34:01.879931+0000","last_undegraded":"2024-08-19T01:34:01.879931+0000","last_fullsized":"2024-08-19T01:34:01.879931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:30.605948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880677+0000","last_change":"2024-08-19T01:33:23.191026+0000","last_active":"2024-08-19T01:34:01.880677+0000","last_peered":"2024-08-19T01:34:01.880677+0000","last_clean":"2024-08-19T01:34:01.880677+0000","last_became_active":"2024-08-19T01:33:23.190382+0000","last_became_peered":"2024-08-19T01:33:23.190382+0000","last_unstale":"2024-08-19T01:34:01.880677+0000","last_undegraded":"2024-08-19T01:34:01.880677+0000","last_fullsized":"2024-08-19T01:34:01.880677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:33:45.065441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883898+0000","last_change":"2024-08-19T01:33:21.288008+0000","last_active":"2024-08-19T01:34:01.883898+0000","last_peered":"2024-08-19T01:34:01.883898+0000","last_clean":"2024-08-19T01:34:01.883898+0000","last_became_active":"2024-08-19T01:33:21.287613+0000","last_became_peered":"2024-08-19T01:33:21.287613+0000","last_unstale":"2024-08-19T01:34:01.883898+0000","last_undegraded":"2024-08-19T01:34:01.883898+0000","last_fullsized":"2024-08-19T01:34:01.883898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:35:18.169564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929091+0000","last_change":"2024-08-19T01:33:23.197201+0000","last_active":"2024-08-19T01:34:01.929091+0000","last_peered":"2024-08-19T01:34:01.929091+0000","last_clean":"2024-08-19T01:34:01.929091+0000","last_became_active":"2024-08-19T01:33:23.196946+0000","last_became_peered":"2024-08-19T01:33:23.196946+0000","last_unstale":"2024-08-19T01:34:01.929091+0000","last_undegraded":"2024-08-19T01:34:01.929091+0000","last_fullsized":"2024-08-19T01:34:01.929091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:14:31.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880120+0000","last_change":"2024-08-19T01:33:21.167109+0000","last_active":"2024-08-19T01:34:01.880120+0000","last_peered":"2024-08-19T01:34:01.880120+0000","last_clean":"2024-08-19T01:34:01.880120+0000","last_became_active":"2024-08-19T01:33:21.166788+0000","last_became_peered":"2024-08-19T01:33:21.166788+0000","last_unstale":"2024-08-19T01:34:01.880120+0000","last_undegraded":"2024-08-19T01:34:01.880120+0000","last_fullsized":"2024-08-19T01:34:01.880120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:44.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878126+0000","last_change":"2024-08-19T01:33:23.199699+0000","last_active":"2024-08-19T01:34:01.878126+0000","last_peered":"2024-08-19T01:34:01.878126+0000","last_clean":"2024-08-19T01:34:01.878126+0000","last_became_active":"2024-08-19T01:33:23.199417+0000","last_became_peered":"2024-08-19T01:33:23.199417+0000","last_unstale":"2024-08-19T01:34:01.878126+0000","last_undegraded":"2024-08-19T01:34:01.878126+0000","last_fullsized":"2024-08-19T01:34:01.878126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:36.519135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878888+0000","last_change":"2024-08-19T01:33:21.164177+0000","last_active":"2024-08-19T01:34:01.878888+0000","last_peered":"2024-08-19T01:34:01.878888+0000","last_clean":"2024-08-19T01:34:01.878888+0000","last_became_active":"2024-08-19T01:33:21.163832+0000","last_became_peered":"2024-08-19T01:33:21.163832+0000","last_unstale":"2024-08-19T01:34:01.878888+0000","last_undegraded":"2024-08-19T01:34:01.878888+0000","last_fullsized":"2024-08-19T01:34:01.878888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:31.907677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882973+0000","last_change":"2024-08-19T01:33:23.195102+0000","last_active":"2024-08-19T01:34:01.882973+0000","last_peered":"2024-08-19T01:34:01.882973+0000","last_clean":"2024-08-19T01:34:01.882973+0000","last_became_active":"2024-08-19T01:33:23.194802+0000","last_became_peered":"2024-08-19T01:33:23.194802+0000","last_unstale":"2024-08-19T01:34:01.882973+0000","last_undegraded":"2024-08-19T01:34:01.882973+0000","last_fullsized":"2024-08-19T01:34:01.882973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:33.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048628+0000","last_change":"2024-08-19T01:33:21.287933+0000","last_active":"2024-08-19T01:34:02.048628+0000","last_peered":"2024-08-19T01:34:02.048628+0000","last_clean":"2024-08-19T01:34:02.048628+0000","last_became_active":"2024-08-19T01:33:21.287568+0000","last_became_peered":"2024-08-19T01:33:21.287568+0000","last_unstale":"2024-08-19T01:34:02.048628+0000","last_undegraded":"2024-08-19T01:34:02.048628+0000","last_fullsized":"2024-08-19T01:34:02.048628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:23.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883283+0000","last_change":"2024-08-19T01:33:23.194320+0000","last_active":"2024-08-19T01:34:01.883283+0000","last_peered":"2024-08-19T01:34:01.883283+0000","last_clean":"2024-08-19T01:34:01.883283+0000","last_became_active":"2024-08-19T01:33:23.194002+0000","last_became_peered":"2024-08-19T01:33:23.194002+0000","last_unstale":"2024-08-19T01:34:01.883283+0000","last_undegraded":"2024-08-19T01:34:01.883283+0000","last_fullsized":"2024-08-19T01:34:01.883283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:59.198997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877938+0000","last_change":"2024-08-19T01:33:21.165421+0000","last_active":"2024-08-19T01:34:01.877938+0000","last_peered":"2024-08-19T01:34:01.877938+0000","last_clean":"2024-08-19T01:34:01.877938+0000","last_became_active":"2024-08-19T01:33:21.165013+0000","last_became_peered":"2024-08-19T01:33:21.165013+0000","last_unstale":"2024-08-19T01:34:01.877938+0000","last_undegraded":"2024-08-19T01:34:01.877938+0000","last_fullsized":"2024-08-19T01:34:01.877938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:07:18.546099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880098+0000","last_change":"2024-08-19T01:33:23.193544+0000","last_active":"2024-08-19T01:34:01.880098+0000","last_peered":"2024-08-19T01:34:01.880098+0000","last_clean":"2024-08-19T01:34:01.880098+0000","last_became_active":"2024-08-19T01:33:23.193213+0000","last_became_peered":"2024-08-19T01:33:23.193213+0000","last_unstale":"2024-08-19T01:34:01.880098+0000","last_undegraded":"2024-08-19T01:34:01.880098+0000","last_fullsized":"2024-08-19T01:34:01.880098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:20:54.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.890071+0000","last_change":"2024-08-19T01:34:01.889627+0000","last_active":"2024-08-19T01:34:01.890071+0000","last_peered":"2024-08-19T01:34:01.890071+0000","last_clean":"2024-08-19T01:34:01.890071+0000","last_became_active":"2024-08-19T01:34:01.885181+0000","last_became_peered":"2024-08-19T01:34:01.885181+0000","last_unstale":"2024-08-19T01:34:01.890071+0000","last_undegraded":"2024-08-19T01:34:01.890071+0000","last_fullsized":"2024-08-19T01:34:01.890071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:51.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878113+0000","last_change":"2024-08-19T01:33:23.196266+0000","last_active":"2024-08-19T01:34:01.878113+0000","last_peered":"2024-08-19T01:34:01.878113+0000","last_clean":"2024-08-19T01:34:01.878113+0000","last_became_active":"2024-08-19T01:33:23.194268+0000","last_became_peered":"2024-08-19T01:33:23.194268+0000","last_unstale":"2024-08-19T01:34:01.878113+0000","last_undegraded":"2024-08-19T01:34:01.878113+0000","last_fullsized":"2024-08-19T01:34:01.878113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:15:38.628746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.890210+0000","last_change":"2024-08-19T01:34:01.890210+0000","last_active":"2024-08-19T01:34:01.890210+0000","last_peered":"2024-08-19T01:34:01.890210+0000","last_clean":"2024-08-19T01:34:01.890210+0000","last_became_active":"2024-08-19T01:34:01.885955+0000","last_became_peered":"2024-08-19T01:34:01.885955+0000","last_unstale":"2024-08-19T01:34:01.890210+0000","last_undegraded":"2024-08-19T01:34:01.890210+0000","last_fullsized":"2024-08-19T01:34:01.890210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.176530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881137+0000","last_change":"2024-08-19T01:33:23.202534+0000","last_active":"2024-08-19T01:34:01.881137+0000","last_peered":"2024-08-19T01:34:01.881137+0000","last_clean":"2024-08-19T01:34:01.881137+0000","last_became_active":"2024-08-19T01:33:23.202191+0000","last_became_peered":"2024-08-19T01:33:23.202191+0000","last_unstale":"2024-08-19T01:34:01.881137+0000","last_undegraded":"2024-08-19T01:34:01.881137+0000","last_fullsized":"2024-08-19T01:34:01.881137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:03.478905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879469+0000","last_change":"2024-08-19T01:33:23.201136+0000","last_active":"2024-08-19T01:34:01.879469+0000","last_peered":"2024-08-19T01:34:01.879469+0000","last_clean":"2024-08-19T01:34:01.879469+0000","last_became_active":"2024-08-19T01:33:23.200864+0000","last_became_peered":"2024-08-19T01:33:23.200864+0000","last_unstale":"2024-08-19T01:34:01.879469+0000","last_undegraded":"2024-08-19T01:34:01.879469+0000","last_fullsized":"2024-08-19T01:34:01.879469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:32:42.091880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883696+0000","last_change":"2024-08-19T01:33:21.172149+0000","last_active":"2024-08-19T01:34:01.883696+0000","last_peered":"2024-08-19T01:34:01.883696+0000","last_clean":"2024-08-19T01:34:01.883696+0000","last_became_active":"2024-08-19T01:33:21.171833+0000","last_became_peered":"2024-08-19T01:33:21.171833+0000","last_unstale":"2024-08-19T01:34:01.883696+0000","last_undegraded":"2024-08-19T01:34:01.883696+0000","last_fullsized":"2024-08-19T01:34:01.883696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:25:11.097212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048466+0000","last_change":"2024-08-19T01:33:23.193675+0000","last_active":"2024-08-19T01:34:02.048466+0000","last_peered":"2024-08-19T01:34:02.048466+0000","last_clean":"2024-08-19T01:34:02.048466+0000","last_became_active":"2024-08-19T01:33:23.188719+0000","last_became_peered":"2024-08-19T01:33:23.188719+0000","last_unstale":"2024-08-19T01:34:02.048466+0000","last_undegraded":"2024-08-19T01:34:02.048466+0000","last_fullsized":"2024-08-19T01:34:02.048466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:54:12.698211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883781+0000","last_change":"2024-08-19T01:33:21.172675+0000","last_active":"2024-08-19T01:34:01.883781+0000","last_peered":"2024-08-19T01:34:01.883781+0000","last_clean":"2024-08-19T01:34:01.883781+0000","last_became_active":"2024-08-19T01:33:21.172320+0000","last_became_peered":"2024-08-19T01:33:21.172320+0000","last_unstale":"2024-08-19T01:34:01.883781+0000","last_undegraded":"2024-08-19T01:34:01.883781+0000","last_fullsized":"2024-08-19T01:34:01.883781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:29:41.511220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883123+0000","last_change":"2024-08-19T01:33:23.203113+0000","last_active":"2024-08-19T01:34:01.883123+0000","last_peered":"2024-08-19T01:34:01.883123+0000","last_clean":"2024-08-19T01:34:01.883123+0000","last_became_active":"2024-08-19T01:33:23.202807+0000","last_became_peered":"2024-08-19T01:33:23.202807+0000","last_unstale":"2024-08-19T01:34:01.883123+0000","last_undegraded":"2024-08-19T01:34:01.883123+0000","last_fullsized":"2024-08-19T01:34:01.883123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:24:21.570929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878197+0000","last_change":"2024-08-19T01:33:21.287382+0000","last_active":"2024-08-19T01:34:01.878197+0000","last_peered":"2024-08-19T01:34:01.878197+0000","last_clean":"2024-08-19T01:34:01.878197+0000","last_became_active":"2024-08-19T01:33:21.287040+0000","last_became_peered":"2024-08-19T01:33:21.287040+0000","last_unstale":"2024-08-19T01:34:01.878197+0000","last_undegraded":"2024-08-19T01:34:01.878197+0000","last_fullsized":"2024-08-19T01:34:01.878197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:01:49.973969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878016+0000","last_change":"2024-08-19T01:33:23.197869+0000","last_active":"2024-08-19T01:34:01.878016+0000","last_peered":"2024-08-19T01:34:01.878016+0000","last_clean":"2024-08-19T01:34:01.878016+0000","last_became_active":"2024-08-19T01:33:23.197529+0000","last_became_peered":"2024-08-19T01:33:23.197529+0000","last_unstale":"2024-08-19T01:34:01.878016+0000","last_undegraded":"2024-08-19T01:34:01.878016+0000","last_fullsized":"2024-08-19T01:34:01.878016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:14.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":91,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:28.065153+0000","last_change":"2024-08-19T01:33:21.172180+0000","last_active":"2024-08-19T01:34:28.065153+0000","last_peered":"2024-08-19T01:34:28.065153+0000","last_clean":"2024-08-19T01:34:28.065153+0000","last_became_active":"2024-08-19T01:33:21.171895+0000","last_became_peered":"2024-08-19T01:33:21.171895+0000","last_unstale":"2024-08-19T01:34:28.065153+0000","last_undegraded":"2024-08-19T01:34:28.065153+0000","last_fullsized":"2024-08-19T01:34:28.065153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:51:38.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68208,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":570,"num_read_kb":356,"num_write":244,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":204624,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449116,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":454083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":1106006,"data_compressed":37649,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874276,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":453901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":454167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":1047576,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":454008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1046474,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":453869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":453443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":515645,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":515649,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":454957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62870,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62866,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62742,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:34:32.700 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-19T01:34:32.701 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-19T01:34:32.701 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-19T01:34:32.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph health --format=json 2024-08-19T01:34:32.955 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 -- 172.21.15.103:0/2626654478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 msgr2=0x7fdd1c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/2626654478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c10e780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009cd0 tx=0x7fdd1402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 -- 172.21.15.103:0/2626654478 shutdown_connections 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/2626654478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c10e780 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/2626654478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1c1018b0 0x7fdd1c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/2626654478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd1c1078b0 0x7fdd1c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 -- 172.21.15.103:0/2626654478 >> 172.21.15.103:0/2626654478 conn(0x7fdd1c0fd610 msgr2=0x7fdd1c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 -- 172.21.15.103:0/2626654478 shutdown_connections 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.449+0000 7fdd20ecb640 1 -- 172.21.15.103:0/2626654478 wait complete. 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 Processor -- start 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 -- start start 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd1c1018b0 0x7fdd1c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1c1078b0 0x7fdd1c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c111b60 con 0x7fdd1c102250 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1119e0 con 0x7fdd1c1018b0 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd20ecb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c111ce0 con 0x7fdd1c1078b0 2024-08-19T01:34:33.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52558/0 (socket says 172.21.15.103:52558) 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd13fff640 1 -- 172.21.15.103:0/3458704700 learned_addr learned my addr 172.21.15.103:0/3458704700 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd1a59c640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd1c1018b0 0x7fdd1c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.450+0000 7fdd1ad9d640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1c1078b0 0x7fdd1c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd13fff640 1 -- 172.21.15.103:0/3458704700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1c1078b0 msgr2=0x7fdd1c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd13fff640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1c1078b0 0x7fdd1c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd13fff640 1 -- 172.21.15.103:0/3458704700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd1c1018b0 msgr2=0x7fdd1c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd13fff640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd1c1018b0 0x7fdd1c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd13fff640 1 -- 172.21.15.103:0/3458704700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c19d070 con 0x7fdd1c102250 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd13fff640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c198600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d770 tx=0x7fdd0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd137fe640 1 -- 172.21.15.103:0/3458704700 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04017070 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd137fe640 1 -- 172.21.15.103:0/3458704700 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd040054d0 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c19d360 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd137fe640 1 -- 172.21.15.103:0/3458704700 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04004cf0 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.451+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c1a4c40 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.452+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c101d10 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.454+0000 7fdd137fe640 1 -- 172.21.15.103:0/3458704700 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdd04004e90 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.455+0000 7fdd137fe640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdd080a2c10 0x7fdd080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.455+0000 7fdd1a59c640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdd080a2c10 0x7fdd080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.455+0000 7fdd137fe640 1 -- 172.21.15.103:0/3458704700 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdd040cefb0 con 0x7fdd1c102250 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.455+0000 7fdd1a59c640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdd080a2c10 0x7fdd080a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00bb60 tx=0x7fdd0c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:33.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.458+0000 7fdd137fe640 1 -- 172.21.15.103:0/3458704700 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdd0409b790 con 0x7fdd1c102250 2024-08-19T01:34:33.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.672+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdd1c199420 con 0x7fdd1c102250 2024-08-19T01:34:33.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.673+0000 7fdd137fe640 1 -- 172.21.15.103:0/3458704700 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdd1c199420 con 0x7fdd1c102250 2024-08-19T01:34:33.675 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:33.675 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdd080a2c10 msgr2=0x7fdd080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdd080a2c10 0x7fdd080a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00bb60 tx=0x7fdd0c00b390 comp rx=0 tx=0).stop 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 msgr2=0x7fdd1c198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c198600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400d770 tx=0x7fdd0400dc30 comp rx=0 tx=0).stop 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 shutdown_connections 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdd080a2c10 0x7fdd080a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1c1078b0 0x7fdd1c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd1c102250 0x7fdd1c198600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 --2- 172.21.15.103:0/3458704700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd1c1018b0 0x7fdd1c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:33.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 >> 172.21.15.103:0/3458704700 conn(0x7fdd1c0fd610 msgr2=0x7fdd1c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:33.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 shutdown_connections 2024-08-19T01:34:33.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:33.675+0000 7fdd20ecb640 1 -- 172.21.15.103:0/3458704700 wait complete. 2024-08-19T01:34:34.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:33 smithi103 ceph-mon[18436]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:34.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:33 smithi103 ceph-mon[18436]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:34.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:33 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3458704700' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-19T01:34:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:33 smithi146 ceph-mon[22098]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:33 smithi146 ceph-mon[22098]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:33 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3458704700' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-19T01:34:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:33 smithi190 ceph-mon[22032]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:34:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:33 smithi190 ceph-mon[22032]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:33 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3458704700' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-19T01:34:34.423 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-19T01:34:34.423 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:35.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 -- 172.21.15.103:0/3820015548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1022a0 msgr2=0x7f8b0c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 --2- 172.21.15.103:0/3820015548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1022a0 0x7f8b0c1026e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009900 tx=0x7f8b0002a480 comp rx=0 tx=0).stop 2024-08-19T01:34:35.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 -- 172.21.15.103:0/3820015548 shutdown_connections 2024-08-19T01:34:35.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 --2- 172.21.15.103:0/3820015548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b0c102c20 0x7f8b0c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 --2- 172.21.15.103:0/3820015548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c1022a0 0x7f8b0c1026e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 --2- 172.21.15.103:0/3820015548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c100b50 0x7f8b0c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 -- 172.21.15.103:0/3820015548 >> 172.21.15.103:0/3820015548 conn(0x7f8b0c0fc500 msgr2=0x7f8b0c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:35.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 -- 172.21.15.103:0/3820015548 shutdown_connections 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.205+0000 7f8b11559640 1 -- 172.21.15.103:0/3820015548 wait complete. 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 Processor -- start 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 -- start start 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c100b50 0x7f8b0c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b0c1022a0 0x7f8b0c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c102c20 0x7f8b0c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c110b60 con 0x7f8b0c100b50 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1109e0 con 0x7f8b0c102c20 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b11559640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c110ce0 con 0x7f8b0c1022a0 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c100b50 0x7f8b0c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c100b50 0x7f8b0c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52566/0 (socket says 172.21.15.103:52566) 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 -- 172.21.15.103:0/1914983850 learned_addr learned my addr 172.21.15.103:0/1914983850 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:35.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0b7fe640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c102c20 0x7f8b0c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 -- 172.21.15.103:0/1914983850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b0c1022a0 msgr2=0x7f8b0c197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0a7fc640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b0c1022a0 0x7f8b0c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b0c1022a0 0x7f8b0c197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 -- 172.21.15.103:0/1914983850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c102c20 msgr2=0x7f8b0c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c102c20 0x7f8b0c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.206+0000 7f8b0affd640 1 -- 172.21.15.103:0/1914983850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c19bfd0 con 0x7f8b0c100b50 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.207+0000 7f8b0affd640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c100b50 0x7f8b0c197020 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00d950 tx=0x7f8afc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.207+0000 7f8af3fff640 1 -- 172.21.15.103:0/1914983850 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc017070 con 0x7f8b0c100b50 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.207+0000 7f8af3fff640 1 -- 172.21.15.103:0/1914983850 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8afc009c30 con 0x7f8b0c100b50 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.207+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c19c2c0 con 0x7f8b0c100b50 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.207+0000 7f8af3fff640 1 -- 172.21.15.103:0/1914983850 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00f620 con 0x7f8b0c100b50 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.207+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a3ba0 con 0x7f8b0c100b50 2024-08-19T01:34:35.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.208+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0c100f30 con 0x7f8b0c100b50 2024-08-19T01:34:35.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.209+0000 7f8af3fff640 1 -- 172.21.15.103:0/1914983850 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8afc0109a0 con 0x7f8b0c100b50 2024-08-19T01:34:35.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.210+0000 7f8af3fff640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8af40a2c10 0x7f8af40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:35.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.210+0000 7f8b0a7fc640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8af40a2c10 0x7f8af40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:35.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.210+0000 7f8af3fff640 1 -- 172.21.15.103:0/1914983850 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8afc0cee40 con 0x7f8b0c100b50 2024-08-19T01:34:35.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.210+0000 7f8b0a7fc640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8af40a2c10 0x7f8af40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006010 tx=0x7f8b00000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:35.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.214+0000 7f8af3fff640 1 -- 172.21.15.103:0/1914983850 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8afc009de0 con 0x7f8b0c100b50 2024-08-19T01:34:35.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.416+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b0c108dc0 con 0x7f8b0c100b50 2024-08-19T01:34:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.417+0000 7f8af3fff640 1 -- 172.21.15.103:0/1914983850 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11989 (secure 0 0 0) 0x7f8afc09bc10 con 0x7f8b0c100b50 2024-08-19T01:34:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:35.419 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-08-19T01:33:34:962864+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:33:34.962843+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:34:35.419 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-08-19T01:34:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8af40a2c10 msgr2=0x7f8af40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8af40a2c10 0x7f8af40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006010 tx=0x7f8b00000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c100b50 msgr2=0x7f8b0c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c100b50 0x7f8b0c197020 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00d950 tx=0x7f8afc00de10 comp rx=0 tx=0).stop 2024-08-19T01:34:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 shutdown_connections 2024-08-19T01:34:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8af40a2c10 0x7f8af40a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b0c102c20 0x7f8b0c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b0c1022a0 0x7f8b0c197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 --2- 172.21.15.103:0/1914983850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0c100b50 0x7f8b0c197020 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 >> 172.21.15.103:0/1914983850 conn(0x7f8b0c0fc500 msgr2=0x7f8b0c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 shutdown_connections 2024-08-19T01:34:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:35.419+0000 7f8b11559640 1 -- 172.21.15.103:0/1914983850 wait complete. 2024-08-19T01:34:36.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:35 smithi103 ceph-mon[18436]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:36.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:35 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1914983850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:36.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:35 smithi146 ceph-mon[22098]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:35 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1914983850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:35 smithi190 ceph-mon[22032]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:35 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1914983850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:36.390 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:36.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.911+0000 7f255a959640 1 -- 172.21.15.103:0/4108626447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 msgr2=0x7f2554102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:36.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.911+0000 7f255a959640 1 --2- 172.21.15.103:0/4108626447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554102e50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2544009900 tx=0x7f254402a480 comp rx=0 tx=0).stop 2024-08-19T01:34:36.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- 172.21.15.103:0/4108626447 shutdown_connections 2024-08-19T01:34:36.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 --2- 172.21.15.103:0/4108626447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f255410bea0 0x7f255410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:36.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 --2- 172.21.15.103:0/4108626447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2554103390 0x7f255410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 --2- 172.21.15.103:0/4108626447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554102e50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- 172.21.15.103:0/4108626447 >> 172.21.15.103:0/4108626447 conn(0x7f25540fc4d0 msgr2=0x7f25540fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- 172.21.15.103:0/4108626447 shutdown_connections 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- 172.21.15.103:0/4108626447 wait complete. 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 Processor -- start 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- start start 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2554103390 0x7f2554198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f255410bea0 0x7f255419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2554111bb0 con 0x7f2554102a70 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2554111a30 con 0x7f2554103390 2024-08-19T01:34:36.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f255a959640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2554111d30 con 0x7f255410bea0 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.912+0000 7f2553fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52586/0 (socket says 172.21.15.103:52586) 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 -- 172.21.15.103:0/2094295534 learned_addr learned my addr 172.21.15.103:0/2094295534 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f254bfff640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2554103390 0x7f2554198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 -- 172.21.15.103:0/2094295534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f255410bea0 msgr2=0x7f255419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2558ef6640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f255410bea0 0x7f255419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f255410bea0 0x7f255419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 -- 172.21.15.103:0/2094295534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2554103390 msgr2=0x7f2554198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2554103390 0x7f2554198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 -- 172.21.15.103:0/2094295534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255419d1c0 con 0x7f2554102a70 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f254bfff640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2554103390 0x7f2554198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2553fff640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554198210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2544006010 tx=0x7f2544004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2551ffb640 1 -- 172.21.15.103:0/2094295534 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254403f030 con 0x7f2554102a70 2024-08-19T01:34:36.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255419d450 con 0x7f2554102a70 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2551ffb640 1 -- 172.21.15.103:0/2094295534 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25440035f0 con 0x7f2554102a70 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f2551ffb640 1 -- 172.21.15.103:0/2094295534 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544035070 con 0x7f2554102a70 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.913+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25541a4d90 con 0x7f2554102a70 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.914+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2554102e50 con 0x7f2554102a70 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.916+0000 7f2551ffb640 1 -- 172.21.15.103:0/2094295534 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2544003b10 con 0x7f2554102a70 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.917+0000 7f2551ffb640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25400a2c10 0x7f25400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.917+0000 7f254bfff640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25400a2c10 0x7f25400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.917+0000 7f2551ffb640 1 -- 172.21.15.103:0/2094295534 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f25440effe0 con 0x7f2554102a70 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.917+0000 7f254bfff640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25400a2c10 0x7f25400a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f253c005fd0 tx=0x7f253c005f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:36.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:36.920+0000 7f2551ffb640 1 -- 172.21.15.103:0/2094295534 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f25440bce10 con 0x7f2554102a70 2024-08-19T01:34:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.120+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2554106f10 con 0x7f2554102a70 2024-08-19T01:34:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.121+0000 7f2551ffb640 1 -- 172.21.15.103:0/2094295534 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11989 (secure 0 0 0) 0x7f25440bc710 con 0x7f2554102a70 2024-08-19T01:34:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:37.124 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-08-19T01:33:34:962864+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:33:34.962843+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:34:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-08-19T01:34:37.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25400a2c10 msgr2=0x7f25400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:37.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25400a2c10 0x7f25400a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f253c005fd0 tx=0x7f253c005f40 comp rx=0 tx=0).stop 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 msgr2=0x7f2554198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554198210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2544006010 tx=0x7f2544004270 comp rx=0 tx=0).stop 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 shutdown_connections 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25400a2c10 0x7f25400a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f255410bea0 0x7f255419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2554103390 0x7f2554198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 --2- 172.21.15.103:0/2094295534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2554102a70 0x7f2554198210 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.123+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 >> 172.21.15.103:0/2094295534 conn(0x7f25540fc4d0 msgr2=0x7f25540fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.124+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 shutdown_connections 2024-08-19T01:34:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:37.124+0000 7f255a959640 1 -- 172.21.15.103:0/2094295534 wait complete. 2024-08-19T01:34:37.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:34:38.084 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:38.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:37 smithi146 ceph-mon[22098]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:38.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:37 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2094295534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:38.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:37 smithi103 ceph-mon[18436]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:38.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:37 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2094295534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:37 smithi190 ceph-mon[22032]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:37 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2094295534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:38.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.609+0000 7f579b300640 1 -- 172.21.15.103:0/116755435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794100b50 msgr2=0x7f5794100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:38.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.609+0000 7f579b300640 1 --2- 172.21.15.103:0/116755435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794100b50 0x7f5794100f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5790009ca0 tx=0x7f579002a950 comp rx=0 tx=0).stop 2024-08-19T01:34:38.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.609+0000 7f579b300640 1 -- 172.21.15.103:0/116755435 shutdown_connections 2024-08-19T01:34:38.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.609+0000 7f579b300640 1 --2- 172.21.15.103:0/116755435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5794102c20 0x7f579410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.609+0000 7f579b300640 1 --2- 172.21.15.103:0/116755435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57941022a0 0x7f57941026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.609+0000 7f579b300640 1 --2- 172.21.15.103:0/116755435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794100b50 0x7f5794100f30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.609+0000 7f579b300640 1 -- 172.21.15.103:0/116755435 >> 172.21.15.103:0/116755435 conn(0x7f57940fc500 msgr2=0x7f57940fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 -- 172.21.15.103:0/116755435 shutdown_connections 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 -- 172.21.15.103:0/116755435 wait complete. 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 Processor -- start 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 -- start start 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5794100b50 0x7f5794199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57941022a0 0x7f57941996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794102c20 0x7f57941a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5794110c00 con 0x7f5794102c20 2024-08-19T01:34:38.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5794110a80 con 0x7f57941022a0 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.610+0000 7f579b300640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5794110d80 con 0x7f5794100b50 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794102c20 0x7f57941a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794102c20 0x7f57941a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52604/0 (socket says 172.21.15.103:52604) 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 -- 172.21.15.103:0/699201223 learned_addr learned my addr 172.21.15.103:0/699201223 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579909c640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5794100b50 0x7f5794199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579889b640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57941022a0 0x7f57941996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 -- 172.21.15.103:0/699201223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5794100b50 msgr2=0x7f5794199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5794100b50 0x7f5794199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 -- 172.21.15.103:0/699201223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57941022a0 msgr2=0x7f57941996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57941022a0 0x7f57941996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 -- 172.21.15.103:0/699201223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57941a4730 con 0x7f5794102c20 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579989d640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794102c20 0x7f57941a41f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f578c009fd0 tx=0x7f578c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f578a7fc640 1 -- 172.21.15.103:0/699201223 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c015020 con 0x7f5794102c20 2024-08-19T01:34:38.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f578a7fc640 1 -- 172.21.15.103:0/699201223 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f578c011ce0 con 0x7f5794102c20 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57941a4900 con 0x7f5794102c20 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f578a7fc640 1 -- 172.21.15.103:0/699201223 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c010d70 con 0x7f5794102c20 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.611+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57941a4e10 con 0x7f5794102c20 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.612+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5794100fb0 con 0x7f5794102c20 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.613+0000 7f578a7fc640 1 -- 172.21.15.103:0/699201223 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f578c018020 con 0x7f5794102c20 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.614+0000 7f578a7fc640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f57800a2c10 0x7f57800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.614+0000 7f579909c640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f57800a2c10 0x7f57800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.614+0000 7f578a7fc640 1 -- 172.21.15.103:0/699201223 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f578c0ce860 con 0x7f5794102c20 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.615+0000 7f579909c640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f57800a2c10 0x7f57800a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f57900093d0 tx=0x7f5790009360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:38.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.618+0000 7f578a7fc640 1 -- 172.21.15.103:0/699201223 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f578c01f020 con 0x7f5794102c20 2024-08-19T01:34:38.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.787+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5794108dc0 con 0x7f5794102c20 2024-08-19T01:34:38.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.788+0000 7f578a7fc640 1 -- 172.21.15.103:0/699201223 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22704 (secure 0 0 0) 0x7f578c09b7a0 con 0x7f5794102c20 2024-08-19T01:34:38.791 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:38.791 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:34:01.874407+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f57800a2c10 msgr2=0x7f57800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f57800a2c10 0x7f57800a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f57900093d0 tx=0x7f5790009360 comp rx=0 tx=0).stop 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794102c20 msgr2=0x7f57941a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794102c20 0x7f57941a41f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f578c009fd0 tx=0x7f578c00b820 comp rx=0 tx=0).stop 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 shutdown_connections 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f57800a2c10 0x7f57800a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5794102c20 0x7f57941a41f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57941022a0 0x7f57941996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 --2- 172.21.15.103:0/699201223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5794100b50 0x7f5794199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 >> 172.21.15.103:0/699201223 conn(0x7f57940fc500 msgr2=0x7f57940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 shutdown_connections 2024-08-19T01:34:38.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:38.791+0000 7f579b300640 1 -- 172.21.15.103:0/699201223 wait complete. 2024-08-19T01:34:39.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:38 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/699201223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:39.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:38 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/699201223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:39.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:38 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/699201223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:34:39.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:34:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:39 smithi146 ceph-mon[22098]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:40.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:39 smithi103 ceph-mon[18436]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:40.282 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:39 smithi190 ceph-mon[22032]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/2876654547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b00ffe30 msgr2=0x7ff0b0100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/2876654547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b00ffe30 0x7ff0b0100290 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009900 tx=0x7ff0a402a410 comp rx=0 tx=0).stop 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/2876654547 shutdown_connections 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/2876654547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b01007d0 0x7ff0b010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/2876654547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b00ffe30 0x7ff0b0100290 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/2876654547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b01068b0 0x7ff0b0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/2876654547 >> 172.21.15.103:0/2876654547 conn(0x7ff0b0076f60 msgr2=0x7ff0b00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/2876654547 shutdown_connections 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/2876654547 wait complete. 2024-08-19T01:34:40.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 Processor -- start 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.313+0000 7ff0b7fa8640 1 -- start start 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00ffe30 0x7ff0b01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b01007d0 0x7ff0b0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b01068b0 0x7ff0b019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0111c70 con 0x7ff0b01007d0 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0111af0 con 0x7ff0b01068b0 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0111df0 con 0x7ff0b00ffe30 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b01007d0 0x7ff0b0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b01007d0 0x7ff0b0198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52620/0 (socket says 172.21.15.103:52620) 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 -- 172.21.15.103:0/3260060163 learned_addr learned my addr 172.21.15.103:0/3260060163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5d44640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00ffe30 0x7ff0b01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 -- 172.21.15.103:0/3260060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00ffe30 msgr2=0x7ff0b01980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00ffe30 0x7ff0b01980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 -- 172.21.15.103:0/3260060163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b01068b0 msgr2=0x7ff0b019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b01068b0 0x7ff0b019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 -- 172.21.15.103:0/3260060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b019d0a0 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b5543640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b01007d0 0x7ff0b0198630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4006010 tx=0x7ff0a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0aeffd640 1 -- 172.21.15.103:0/3260060163 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a403f030 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b019d330 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0aeffd640 1 -- 172.21.15.103:0/3260060163 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0a40035f0 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0aeffd640 1 -- 172.21.15.103:0/3260060163 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4035070 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.314+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b01a4c70 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.315+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b0103150 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.316+0000 7ff0aeffd640 1 -- 172.21.15.103:0/3260060163 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff0a4003b10 con 0x7ff0b01007d0 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.317+0000 7ff0aeffd640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0980a2c10 0x7ff0980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:40.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.317+0000 7ff0b5d44640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0980a2c10 0x7ff0980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:40.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.317+0000 7ff0aeffd640 1 -- 172.21.15.103:0/3260060163 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff0a40efdd0 con 0x7ff0b01007d0 2024-08-19T01:34:40.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.317+0000 7ff0b5d44640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0980a2c10 0x7ff0980a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0007350 tx=0x7ff0a00072e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:40.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.321+0000 7ff0aeffd640 1 -- 172.21.15.103:0/3260060163 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff0a40bcc00 con 0x7ff0b01007d0 2024-08-19T01:34:40.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.521+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0b01a4e70 con 0x7ff0b01007d0 2024-08-19T01:34:40.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.521+0000 7ff0aeffd640 1 -- 172.21.15.103:0/3260060163 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11989 (secure 0 0 0) 0x7ff0a40bc500 con 0x7ff0b01007d0 2024-08-19T01:34:40.524 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:40.524 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-08-19T01:33:34:962864+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:33:34.962843+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:34:40.524 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0980a2c10 msgr2=0x7ff0980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0980a2c10 0x7ff0980a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0007350 tx=0x7ff0a00072e0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b01007d0 msgr2=0x7ff0b0198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b01007d0 0x7ff0b0198630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4006010 tx=0x7ff0a4004270 comp rx=0 tx=0).stop 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 shutdown_connections 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0980a2c10 0x7ff0980a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0b01068b0 0x7ff0b019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0b01007d0 0x7ff0b0198630 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 --2- 172.21.15.103:0/3260060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0b00ffe30 0x7ff0b01980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:40.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 >> 172.21.15.103:0/3260060163 conn(0x7ff0b0076f60 msgr2=0x7ff0b0104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:40.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 shutdown_connections 2024-08-19T01:34:40.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:40.524+0000 7ff0b7fa8640 1 -- 172.21.15.103:0/3260060163 wait complete. 2024-08-19T01:34:41.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:40 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3260060163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:41.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:40 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3260060163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:41.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-19T01:33:23.991203+0000', 'modified': '2024-08-19T01:33:34.962843+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24485, 'mds_1': 24478, 'mds_2': 24466, 'mds_3': 24454, 'mds_4': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/2158943857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2158943857}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2158943857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14592': {'gid': 14592, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/1914348964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 1914348964}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 1914348964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/1682379250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 1682379250}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 1682379250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/1574351198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 1574351198}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 1574351198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6835/3609998196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 3609998196}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 3609998196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.146:6837/3498679319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 3498679319}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 3498679319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.190:6835/794893065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 794893065}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 794893065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.190:6837/66432346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 66432346}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 66432346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.190:6839/639745782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 639745782}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 639745782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.146:6839/1128555603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 1128555603}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 1128555603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24485, 'qdb_cluster': [24485, 24466, 24454, 24478, 24452]} 2024-08-19T01:34:41.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-19T01:34:41.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-19T01:34:41.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:40 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3260060163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:34:41.502 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 -- 172.21.15.103:0/1252524008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8102c20 msgr2=0x7f48e810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 --2- 172.21.15.103:0/1252524008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8102c20 0x7f48e810d770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009ca0 tx=0x7f48e002a950 comp rx=0 tx=0).stop 2024-08-19T01:34:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 -- 172.21.15.103:0/1252524008 shutdown_connections 2024-08-19T01:34:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 --2- 172.21.15.103:0/1252524008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8102c20 0x7f48e810d770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 --2- 172.21.15.103:0/1252524008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e81022a0 0x7f48e81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 --2- 172.21.15.103:0/1252524008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48e8100b50 0x7f48e8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 -- 172.21.15.103:0/1252524008 >> 172.21.15.103:0/1252524008 conn(0x7f48e80fc500 msgr2=0x7f48e80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 -- 172.21.15.103:0/1252524008 shutdown_connections 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 -- 172.21.15.103:0/1252524008 wait complete. 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 Processor -- start 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.011+0000 7f48eefaf640 1 -- start start 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48eefaf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8100b50 0x7f48e81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48eefaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e81022a0 0x7f48e8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48eefaf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48e8102c20 0x7f48e819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48eefaf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48e8110b60 con 0x7f48e8100b50 2024-08-19T01:34:42.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48eefaf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f48e81109e0 con 0x7f48e8102c20 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48eefaf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48e8110ce0 con 0x7f48e81022a0 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8100b50 0x7f48e81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8100b50 0x7f48e81980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43624/0 (socket says 172.21.15.103:43624) 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2591809547 learned_addr learned my addr 172.21.15.103:0/2591809547 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48e7fff640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e81022a0 0x7f48e8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ed54c640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48e8102c20 0x7f48e819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2591809547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e81022a0 msgr2=0x7f48e8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e81022a0 0x7f48e8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2591809547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48e8102c20 msgr2=0x7f48e819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48e8102c20 0x7f48e819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2591809547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e819d090 con 0x7f48e8100b50 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ed54c640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48e8102c20 0x7f48e819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:42.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8100b50 0x7f48e81980e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00d950 tx=0x7f48dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48e5ffb640 1 -- 172.21.15.103:0/2591809547 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc017030 con 0x7f48e8100b50 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e819d380 con 0x7f48e8100b50 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48e5ffb640 1 -- 172.21.15.103:0/2591809547 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48dc004020 con 0x7f48e8100b50 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.012+0000 7f48e5ffb640 1 -- 172.21.15.103:0/2591809547 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0047a0 con 0x7f48e8100b50 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.013+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e81a4c60 con 0x7f48e8100b50 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48e5ffb640 1 -- 172.21.15.103:0/2591809547 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f48dc0041d0 con 0x7f48e8100b50 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 Processor -- start 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 -- start start 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48e5ffb640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f48d80a2c10 0x7f48d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75b8810 0x5608e75bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608e7512710 con 0x5608e75adf50 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5608e74aa770 con 0x5608e75ad630 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48f13ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608e750e6d0 con 0x5608e75b8810 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.015+0000 7f48ecd4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43626/0 (socket says 172.21.15.103:43626) 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48e7fff640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f48d80a2c10 0x7f48d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2811693437 learned_addr learned my addr 172.21.15.103:0/2811693437 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ed54c640 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48e5ffb640 1 -- 172.21.15.103:0/2591809547 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f48dc0cec80 con 0x7f48e8100b50 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ed54c640 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75b8810 0x5608e75bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2811693437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75b8810 msgr2=0x5608e75bac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75b8810 0x5608e75bac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2811693437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 msgr2=0x5608e75ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 -- 172.21.15.103:0/2811693437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e74b13e0 con 0x5608e75adf50 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48e7fff640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f48d80a2c10 0x7f48d80a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f48e80ffd10 tx=0x7f48d4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75b82d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0cc5d0 tx=0x7f48dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e10a18bb3a7dfb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48d2ffd640 1 -- 172.21.15.103:0/2811693437 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc09bab0 con 0x5608e75adf50 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48d2ffd640 1 -- 172.21.15.103:0/2811693437 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48dc09b3b0 con 0x5608e75adf50 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.016+0000 7f48d2ffd640 1 -- 172.21.15.103:0/2811693437 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc0cc0c0 con 0x5608e75adf50 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 -- 172.21.15.103:0/2811693437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 msgr2=0x5608e75b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75b82d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0cc5d0 tx=0x7f48dc000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 -- 172.21.15.103:0/2811693437 shutdown_connections 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75b8810 0x5608e75bac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75b82d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2811693437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 -- 172.21.15.103:0/2811693437 >> 172.21.15.103:0/2811693437 conn(0x5608e75a9bc0 msgr2=0x5608e75a9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 -- 172.21.15.103:0/2811693437 shutdown_connections 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 -- 172.21.15.103:0/2811693437 wait complete. 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 -- start start 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.017+0000 7f48f13ad740 1 Processor -- start 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75b8810 0x5608e75b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48f13ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608e75bb380 con 0x5608e75b8810 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48f13ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5608e74cd970 con 0x5608e75ad630 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48f13ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608e75bb500 con 0x5608e75adf50 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48f13ad740 10 client.0 _ll_register_callbacks cb 0x5608e72217b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75b8810 0x5608e75b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48e7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75b8810 0x5608e75b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43634/0 (socket says 172.21.15.103:43634) 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 -- 172.21.15.103:0/2766732480 learned_addr learned my addr 172.21.15.103:0/2766732480 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 -- 172.21.15.103:0/2766732480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 msgr2=0x5608e75b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 -- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 msgr2=0x5608e75b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ed54c640 1 -- 172.21.15.103:0/2766732480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e72842b0 con 0x5608e75b8810 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.018+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48ed54c640 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75b8810 0x5608e75b6230 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48e002f3a0 tx=0x7f48e002f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200ba8f53ad4d9fd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:42.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48c7fff640 1 -- 172.21.15.103:0/2766732480 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0034040 con 0x5608e75b8810 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48c7fff640 1 -- 172.21.15.103:0/2766732480 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f48e0003b70 con 0x5608e75b8810 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48dc09b3b0 con 0x5608e75b8810 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48c7fff640 1 -- 172.21.15.103:0/2766732480 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f48e0003d70 con 0x5608e75b8810 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 10 client.14952 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608e75c7030 con 0x5608e75b8810 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48c7fff640 1 -- 172.21.15.103:0/2766732480 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48dc09b3b0 con 0x5608e75b8810 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48c7fff640 1 -- 172.21.15.103:0/2766732480 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f48e0046070 con 0x5608e75b8810 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 10 client.14952 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 10 client.14952 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 20 client.14952 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e75c96c0 0x5608e75cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 4 client.14952 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:42.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.019+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e751fc80 con 0x5608e75c96c0 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.020+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e75c96c0 0x5608e75cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.020+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e75c96c0 0x5608e75cab30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42b5d380d3cb2fab server_cookie=79837c57e8a4be53 in_seq=0 out_seq=0 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.020+0000 7f48c7fff640 10 client.14952 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48c7fff640 1 -- 172.21.15.103:0/2766732480 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608e751fc80 con 0x5608e75c96c0 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48c7fff640 10 client.14952 handle_command_reply: tid=0 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 client.14952 shutdown 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75b8810 msgr2=0x5608e75b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75b8810 0x5608e75b6230 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48e002f3a0 tx=0x7f48e002f3d0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 shutdown_connections 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e75c96c0 0x5608e75cab30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75b8810 0x5608e75b6230 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48c7fff640 0 client.14952 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 --2- 172.21.15.103:0/2766732480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75ad630 0x5608e75b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 >> 172.21.15.103:0/2766732480 conn(0x5608e75a9bc0 msgr2=0x5608e75a8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 shutdown_connections 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 1 -- 172.21.15.103:0/2766732480 wait complete. 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.021+0000 7f48f13ad740 20 client.14952 trim_cache size 0 max 16384 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.038+0000 7f48f13ad740 1 Processor -- start 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.038+0000 7f48f13ad740 1 -- start start 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75ad630 0x5608e75a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e75c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24452, 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 68.094555240000005, 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:34:42.064 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 103.162861947 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48f13ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608e75c8e70 con 0x5608e75adf50 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48f13ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5608e75c9540 con 0x5608e75b8810 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48f13ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608e75c9c10 con 0x5608e75ad630 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75d3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43638/0 (socket says 172.21.15.103:43638) 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 -- 172.21.15.103:0/360090325 learned_addr learned my addr 172.21.15.103:0/360090325 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75ad630 0x5608e75a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48ed54c640 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e75c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 -- 172.21.15.103:0/360090325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75ad630 msgr2=0x5608e75a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75ad630 0x5608e75a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 -- 172.21.15.103:0/360090325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 msgr2=0x5608e75c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e75c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 -- 172.21.15.103:0/360090325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e75a5130 con 0x5608e75adf50 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75ad630 0x5608e75a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48ed54c640 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e75c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48e7fff640 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75d3200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48d4011fa0 tx=0x7f48d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaa4f1463b3b0b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48d27fc640 1 -- 172.21.15.103:0/360090325 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4003cc0 con 0x5608e75adf50 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48d27fc640 1 -- 172.21.15.103:0/360090325 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48d40054d0 con 0x5608e75adf50 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.039+0000 7f48d27fc640 1 -- 172.21.15.103:0/360090325 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d401e420 con 0x5608e75adf50 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 -- 172.21.15.103:0/360090325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 msgr2=0x5608e75d3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75d3200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48d4011fa0 tx=0x7f48d4014000 comp rx=0 tx=0).stop 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 -- 172.21.15.103:0/360090325 shutdown_connections 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e75c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75adf50 0x5608e75d3200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 --2- 172.21.15.103:0/360090325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75ad630 0x5608e75a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 -- 172.21.15.103:0/360090325 >> 172.21.15.103:0/360090325 conn(0x5608e75a9bc0 msgr2=0x5608e75b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 -- 172.21.15.103:0/360090325 shutdown_connections 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 -- 172.21.15.103:0/360090325 wait complete. 2024-08-19T01:34:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 -- start start 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.040+0000 7f48f13ad740 1 Processor -- start 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75ad630 0x5608e75cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48f13ad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e7636e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48f13ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608e75ca9b0 con 0x5608e75ad630 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48f13ad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5608e75ca2e0 con 0x5608e75b8810 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48f13ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608e75cb080 con 0x5608e75adf50 2024-08-19T01:34:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48f13ad740 10 client.0 _ll_register_callbacks cb 0x7f48e0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75ad630 0x5608e75cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75ad630 0x5608e75cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43642/0 (socket says 172.21.15.103:43642) 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 -- 172.21.15.103:0/1595851127 learned_addr learned my addr 172.21.15.103:0/1595851127 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ed54c640 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e7636e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48e7fff640 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 -- 172.21.15.103:0/1595851127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 msgr2=0x5608e75cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 -- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 msgr2=0x5608e7636e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e7636e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 -- 172.21.15.103:0/1595851127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e75ccb60 con 0x5608e75ad630 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48ecd4b640 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75ad630 0x5608e75cbd30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48dc005170 tx=0x7f48dc0cefd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdd7a960dc186cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:42.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.041+0000 7f48c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48c7fff640 1 -- 172.21.15.103:0/1595851127 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc09b3b0 con 0x5608e75ad630 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48c7fff640 1 -- 172.21.15.103:0/1595851127 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f48dc0be830 con 0x5608e75ad630 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d40054d0 con 0x5608e75ad630 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48c7fff640 1 -- 172.21.15.103:0/1595851127 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f48dc083020 con 0x5608e75ad630 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 10 client.14958 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608e76383e0 con 0x5608e75ad630 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48c7fff640 1 -- 172.21.15.103:0/1595851127 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48d40054d0 con 0x5608e75ad630 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48c7fff640 1 -- 172.21.15.103:0/1595851127 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f48dc009a30 con 0x5608e75ad630 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 10 client.14958 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 10 client.14958 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 20 client.14958 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:42.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e763cbd0 0x5608e763b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 4 client.14958 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.042+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "status"}) -- 0x5608e751a190 con 0x5608e763cbd0 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48e7fff640 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e763cbd0 0x5608e763b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48e7fff640 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e763cbd0 0x5608e763b8e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd49f2c5ae9ea0a7 server_cookie=8883b2095a2e692c in_seq=0 out_seq=0 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48c7fff640 10 client.14958 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48c7fff640 1 -- 172.21.15.103:0/1595851127 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5608e751a190 con 0x5608e763cbd0 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48c7fff640 10 client.14958 handle_command_reply: tid=0 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 client.14958 shutdown 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75ad630 msgr2=0x5608e75cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75ad630 0x5608e75cbd30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48dc005170 tx=0x7f48dc0cefd0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 shutdown_connections 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5608e763cbd0 0x5608e763b8e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608e75b8810 0x5608e7636e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608e75adf50 0x5608e75cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48c7fff640 0 client.14958 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 --2- 172.21.15.103:0/1595851127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608e75ad630 0x5608e75cbd30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 >> 172.21.15.103:0/1595851127 conn(0x5608e75a9bc0 msgr2=0x5608e75a8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 shutdown_connections 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 1 -- 172.21.15.103:0/1595851127 wait complete. 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.043+0000 7f48f13ad740 20 client.14958 trim_cache size 0 max 16384 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f48d80a2c10 msgr2=0x7f48d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f48d80a2c10 0x7f48d80a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f48e80ffd10 tx=0x7f48d4005f90 comp rx=0 tx=0).stop 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8100b50 msgr2=0x7f48e81980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8100b50 0x7f48e81980e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00d950 tx=0x7f48dc00de10 comp rx=0 tx=0).stop 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 shutdown_connections 2024-08-19T01:34:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f48d80a2c10 0x7f48d80a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48e8102c20 0x7f48e819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48e81022a0 0x7f48e8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 --2- 172.21.15.103:0/2591809547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48e8100b50 0x7f48e81980e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:42.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 >> 172.21.15.103:0/2591809547 conn(0x7f48e80fc500 msgr2=0x7f48e8069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:42.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 shutdown_connections 2024-08-19T01:34:42.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:42.044+0000 7f48eefaf640 1 -- 172.21.15.103:0/2591809547 wait complete. 2024-08-19T01:34:42.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:41 smithi146 ceph-mon[22098]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:41 smithi103 ceph-mon[18436]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:42.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:41 smithi190 ceph-mon[22032]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:42.755 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.f status 2024-08-19T01:34:43.050 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:43.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:42 smithi103 ceph-mon[18436]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:43.280 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:42 smithi146 ceph-mon[22098]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:43.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:42 smithi190 ceph-mon[22032]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 -- 172.21.15.103:0/1774744283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581018b0 msgr2=0x7efc58101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/1774744283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581018b0 0x7efc58101d10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc40009930 tx=0x7efc4002a420 comp rx=0 tx=0).stop 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 -- 172.21.15.103:0/1774744283 shutdown_connections 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/1774744283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc58102250 0x7efc5810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/1774744283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581018b0 0x7efc58101d10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/1774744283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc581078b0 0x7efc58107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 -- 172.21.15.103:0/1774744283 >> 172.21.15.103:0/1774744283 conn(0x7efc580fd5f0 msgr2=0x7efc580ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 -- 172.21.15.103:0/1774744283 shutdown_connections 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.567+0000 7efc5cfb1640 1 -- 172.21.15.103:0/1774744283 wait complete. 2024-08-19T01:34:43.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 Processor -- start 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 -- start start 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc581018b0 0x7efc581981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc58102250 0x7efc581986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581078b0 0x7efc5819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc58111b80 con 0x7efc581078b0 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efc58111a00 con 0x7efc581018b0 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.568+0000 7efc5cfb1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efc58111d00 con 0x7efc58102250 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc58102250 0x7efc581986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc58102250 0x7efc581986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:48022/0 (socket says 172.21.15.103:48022) 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc4dd9b640 1 -- 172.21.15.103:0/984298463 learned_addr learned my addr 172.21.15.103:0/984298463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc5659c640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc581018b0 0x7efc581981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc56d9d640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581078b0 0x7efc5819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc56d9d640 1 -- 172.21.15.103:0/984298463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc58102250 msgr2=0x7efc581986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc56d9d640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc58102250 0x7efc581986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc56d9d640 1 -- 172.21.15.103:0/984298463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc581018b0 msgr2=0x7efc581981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc56d9d640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc581018b0 0x7efc581981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc56d9d640 1 -- 172.21.15.103:0/984298463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5819d150 con 0x7efc581078b0 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc56d9d640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581078b0 0x7efc5819ca70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efc5000be20 tx=0x7efc5000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc4ffff640 1 -- 172.21.15.103:0/984298463 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50019070 con 0x7efc581078b0 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc4ffff640 1 -- 172.21.15.103:0/984298463 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc500148f0 con 0x7efc581078b0 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5819d440 con 0x7efc581078b0 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc4ffff640 1 -- 172.21.15.103:0/984298463 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc500127f0 con 0x7efc581078b0 2024-08-19T01:34:43.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.569+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc581a4d20 con 0x7efc581078b0 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc4ffff640 1 -- 172.21.15.103:0/984298463 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efc5001c070 con 0x7efc581078b0 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 Processor -- start 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 -- start start 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bf630 0x5641ea2bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bff50 0x5641ea2ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5641ea224710 con 0x5641ea2ca810 2024-08-19T01:34:43.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5641ea1bc770 con 0x5641ea2bff50 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.572+0000 7efc5f3af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2206d0 con 0x5641ea2bf630 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43676/0 (socket says 172.21.15.103:43676) 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4dd9b640 1 -- 172.21.15.103:0/1997710046 learned_addr learned my addr 172.21.15.103:0/1997710046 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc5659c640 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bff50 0x5641ea2ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bf630 0x5641ea2bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4ffff640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efc440a2c10 0x7efc440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc5659c640 1 -- 172.21.15.103:0/1997710046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bf630 msgr2=0x5641ea2bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc5659c640 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bf630 0x5641ea2bfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc5659c640 1 -- 172.21.15.103:0/1997710046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 msgr2=0x5641ea2ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc5659c640 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc5659c640 1 -- 172.21.15.103:0/1997710046 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ea1c33e0 con 0x5641ea2bff50 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc56d9d640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efc440a2c10 0x7efc440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4ffff640 1 -- 172.21.15.103:0/984298463 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efc500ce8b0 con 0x7efc581078b0 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc5659c640 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bff50 0x5641ea2ca2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc48005fd0 tx=0x7efc480107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37dcb7de97e4be6a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4cd99640 1 -- 172.21.15.103:0/1997710046 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4801c480 con 0x5641ea2bff50 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4cd99640 1 -- 172.21.15.103:0/1997710046 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc4801cad0 con 0x5641ea2bff50 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc4cd99640 1 -- 172.21.15.103:0/1997710046 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48011b50 con 0x5641ea2bff50 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.573+0000 7efc56d9d640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efc440a2c10 0x7efc440a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc50097b70 tx=0x7efc5006e260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 -- 172.21.15.103:0/1997710046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bff50 msgr2=0x5641ea2ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bff50 0x5641ea2ca2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc48005fd0 tx=0x7efc480107a0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 -- 172.21.15.103:0/1997710046 shutdown_connections 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bff50 0x5641ea2ca2d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1997710046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bf630 0x5641ea2bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 -- 172.21.15.103:0/1997710046 >> 172.21.15.103:0/1997710046 conn(0x5641ea2bbbc0 msgr2=0x5641ea2bbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 -- 172.21.15.103:0/1997710046 shutdown_connections 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.574+0000 7efc5f3af740 1 -- 172.21.15.103:0/1997710046 wait complete. 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 -- start start 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 Processor -- start 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2d17f0 con 0x5641ea2ca810 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5641ea1df970 con 0x5641ea2bf630 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2d1970 con 0x5641ea2bff50 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5f3af740 10 client.0 _ll_register_callbacks cb 0x5641e9f337b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:43.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43692/0 (socket says 172.21.15.103:43692) 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 -- 172.21.15.103:0/1440633819 learned_addr learned my addr 172.21.15.103:0/1440633819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc5659c640 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 -- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 msgr2=0x5641ea2c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 -- 172.21.15.103:0/1440633819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 msgr2=0x5641ea2c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.575+0000 7efc4dd9b640 1 -- 172.21.15.103:0/1440633819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e9f962b0 con 0x5641ea2ca810 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c8200 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc40009a50 tx=0x7efc40032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388087081a40dd91 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc3dffb640 1 -- 172.21.15.103:0/1440633819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40035040 con 0x5641ea2ca810 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc3dffb640 1 -- 172.21.15.103:0/1440633819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efc40005020 con 0x5641ea2ca810 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc4801cad0 con 0x5641ea2ca810 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc3dffb640 1 -- 172.21.15.103:0/1440633819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc400318d0 con 0x5641ea2ca810 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 10 client.14970 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5641ea2dd480 con 0x5641ea2ca810 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc3dffb640 1 -- 172.21.15.103:0/1440633819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc4801cad0 con 0x5641ea2ca810 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc3dffb640 1 -- 172.21.15.103:0/1440633819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7efc400035f0 con 0x5641ea2ca810 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 10 client.14970 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 10 client.14970 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 20 client.14970 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea2e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 4 client.14970 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:43.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.576+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641ea231c80 con 0x5641ea2dfa30 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.577+0000 7efc5659c640 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea2e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.577+0000 7efc5659c640 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea2e0ea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=667c3fa5b47dc7b5 server_cookie=6d1e83dfa84f3547 in_seq=0 out_seq=0 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.577+0000 7efc3dffb640 10 client.14970 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc3dffb640 1 -- 172.21.15.103:0/1440633819 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641ea231c80 con 0x5641ea2dfa30 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc3dffb640 10 client.14970 handle_command_reply: tid=0 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 client.14970 shutdown 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 msgr2=0x5641ea2c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c8200 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efc40009a50 tx=0x7efc40032c30 comp rx=0 tx=0).stop 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 shutdown_connections 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea2e0ea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c8200 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc3dffb640 0 client.14970 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:34:43.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 --2- 172.21.15.103:0/1440633819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 >> 172.21.15.103:0/1440633819 conn(0x5641ea2bbbc0 msgr2=0x5641ea2bad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 shutdown_connections 2024-08-19T01:34:43.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 1 -- 172.21.15.103:0/1440633819 wait complete. 2024-08-19T01:34:43.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.578+0000 7efc5f3af740 20 client.14970 trim_cache size 0 max 16384 2024-08-19T01:34:43.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 Processor -- start 2024-08-19T01:34:43.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 -- start start 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2df1e0 con 0x5641ea2ca810 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2df8b0 con 0x5641ea2bf630 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5f3af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2dff80 con 0x5641ea2bff50 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43706/0 (socket says 172.21.15.103:43706) 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 -- 172.21.15.103:0/3562455554 learned_addr learned my addr 172.21.15.103:0/3562455554 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5659c640 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 -- 172.21.15.103:0/3562455554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 msgr2=0x5641ea2c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 -- 172.21.15.103:0/3562455554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 msgr2=0x5641ea2c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 -- 172.21.15.103:0/3562455554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ea2bf460 con 0x5641ea2ca810 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc5659c640 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c4f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efc40032d30 tx=0x7efc40041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fc4d24fe6b9b5f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.596+0000 7efc3ffff640 1 -- 172.21.15.103:0/3562455554 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40035070 con 0x5641ea2ca810 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc3ffff640 1 -- 172.21.15.103:0/3562455554 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc400035f0 con 0x5641ea2ca810 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc3ffff640 1 -- 172.21.15.103:0/3562455554 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40032310 con 0x5641ea2ca810 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 -- 172.21.15.103:0/3562455554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 msgr2=0x5641ea2c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c4f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efc40032d30 tx=0x7efc40041040 comp rx=0 tx=0).stop 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 -- 172.21.15.103:0/3562455554 shutdown_connections 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2c4f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 --2- 172.21.15.103:0/3562455554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 -- 172.21.15.103:0/3562455554 >> 172.21.15.103:0/3562455554 conn(0x5641ea2bbbc0 msgr2=0x5641ea2c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 -- 172.21.15.103:0/3562455554 shutdown_connections 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 -- 172.21.15.103:0/3562455554 wait complete. 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.597+0000 7efc5f3af740 1 -- start start 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 1 Processor -- start 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2e3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2e0d20 con 0x5641ea2ca810 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2e0650 con 0x5641ea2bf630 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5641ea2e13f0 con 0x5641ea2bff50 2024-08-19T01:34:43.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5f3af740 10 client.0 _ll_register_callbacks cb 0x7efc40031790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2e7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43722/0 (socket says 172.21.15.103:43722) 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 -- 172.21.15.103:0/851301900 learned_addr learned my addr 172.21.15.103:0/851301900 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5659c640 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 -- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 msgr2=0x5641ea2e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2e3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 -- 172.21.15.103:0/851301900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 msgr2=0x5641ea2e3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 -- 172.21.15.103:0/851301900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ea2b7130 con 0x5641ea2ca810 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc5659c640 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc4dd9b640 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2e7d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc40001eb0 tx=0x7efc400049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a080ec0e5e1c59d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.598+0000 7efc3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc3dffb640 1 -- 172.21.15.103:0/851301900 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40035040 con 0x5641ea2ca810 2024-08-19T01:34:43.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc3dffb640 1 -- 172.21.15.103:0/851301900 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efc40032090 con 0x5641ea2ca810 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc400035f0 con 0x5641ea2ca810 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc3dffb640 1 -- 172.21.15.103:0/851301900 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc40068020 con 0x5641ea2ca810 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 10 client.14976 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5641ea34a820 con 0x5641ea2ca810 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc3dffb640 1 -- 172.21.15.103:0/851301900 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc400035f0 con 0x5641ea2ca810 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc3dffb640 1 -- 172.21.15.103:0/851301900 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7efc400644b0 con 0x5641ea2ca810 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 10 client.14976 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 10 client.14976 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 20 client.14976 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea34de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 4 client.14976 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "status"}) -- 0x5641ea22c190 con 0x5641ea2dfa30 2024-08-19T01:34:43.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.599+0000 7efc5659c640 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea34de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:43.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc3dffb640 10 client.14976 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:34:43.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5659c640 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea34de00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a89f152925d2d301 server_cookie=587b1a74122062e3 in_seq=0 out_seq=0 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc3dffb640 1 -- 172.21.15.103:0/851301900 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5641ea22c190 con 0x5641ea2dfa30 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc3dffb640 10 client.14976 handle_command_reply: tid=0 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 client.14976 shutdown 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 msgr2=0x5641ea2e7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2e7d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc40001eb0 tx=0x7efc400049a0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 shutdown_connections 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5641ea2dfa30 0x5641ea34de00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5641ea2ca810 0x5641ea2e7d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5641ea2bff50 0x5641ea2e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 --2- 172.21.15.103:0/851301900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5641ea2bf630 0x5641ea2e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 >> 172.21.15.103:0/851301900 conn(0x5641ea2bbbc0 msgr2=0x5641ea2d9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc3dffb640 0 client.14976 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 shutdown_connections 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 1 -- 172.21.15.103:0/851301900 wait complete. 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.600+0000 7efc5f3af740 20 client.14976 trim_cache size 0 max 16384 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:34:43.609 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:34:43.610 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 70.659311036000005, 2024-08-19T01:34:43.610 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:34:43.610 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:43.610 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:43.610 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 110.12278878399999 2024-08-19T01:34:43.610 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efc440a2c10 msgr2=0x7efc440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efc440a2c10 0x7efc440a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc50097b70 tx=0x7efc5006e260 comp rx=0 tx=0).stop 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581078b0 msgr2=0x7efc5819ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581078b0 0x7efc5819ca70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efc5000be20 tx=0x7efc5000bf10 comp rx=0 tx=0).stop 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 shutdown_connections 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efc440a2c10 0x7efc440a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc581078b0 0x7efc5819ca70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc58102250 0x7efc581986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 --2- 172.21.15.103:0/984298463 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc581018b0 0x7efc581981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 >> 172.21.15.103:0/984298463 conn(0x7efc580fd5f0 msgr2=0x7efc580fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 shutdown_connections 2024-08-19T01:34:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:43.601+0000 7efc5cfb1640 1 -- 172.21.15.103:0/984298463 wait complete. 2024-08-19T01:34:44.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.c status 2024-08-19T01:34:44.582 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:45.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3939520626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 msgr2=0x7fbe1410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3939520626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe1410d770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009ca0 tx=0x7fbe0c02a950 comp rx=0 tx=0).stop 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3939520626 shutdown_connections 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3939520626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe1410d770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3939520626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe141022a0 0x7fbe141026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3939520626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14100b50 0x7fbe14100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3939520626 >> 172.21.15.103:0/3939520626 conn(0x7fbe140fc500 msgr2=0x7fbe140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3939520626 shutdown_connections 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.086+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3939520626 wait complete. 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 Processor -- start 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 -- start start 2024-08-19T01:34:45.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14100b50 0x7fbe14198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe141022a0 0x7fbe141994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe141a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14110b80 con 0x7fbe14102c20 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14110a00 con 0x7fbe141022a0 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe1b4ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14110d00 con 0x7fbe14100b50 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe141a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe141a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43744/0 (socket says 172.21.15.103:43744) 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 -- 172.21.15.103:0/3368605474 learned_addr learned my addr 172.21.15.103:0/3368605474 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe18a65640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe141022a0 0x7fbe141994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19266640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14100b50 0x7fbe14198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 -- 172.21.15.103:0/3368605474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14100b50 msgr2=0x7fbe14198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14100b50 0x7fbe14198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 -- 172.21.15.103:0/3368605474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe141022a0 msgr2=0x7fbe141994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe141022a0 0x7fbe141994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 -- 172.21.15.103:0/3368605474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe141a4550 con 0x7fbe14102c20 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.087+0000 7fbe19a67640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe141a4010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02ae70 tx=0x7fbe0c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.088+0000 7fbe067fc640 1 -- 172.21.15.103:0/3368605474 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c034040 con 0x7fbe14102c20 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.088+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141a4720 con 0x7fbe14102c20 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.088+0000 7fbe067fc640 1 -- 172.21.15.103:0/3368605474 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe0c0308f0 con 0x7fbe14102c20 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.088+0000 7fbe067fc640 1 -- 172.21.15.103:0/3368605474 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c004780 con 0x7fbe14102c20 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.088+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141a4b80 con 0x7fbe14102c20 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.090+0000 7fbe067fc640 1 -- 172.21.15.103:0/3368605474 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbe0c037020 con 0x7fbe14102c20 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 Processor -- start 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 -- start start 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3f988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fa76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3faa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616f3efd710 con 0x5616f3f98510 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5616f3e95770 con 0x5616f3fa7c20 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5616f3ef96d0 con 0x5616f3f98e30 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe18a65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3f988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe18a65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3f988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43750/0 (socket says 172.21.15.103:43750) 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe18a65640 1 -- 172.21.15.103:0/1740884029 learned_addr learned my addr 172.21.15.103:0/1740884029 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe067fc640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe000a2c10 0x7fbe000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe19266640 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fa76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.091+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3faa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe19a67640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe000a2c10 0x7fbe000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe18a65640 1 -- 172.21.15.103:0/1740884029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 msgr2=0x5616f3fa76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fa76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe18a65640 1 -- 172.21.15.103:0/1740884029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 msgr2=0x5616f3faa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3faa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe067fc640 1 -- 172.21.15.103:0/3368605474 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbe0c0f0140 con 0x7fbe14102c20 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe18a65640 1 -- 172.21.15.103:0/1740884029 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f3e9c3e0 con 0x5616f3f98510 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3f988f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbe100058a0 tx=0x7fbe1000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864e9a1f0b98bf45 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbe19a67640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe000a2c10 0x7fbe000a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0b8de0 tx=0x7fbe0c08e990 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbdff7fe640 1 -- 172.21.15.103:0/1740884029 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100120d0 con 0x5616f3f98510 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbdff7fe640 1 -- 172.21.15.103:0/1740884029 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe10005020 con 0x5616f3f98510 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.092+0000 7fbdff7fe640 1 -- 172.21.15.103:0/1740884029 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100049e0 con 0x5616f3f98510 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1740884029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 msgr2=0x5616f3f988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3f988f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbe100058a0 tx=0x7fbe1000b840 comp rx=0 tx=0).stop 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1740884029 shutdown_connections 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3faa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fa76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1740884029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3f988f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1740884029 >> 172.21.15.103:0/1740884029 conn(0x5616f3f94bc0 msgr2=0x5616f3f94fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1740884029 shutdown_connections 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1740884029 wait complete. 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 -- start start 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.093+0000 7fbe1d8c8740 1 Processor -- start 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 0x5616f3fa4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98e30 0x5616f3fa5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616f3faa7f0 con 0x5616f3f98e30 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5616f3eb8970 con 0x5616f3f98510 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5616f3faa970 con 0x5616f3fa7c20 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 10 client.0 _ll_register_callbacks cb 0x5616f3c0c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:45.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98e30 0x5616f3fa5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98e30 0x5616f3fa5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43760/0 (socket says 172.21.15.103:43760) 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 -- 172.21.15.103:0/1747758313 learned_addr learned my addr 172.21.15.103:0/1747758313 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 0x5616f3fa4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 -- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3fa7c20 msgr2=0x5616f3fa5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 -- 172.21.15.103:0/1747758313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 msgr2=0x5616f3fa4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 0x5616f3fa4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 -- 172.21.15.103:0/1747758313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f3c6f2b0 con 0x5616f3f98e30 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe19266640 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98e30 0x5616f3fa5000 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbe08010500 tx=0x7fbe080109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedb9fb66e3a5566 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbdfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbdfcff9640 1 -- 172.21.15.103:0/1747758313 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0801a020 con 0x5616f3f98e30 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbdfcff9640 1 -- 172.21.15.103:0/1747758313 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbe08016690 con 0x5616f3f98e30 2024-08-19T01:34:45.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe10005020 con 0x5616f3f98e30 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbdfcff9640 1 -- 172.21.15.103:0/1747758313 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe08016890 con 0x5616f3f98e30 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.094+0000 7fbe1d8c8740 10 client.14988 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5616f3fb6490 con 0x5616f3f98e30 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbdfcff9640 1 -- 172.21.15.103:0/1747758313 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe10005020 con 0x5616f3f98e30 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbdfcff9640 1 -- 172.21.15.103:0/1747758313 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbe08014c00 con 0x5616f3f98e30 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe1d8c8740 10 client.14988 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe1d8c8740 10 client.14988 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe1d8c8740 20 client.14988 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f3fb8b30 0x5616f3fb9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe1d8c8740 4 client.14988 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616f3f0ac80 con 0x5616f3fb8b30 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.095+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f3fb8b30 0x5616f3fb9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.096+0000 7fbe18a65640 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f3fb8b30 0x5616f3fb9fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c55029350e5f081 server_cookie=81bcd50b219c86cd in_seq=0 out_seq=0 2024-08-19T01:34:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.096+0000 7fbdfcff9640 10 client.14988 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbdfcff9640 1 -- 172.21.15.103:0/1747758313 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616f3f0ac80 con 0x5616f3fb8b30 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbdfcff9640 10 client.14988 handle_command_reply: tid=0 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 client.14988 shutdown 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98e30 msgr2=0x5616f3fa5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98e30 0x5616f3fa5000 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbe08010500 tx=0x7fbe080109c0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 shutdown_connections 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f3fb8b30 0x5616f3fb9fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98e30 0x5616f3fa5000 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbdfcff9640 0 client.14988 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/1747758313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 0x5616f3fa4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 >> 172.21.15.103:0/1747758313 conn(0x5616f3f94bc0 msgr2=0x5616f3f9a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 shutdown_connections 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/1747758313 wait complete. 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.097+0000 7fbe1d8c8740 20 client.14988 trim_cache size 0 max 16384 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 Processor -- start 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 -- start start 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 0x5616f3fb98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fa3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616f3fb82e0 con 0x5616f3fa7c20 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5616f3fb89b0 con 0x5616f3f98510 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5616f3fb9080 con 0x5616f3f98e30 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43768/0 (socket says 172.21.15.103:43768) 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 -- 172.21.15.103:0/3511003105 learned_addr learned my addr 172.21.15.103:0/3511003105 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:45.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 -- 172.21.15.103:0/3511003105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 msgr2=0x5616f3fa3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 --2- 172.21.15.103:0/3511003105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fa3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 -- 172.21.15.103:0/3511003105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 msgr2=0x5616f3fb98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 --2- 172.21.15.103:0/3511003105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 0x5616f3fb98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.115+0000 7fbe18a65640 1 -- 172.21.15.103:0/3511003105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f3f95b20 con 0x5616f3fa7c20 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe18a65640 1 --2- 172.21.15.103:0/3511003105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbe100057c0 tx=0x7fbe10005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da26e472f78a36 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbdfeffd640 1 -- 172.21.15.103:0/3511003105 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000e720 con 0x5616f3fa7c20 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbdfeffd640 1 -- 172.21.15.103:0/3511003105 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe1000ed70 con 0x5616f3fa7c20 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbdfeffd640 1 -- 172.21.15.103:0/3511003105 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100039e0 con 0x5616f3fa7c20 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/3511003105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3fa7c20 msgr2=0x5616f3fa5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/3511003105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbe100057c0 tx=0x7fbe10005750 comp rx=0 tx=0).stop 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/3511003105 shutdown_connections 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/3511003105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3fa7c20 0x5616f3fa5390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/3511003105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fa3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/3511003105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3f98510 0x5616f3fb98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/3511003105 >> 172.21.15.103:0/3511003105 conn(0x5616f3f94bc0 msgr2=0x5616f3fa3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/3511003105 shutdown_connections 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.116+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/3511003105 wait complete. 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 -- start start 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 Processor -- start 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3fc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fbbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3fbc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616f3fb9e20 con 0x5616f3f98510 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5616f3fb9750 con 0x5616f3fa7c20 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5616f3fba4f0 con 0x5616f3f98e30 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe1d8c8740 10 client.0 _ll_register_callbacks cb 0x7fbe0800b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe18a65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3fc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe18a65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3fc2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43780/0 (socket says 172.21.15.103:43780) 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe18a65640 1 -- 172.21.15.103:0/2248860441 learned_addr learned my addr 172.21.15.103:0/2248860441 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe19266640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fbbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.117+0000 7fbe18a65640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3fbc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe18a65640 1 -- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 msgr2=0x5616f3fbbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe18a65640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fbbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe18a65640 1 -- 172.21.15.103:0/2248860441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 msgr2=0x5616f3fbc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe18a65640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3fbc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe18a65640 1 -- 172.21.15.103:0/2248860441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f3fbcab0 con 0x5616f3f98510 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe19266640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fbbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe18a65640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3fc2a10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005d00 tx=0x7fbe10001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f49644ef98dca5b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbdfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbdfcff9640 1 -- 172.21.15.103:0/2248860441 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10005310 con 0x5616f3f98510 2024-08-19T01:34:45.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbdfcff9640 1 -- 172.21.15.103:0/2248860441 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbe100054b0 con 0x5616f3f98510 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1000ed70 con 0x5616f3f98510 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbdfcff9640 1 -- 172.21.15.103:0/2248860441 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe1000e720 con 0x5616f3f98510 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 10 client.14994 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5616f40232a0 con 0x5616f3f98510 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbdfcff9640 1 -- 172.21.15.103:0/2248860441 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe1000ed70 con 0x5616f3f98510 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-08-19T01:34:45.139 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 73.184483337000003, 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 108.911452814 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:34:45.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbdfcff9640 1 -- 172.21.15.103:0/2248860441 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbe10033030 con 0x5616f3f98510 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 10 client.14994 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 10 client.14994 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 20 client.14994 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f40276c0 0x5616f40263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 4 client.14994 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.118+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "status"}) -- 0x5616f3f05190 con 0x5616f40276c0 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.119+0000 7fbe19266640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f40276c0 0x5616f40263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.119+0000 7fbe19266640 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f40276c0 0x5616f40263d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b5c2eb3b48c0a29 server_cookie=516c4561f0d09bb in_seq=0 out_seq=0 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.119+0000 7fbdfcff9640 10 client.14994 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.119+0000 7fbdfcff9640 1 -- 172.21.15.103:0/2248860441 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5616f3f05190 con 0x5616f40276c0 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.119+0000 7fbdfcff9640 10 client.14994 handle_command_reply: tid=0 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 client.14994 shutdown 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 msgr2=0x5616f3fc2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3fc2a10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005d00 tx=0x7fbe10001e50 comp rx=0 tx=0).stop 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 shutdown_connections 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5616f40276c0 0x5616f40263d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5616f3fa7c20 0x5616f3fbc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbdfcff9640 0 client.14994 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5616f3f98e30 0x5616f3fbbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 --2- 172.21.15.103:0/2248860441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616f3f98510 0x5616f3fc2a10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 >> 172.21.15.103:0/2248860441 conn(0x5616f3f94bc0 msgr2=0x5616f3fa3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 shutdown_connections 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 1 -- 172.21.15.103:0/2248860441 wait complete. 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1d8c8740 20 client.14994 trim_cache size 0 max 16384 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe000a2c10 msgr2=0x7fbe000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe000a2c10 0x7fbe000a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0b8de0 tx=0x7fbe0c08e990 comp rx=0 tx=0).stop 2024-08-19T01:34:45.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 msgr2=0x7fbe141a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe141a4010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02ae70 tx=0x7fbe0c02f5d0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 shutdown_connections 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe000a2c10 0x7fbe000a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe14102c20 0x7fbe141a4010 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe141022a0 0x7fbe141994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 --2- 172.21.15.103:0/3368605474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14100b50 0x7fbe14198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.120+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 >> 172.21.15.103:0/3368605474 conn(0x7fbe140fc500 msgr2=0x7fbe140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.121+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 shutdown_connections 2024-08-19T01:34:45.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:45.121+0000 7fbe1b4ca640 1 -- 172.21.15.103:0/3368605474 wait complete. 2024-08-19T01:34:45.894 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-19T01:34:46.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:45 smithi190 ceph-mon[22032]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:46.149 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:45 smithi146 ceph-mon[22098]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:46.271 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:45 smithi103 ceph-mon[18436]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 -- 172.21.15.103:0/39727584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 msgr2=0x7feefc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 --2- 172.21.15.103:0/39727584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc101d10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feee8009930 tx=0x7feee802a420 comp rx=0 tx=0).stop 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 -- 172.21.15.103:0/39727584 shutdown_connections 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 --2- 172.21.15.103:0/39727584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc102250 0x7feefc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 --2- 172.21.15.103:0/39727584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc101d10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 --2- 172.21.15.103:0/39727584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feefc1078b0 0x7feefc107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 -- 172.21.15.103:0/39727584 >> 172.21.15.103:0/39727584 conn(0x7feefc0fd5f0 msgr2=0x7feefc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 -- 172.21.15.103:0/39727584 shutdown_connections 2024-08-19T01:34:46.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 -- 172.21.15.103:0/39727584 wait complete. 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 Processor -- start 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.647+0000 7fef02e79640 1 -- start start 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef02e79640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef02e79640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feefc102250 0x7feefc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef02e79640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc1078b0 0x7feefc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef02e79640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feefc111c00 con 0x7feefc1018b0 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef02e79640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feefc111a80 con 0x7feefc102250 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef02e79640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feefc111d80 con 0x7feefc1078b0 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43802/0 (socket says 172.21.15.103:43802) 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 -- 172.21.15.103:0/2855886400 learned_addr learned my addr 172.21.15.103:0/2855886400 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:46.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7feefbfff640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feefc102250 0x7feefc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 -- 172.21.15.103:0/2855886400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc1078b0 msgr2=0x7feefc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc1078b0 0x7feefc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 -- 172.21.15.103:0/2855886400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feefc102250 msgr2=0x7feefc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feefc102250 0x7feefc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 -- 172.21.15.103:0/2855886400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc19f410 con 0x7feefc1018b0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef00c15640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc19a460 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feef000b4a0 tx=0x7feef000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7feef9ffb640 1 -- 172.21.15.103:0/2855886400 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0014070 con 0x7feefc1018b0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7feef9ffb640 1 -- 172.21.15.103:0/2855886400 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feef0010ac0 con 0x7feefc1018b0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc19f700 con 0x7feefc1018b0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.648+0000 7feef9ffb640 1 -- 172.21.15.103:0/2855886400 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0017c30 con 0x7feefc1018b0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.649+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc1a6fe0 con 0x7feefc1018b0 2024-08-19T01:34:46.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.650+0000 7feef9ffb640 1 -- 172.21.15.103:0/2855886400 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7feef0017070 con 0x7feefc1018b0 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7feef9ffb640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feeec0a2c10 0x7feeec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 Processor -- start 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7feefbfff640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feeec0a2c10 0x7feeec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 -- start start 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7feef9ffb640 1 -- 172.21.15.103:0/2855886400 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7feef00ce440 con 0x7feefc1018b0 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e605a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605f50 0x55636e6102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e612c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55636e56a710 con 0x55636e610810 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55636e502770 con 0x55636e605630 2024-08-19T01:34:46.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.651+0000 7fef05277740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55636e5666d0 con 0x55636e605f50 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e612c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e612c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43812/0 (socket says 172.21.15.103:43812) 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 -- 172.21.15.103:0/3894153222 learned_addr learned my addr 172.21.15.103:0/3894153222 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7feefbfff640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feeec0a2c10 0x7feeec0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feee8006010 tx=0x7feee8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef00c15640 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e605a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 -- 172.21.15.103:0/3894153222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605f50 msgr2=0x55636e6102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605f50 0x55636e6102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 -- 172.21.15.103:0/3894153222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 msgr2=0x55636e605a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e605a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 -- 172.21.15.103:0/3894153222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636e5093e0 con 0x55636e610810 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef00c15640 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e605a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7fef01416640 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e612c00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feef4009d00 tx=0x7feef400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51156ef9684b52 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7feee6ffd640 1 -- 172.21.15.103:0/3894153222 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef400bd40 con 0x55636e610810 2024-08-19T01:34:46.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7feee6ffd640 1 -- 172.21.15.103:0/3894153222 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feef4017070 con 0x55636e610810 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.652+0000 7feee6ffd640 1 -- 172.21.15.103:0/3894153222 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40115f0 con 0x55636e610810 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 -- 172.21.15.103:0/3894153222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 msgr2=0x55636e612c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e612c00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feef4009d00 tx=0x7feef400b420 comp rx=0 tx=0).stop 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 -- 172.21.15.103:0/3894153222 shutdown_connections 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e612c00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605f50 0x55636e6102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 --2- 172.21.15.103:0/3894153222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e605a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 -- 172.21.15.103:0/3894153222 >> 172.21.15.103:0/3894153222 conn(0x55636e601bc0 msgr2=0x55636e601fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 -- 172.21.15.103:0/3894153222 shutdown_connections 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 -- 172.21.15.103:0/3894153222 wait complete. 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 -- start start 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.653+0000 7fef05277740 1 Processor -- start 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef05277740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605630 0x55636e60d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef05277740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605f50 0x55636e60dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef05277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e60e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef05277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55636e6133e0 con 0x55636e610810 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef05277740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55636e525970 con 0x55636e605f50 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef05277740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55636e613560 con 0x55636e605630 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef05277740 10 client.0 _ll_register_callbacks cb 0x55636e2797b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e60e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e60e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43824/0 (socket says 172.21.15.103:43824) 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 -- 172.21.15.103:0/4168897505 learned_addr learned my addr 172.21.15.103:0/4168897505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef00c15640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605630 0x55636e60d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7feefbfff640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605f50 0x55636e60dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 -- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605630 msgr2=0x55636e60d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605630 0x55636e60d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 -- 172.21.15.103:0/4168897505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605f50 msgr2=0x55636e60dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605f50 0x55636e60dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 -- 172.21.15.103:0/4168897505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636e2dc2b0 con 0x55636e610810 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7feefbfff640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605f50 0x55636e60dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7fef01416640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e60e140 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feef4012ef0 tx=0x7feef4007210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84bcaac0ea26a8b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.654+0000 7feedbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7feedbfff640 1 -- 172.21.15.103:0/4168897505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4021070 con 0x55636e610810 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7feedbfff640 1 -- 172.21.15.103:0/4168897505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7feef401b3f0 con 0x55636e610810 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef4017070 con 0x55636e610810 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7feedbfff640 1 -- 172.21.15.103:0/4168897505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feef401b630 con 0x55636e610810 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 10 client.15006 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55636e61efb0 con 0x55636e610810 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7feedbfff640 1 -- 172.21.15.103:0/4168897505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feef4017070 con 0x55636e610810 2024-08-19T01:34:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7feedbfff640 1 -- 172.21.15.103:0/4168897505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7feef4011070 con 0x55636e610810 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 10 client.15006 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 10 client.15006 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 20 client.15006 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e6213b0 0x55636e622820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 4 client.15006 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.655+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55636e577c80 con 0x55636e6213b0 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.656+0000 7fef00c15640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e6213b0 0x55636e622820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.656+0000 7feedbfff640 10 client.15006 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.656+0000 7fef00c15640 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e6213b0 0x55636e622820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93ea37969c287845 server_cookie=e75c54c9a0175e33 in_seq=0 out_seq=0 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7feedbfff640 1 -- 172.21.15.103:0/4168897505 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55636e577c80 con 0x55636e6213b0 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7feedbfff640 10 client.15006 handle_command_reply: tid=0 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 client.15006 shutdown 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 msgr2=0x55636e60e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e60e140 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7feef4012ef0 tx=0x7feef4007210 comp rx=0 tx=0).stop 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 shutdown_connections 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e6213b0 0x55636e622820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e610810 0x55636e60e140 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605f50 0x55636e60dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7feedbfff640 0 client.15006 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 --2- 172.21.15.103:0/4168897505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e605630 0x55636e60d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 >> 172.21.15.103:0/4168897505 conn(0x55636e601bc0 msgr2=0x55636e600d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 shutdown_connections 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 1 -- 172.21.15.103:0/4168897505 wait complete. 2024-08-19T01:34:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.657+0000 7fef05277740 20 client.15006 trim_cache size 0 max 16384 2024-08-19T01:34:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 Processor -- start 2024-08-19T01:34:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 -- start start 2024-08-19T01:34:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e608090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e60a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e60ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55636e620b60 con 0x55636e605f50 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55636e621230 con 0x55636e605630 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef05277740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55636e621900 con 0x55636e601bc0 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e60ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e60ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43828/0 (socket says 172.21.15.103:43828) 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef01416640 1 -- 172.21.15.103:0/374430728 learned_addr learned my addr 172.21.15.103:0/374430728 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7fef00c15640 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e608090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.675+0000 7feefbfff640 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e60a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7fef01416640 1 -- 172.21.15.103:0/374430728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 msgr2=0x55636e608090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7fef01416640 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e608090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7fef01416640 1 -- 172.21.15.103:0/374430728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 msgr2=0x55636e60a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7fef01416640 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e60a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7fef01416640 1 -- 172.21.15.103:0/374430728 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636e5fd130 con 0x55636e605f50 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7fef01416640 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e60ae80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feef40095b0 tx=0x7feef4009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62415d46d9a1b6e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7feee67fc640 1 -- 172.21.15.103:0/374430728 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4035070 con 0x55636e605f50 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7feee67fc640 1 -- 172.21.15.103:0/374430728 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feef4017040 con 0x55636e605f50 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7feee67fc640 1 -- 172.21.15.103:0/374430728 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4021070 con 0x55636e605f50 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.676+0000 7fef05277740 1 -- 172.21.15.103:0/374430728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 msgr2=0x55636e60ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e60ae80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feef40095b0 tx=0x7feef4009690 comp rx=0 tx=0).stop 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- 172.21.15.103:0/374430728 shutdown_connections 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e60ae80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e60a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 --2- 172.21.15.103:0/374430728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e608090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- 172.21.15.103:0/374430728 >> 172.21.15.103:0/374430728 conn(0x55636e620780 msgr2=0x55636e60f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- 172.21.15.103:0/374430728 shutdown_connections 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- 172.21.15.103:0/374430728 wait complete. 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- start start 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 Processor -- start 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e6268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e626e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e627360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55636e6226a0 con 0x55636e605f50 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55636e621fd0 con 0x55636e605630 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.677+0000 7fef05277740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55636e622d70 con 0x55636e601bc0 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef05277740 10 client.0 _ll_register_callbacks cb 0x7feef4009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e627360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e627360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43840/0 (socket says 172.21.15.103:43840) 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 -- 172.21.15.103:0/2249041726 learned_addr learned my addr 172.21.15.103:0/2249041726 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef00c15640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e6268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7feefbfff640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e626e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 -- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 msgr2=0x55636e6268e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e6268e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 -- 172.21.15.103:0/2249041726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 msgr2=0x55636e626e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e626e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 -- 172.21.15.103:0/2249041726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636e68ab70 con 0x55636e605f50 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7feefbfff640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e626e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef01416640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e627360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feef4004c00 tx=0x7feef4025ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dba9a6e34f8a698 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7feedbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7feedbfff640 1 -- 172.21.15.103:0/2249041726 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4035070 con 0x55636e605f50 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7feedbfff640 1 -- 172.21.15.103:0/2249041726 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7feef401b7d0 con 0x55636e605f50 2024-08-19T01:34:46.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef4017040 con 0x55636e605f50 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7feedbfff640 1 -- 172.21.15.103:0/2249041726 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feef4040070 con 0x55636e605f50 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef05277740 10 client.15012 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55636e68be50 con 0x55636e605f50 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.678+0000 7feedbfff640 1 -- 172.21.15.103:0/2249041726 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feef4017040 con 0x55636e605f50 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7feedbfff640 1 -- 172.21.15.103:0/2249041726 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7feef4011050 con 0x55636e605f50 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef05277740 10 client.15012 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef05277740 10 client.15012 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef05277740 20 client.15012 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef05277740 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e610810 0x55636e68f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef05277740 4 client.15012 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "status"}) -- 0x55636e572190 con 0x55636e610810 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef00c15640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e610810 0x55636e68f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7fef00c15640 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e610810 0x55636e68f470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cddabfd8f16ae35f server_cookie=d509357a4b0a8fec in_seq=0 out_seq=0 2024-08-19T01:34:46.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.679+0000 7feedbfff640 10 client.15012 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7feedbfff640 1 -- 172.21.15.103:0/2249041726 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55636e572190 con 0x55636e610810 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7feedbfff640 10 client.15012 handle_command_reply: tid=0 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 client.15012 shutdown 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 msgr2=0x55636e627360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e627360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feef4004c00 tx=0x7feef4025ba0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 shutdown_connections 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55636e610810 0x55636e68f470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55636e605f50 0x55636e627360 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7feedbfff640 0 client.15012 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55636e605630 0x55636e626e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 --2- 172.21.15.103:0/2249041726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55636e601bc0 0x55636e6268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 >> 172.21.15.103:0/2249041726 conn(0x55636e620780 msgr2=0x55636e600520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 shutdown_connections 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 1 -- 172.21.15.103:0/2249041726 wait complete. 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.680+0000 7fef05277740 20 client.15012 trim_cache size 0 max 16384 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24478, 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:34:46.690 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:34:46.691 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:34:46.691 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 75.752714835999996, 2024-08-19T01:34:46.691 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:34:46.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:46.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:46.691 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 105.123357035 2024-08-19T01:34:46.691 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feeec0a2c10 msgr2=0x7feeec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feeec0a2c10 0x7feeec0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feee8006010 tx=0x7feee8000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 msgr2=0x7feefc19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc19a460 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feef000b4a0 tx=0x7feef000b960 comp rx=0 tx=0).stop 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 shutdown_connections 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feeec0a2c10 0x7feeec0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc1078b0 0x7feefc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feefc102250 0x7feefc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 --2- 172.21.15.103:0/2855886400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feefc1018b0 0x7feefc19a460 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 >> 172.21.15.103:0/2855886400 conn(0x7feefc0fd5f0 msgr2=0x7feefc10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 shutdown_connections 2024-08-19T01:34:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:46.681+0000 7fef02e79640 1 -- 172.21.15.103:0/2855886400 wait complete. 2024-08-19T01:34:47.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-19T01:34:47.801 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:48.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:47 smithi103 ceph-mon[18436]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:48.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:47 smithi190 ceph-mon[22032]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:47 smithi146 ceph-mon[22098]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:48.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 -- 172.21.15.103:0/2505618864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0294101150 msgr2=0x7f029410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2505618864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0294101150 0x7f029410d680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0284009930 tx=0x7f028402a420 comp rx=0 tx=0).stop 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 -- 172.21.15.103:0/2505618864 shutdown_connections 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2505618864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0294101150 0x7f029410d680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2505618864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02941007b0 0x7f0294100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2505618864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02941067b0 0x7f0294106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 -- 172.21.15.103:0/2505618864 >> 172.21.15.103:0/2505618864 conn(0x7f02940fc4d0 msgr2=0x7f02940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 -- 172.21.15.103:0/2505618864 shutdown_connections 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 -- 172.21.15.103:0/2505618864 wait complete. 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 Processor -- start 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.298+0000 7f0298ba3640 1 -- start start 2024-08-19T01:34:48.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0298ba3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02941007b0 0x7f0294196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0298ba3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0294101150 0x7f02941974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0298ba3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02941067b0 0x7f029419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0298ba3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0294110a60 con 0x7f02941067b0 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0298ba3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02941108e0 con 0x7f02941007b0 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0298ba3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0294110be0 con 0x7f0294101150 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02941067b0 0x7f029419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02941067b0 0x7f029419b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43870/0 (socket says 172.21.15.103:43870) 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 -- 172.21.15.103:0/2117055016 learned_addr learned my addr 172.21.15.103:0/2117055016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f029259c640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02941007b0 0x7f0294196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0291d9b640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0294101150 0x7f02941974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 -- 172.21.15.103:0/2117055016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0294101150 msgr2=0x7f02941974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0294101150 0x7f02941974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 -- 172.21.15.103:0/2117055016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02941007b0 msgr2=0x7f0294196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02941007b0 0x7f0294196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 -- 172.21.15.103:0/2117055016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029419bf50 con 0x7f02941067b0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0292d9d640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02941067b0 0x7f029419b870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0284005940 tx=0x7f0284004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f02837fe640 1 -- 172.21.15.103:0/2117055016 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028403f030 con 0x7f02941067b0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029419c1e0 con 0x7f02941067b0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f02837fe640 1 -- 172.21.15.103:0/2117055016 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0284003aa0 con 0x7f02941067b0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.299+0000 7f02837fe640 1 -- 172.21.15.103:0/2117055016 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284035070 con 0x7f02941067b0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.300+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941a3b20 con 0x7f02941067b0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.302+0000 7f02837fe640 1 -- 172.21.15.103:0/2117055016 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0284042030 con 0x7f02941067b0 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.302+0000 7f029afa1740 1 Processor -- start 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.302+0000 7f029afa1740 1 -- start start 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029afa1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea327674b0 0x55ea32767890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029afa1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea327744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029afa1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32776e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029afa1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea326cc710 con 0x55ea32774a20 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029afa1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ea32664770 con 0x55ea327674b0 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029afa1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ea326c86d0 con 0x55ea32767dd0 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f02837fe640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f027c0a2c10 0x7f027c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32776e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0292d9d640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f027c0a2c10 0x7f027c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32776e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43878/0 (socket says 172.21.15.103:43878) 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 -- 172.21.15.103:0/517104144 learned_addr learned my addr 172.21.15.103:0/517104144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea327674b0 0x55ea32767890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f02837fe640 1 -- 172.21.15.103:0/2117055016 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f02840ef920 con 0x7f02941067b0 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029259c640 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea327744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 -- 172.21.15.103:0/517104144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 msgr2=0x55ea327744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea327744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 -- 172.21.15.103:0/517104144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea327674b0 msgr2=0x55ea32767890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea327674b0 0x55ea32767890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 -- 172.21.15.103:0/517104144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3266b3e0 con 0x55ea32774a20 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f029259c640 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea327744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0292d9d640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f027c0a2c10 0x7f027c0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f02840bbd50 tx=0x7f0284087170 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0291d9b640 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32776e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0288012840 tx=0x7f0288009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7902545aabd8b7ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0280ff9640 1 -- 172.21.15.103:0/517104144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288010a70 con 0x55ea32774a20 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0280ff9640 1 -- 172.21.15.103:0/517104144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0288010c10 con 0x55ea32774a20 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.303+0000 7f0280ff9640 1 -- 172.21.15.103:0/517104144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028800c390 con 0x55ea32774a20 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 -- 172.21.15.103:0/517104144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 msgr2=0x55ea32776e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32776e10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0288012840 tx=0x7f0288009960 comp rx=0 tx=0).stop 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 -- 172.21.15.103:0/517104144 shutdown_connections 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32776e10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea327744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 --2- 172.21.15.103:0/517104144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea327674b0 0x55ea32767890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 -- 172.21.15.103:0/517104144 >> 172.21.15.103:0/517104144 conn(0x55ea32763bc0 msgr2=0x55ea32763fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 -- 172.21.15.103:0/517104144 shutdown_connections 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.304+0000 7f029afa1740 1 -- 172.21.15.103:0/517104144 wait complete. 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 -- start start 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 Processor -- start 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea327719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32771ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32772420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea32777650 con 0x55ea32774a20 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ea32687970 con 0x55ea32767dd0 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ea327777d0 con 0x55ea327674b0 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029afa1740 10 client.0 _ll_register_callbacks cb 0x55ea323db7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:48.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32772420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32772420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43894/0 (socket says 172.21.15.103:43894) 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 -- 172.21.15.103:0/3821347599 learned_addr learned my addr 172.21.15.103:0/3821347599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f029259c640 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32771ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea327719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 -- 172.21.15.103:0/3821347599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 msgr2=0x55ea327719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea327719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 -- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 msgr2=0x55ea32771ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32771ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.305+0000 7f0291d9b640 1 -- 172.21.15.103:0/3821347599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3243e2b0 con 0x55ea32774a20 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f0291d9b640 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32772420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0288002230 tx=0x7f028801dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9459856cb129bc39 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f0279ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f0279ffb640 1 -- 172.21.15.103:0/3821347599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288004a40 con 0x55ea32774a20 2024-08-19T01:34:48.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f0279ffb640 1 -- 172.21.15.103:0/3821347599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0288004be0 con 0x55ea32774a20 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0288010c10 con 0x55ea32774a20 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f0279ffb640 1 -- 172.21.15.103:0/3821347599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f028801d460 con 0x55ea32774a20 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 10 client.15024 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea32783290 con 0x55ea32774a20 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f0279ffb640 1 -- 172.21.15.103:0/3821347599 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0288010c10 con 0x55ea32774a20 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f0279ffb640 1 -- 172.21.15.103:0/3821347599 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f028800c070 con 0x55ea32774a20 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 10 client.15024 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 10 client.15024 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 20 client.15024 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea32785730 0x55ea32786ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 4 client.15024 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.306+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea326d9c80 con 0x55ea32785730 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.307+0000 7f029259c640 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea32785730 0x55ea32786ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.307+0000 7f0279ffb640 10 client.15024 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.307+0000 7f029259c640 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea32785730 0x55ea32786ba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7748cd013bf4c989 server_cookie=571385f229312f84 in_seq=0 out_seq=0 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f0279ffb640 1 -- 172.21.15.103:0/3821347599 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea326d9c80 con 0x55ea32785730 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f0279ffb640 10 client.15024 handle_command_reply: tid=0 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 client.15024 shutdown 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 msgr2=0x55ea32772420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32772420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0288002230 tx=0x7f028801dca0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 shutdown_connections 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea32785730 0x55ea32786ba0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32772420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32771ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 --2- 172.21.15.103:0/3821347599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea327719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f0279ffb640 0 client.15024 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 >> 172.21.15.103:0/3821347599 conn(0x55ea32763bc0 msgr2=0x55ea32769880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:48.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 shutdown_connections 2024-08-19T01:34:48.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 1 -- 172.21.15.103:0/3821347599 wait complete. 2024-08-19T01:34:48.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.308+0000 7f029afa1740 20 client.15024 trim_cache size 0 max 16384 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 Processor -- start 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 -- start start 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea327674b0 0x55ea32774100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea32785e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32774a20 0x55ea32763fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea32784ee0 con 0x55ea327674b0 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ea327855b0 con 0x55ea32774a20 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f029afa1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ea32785c80 con 0x55ea32767dd0 2024-08-19T01:34:48.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.326+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea327674b0 0x55ea32774100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea327674b0 0x55ea32774100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43904/0 (socket says 172.21.15.103:43904) 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 -- 172.21.15.103:0/918117549 learned_addr learned my addr 172.21.15.103:0/918117549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 -- 172.21.15.103:0/918117549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 msgr2=0x55ea32785e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 --2- 172.21.15.103:0/918117549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea32785e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 -- 172.21.15.103:0/918117549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32774a20 msgr2=0x55ea32763fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 --2- 172.21.15.103:0/918117549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32774a20 0x55ea32763fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 -- 172.21.15.103:0/918117549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3275f130 con 0x55ea327674b0 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f0291d9b640 1 --2- 172.21.15.103:0/918117549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea327674b0 0x55ea32774100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0288005c40 tx=0x7f02880097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b093d3eeb1f31e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f027bfff640 1 -- 172.21.15.103:0/918117549 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288033070 con 0x55ea327674b0 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f027bfff640 1 -- 172.21.15.103:0/918117549 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02880054b0 con 0x55ea327674b0 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.327+0000 7f027bfff640 1 -- 172.21.15.103:0/918117549 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880049e0 con 0x55ea327674b0 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 -- 172.21.15.103:0/918117549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea327674b0 msgr2=0x55ea32774100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 --2- 172.21.15.103:0/918117549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea327674b0 0x55ea32774100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0288005c40 tx=0x7f02880097f0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 -- 172.21.15.103:0/918117549 shutdown_connections 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 --2- 172.21.15.103:0/918117549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32774a20 0x55ea32763fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 --2- 172.21.15.103:0/918117549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea32767dd0 0x55ea32785e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 --2- 172.21.15.103:0/918117549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea327674b0 0x55ea32774100 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 -- 172.21.15.103:0/918117549 >> 172.21.15.103:0/918117549 conn(0x55ea32763bc0 msgr2=0x55ea32770140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 -- 172.21.15.103:0/918117549 shutdown_connections 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 -- 172.21.15.103:0/918117549 wait complete. 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 -- start start 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.328+0000 7f029afa1740 1 Processor -- start 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea32789900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32788120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32788660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ea32786a20 con 0x55ea32774a20 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ea32786350 con 0x55ea32767dd0 2024-08-19T01:34:48.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ea327870f0 con 0x55ea327674b0 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 10 client.0 _ll_register_callbacks cb 0x7f02880126a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32788660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32788660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43906/0 (socket says 172.21.15.103:43906) 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 -- 172.21.15.103:0/1437641282 learned_addr learned my addr 172.21.15.103:0/1437641282 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea32789900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029259c640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32788120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 -- 172.21.15.103:0/1437641282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 msgr2=0x55ea32789900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea32789900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 -- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 msgr2=0x55ea32788120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32788120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 -- 172.21.15.103:0/1437641282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea32788e90 con 0x55ea32774a20 2024-08-19T01:34:48.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029259c640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32788120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0291d9b640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32788660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f028800a010 tx=0x7f0288005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee869954536256bb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0279ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0279ffb640 1 -- 172.21.15.103:0/1437641282 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288033040 con 0x55ea32774a20 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0279ffb640 1 -- 172.21.15.103:0/1437641282 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0288010e20 con 0x55ea32774a20 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02880054b0 con 0x55ea32774a20 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.329+0000 7f0279ffb640 1 -- 172.21.15.103:0/1437641282 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0288005000 con 0x55ea32774a20 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 10 client.15030 fetch_fsmap learned FSMap version 26 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ea327f1eb0 con 0x55ea32774a20 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f0279ffb640 1 -- 172.21.15.103:0/1437641282 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02880054b0 con 0x55ea32774a20 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f0279ffb640 1 -- 172.21.15.103:0/1437641282 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f0288045070 con 0x55ea32774a20 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 10 client.15030 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 10 client.15030 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 20 client.15030 populate_metadata read hostname 'smithi103' 2024-08-19T01:34:48.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea327f65a0 0x55ea327f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 4 client.15030 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "status"}) -- 0x55ea326d4190 con 0x55ea327f65a0 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.330+0000 7f029259c640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea327f65a0 0x55ea327f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f0279ffb640 10 client.15030 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029259c640 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea327f65a0 0x55ea327f52b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad6f9d0e3fbdf40 server_cookie=a9a5d77de7387fa1 in_seq=0 out_seq=0 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f0279ffb640 1 -- 172.21.15.103:0/1437641282 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ea326d4190 con 0x55ea327f65a0 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f0279ffb640 10 client.15030 handle_command_reply: tid=0 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 client.15030 shutdown 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 msgr2=0x55ea32788660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32788660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f028800a010 tx=0x7f0288005bd0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 shutdown_connections 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ea327f65a0 0x55ea327f52b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ea32774a20 0x55ea32788660 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ea32767dd0 0x55ea32788120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f0279ffb640 0 client.15030 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 --2- 172.21.15.103:0/1437641282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ea327674b0 0x55ea32789900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 >> 172.21.15.103:0/1437641282 conn(0x55ea32763bc0 msgr2=0x55ea32780240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 shutdown_connections 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 1 -- 172.21.15.103:0/1437641282 wait complete. 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.331+0000 7f029afa1740 20 client.15030 trim_cache size 0 max 16384 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:34:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24485, 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 84.158725775999997, 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 96.996994795000006 2024-08-19T01:34:48.343 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f027c0a2c10 msgr2=0x7f027c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f027c0a2c10 0x7f027c0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f02840bbd50 tx=0x7f0284087170 comp rx=0 tx=0).stop 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02941067b0 msgr2=0x7f029419b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02941067b0 0x7f029419b870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0284005940 tx=0x7f0284004270 comp rx=0 tx=0).stop 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 shutdown_connections 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f027c0a2c10 0x7f027c0a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02941067b0 0x7f029419b870 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0294101150 0x7f02941974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 --2- 172.21.15.103:0/2117055016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02941007b0 0x7f0294196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 >> 172.21.15.103:0/2117055016 conn(0x7f02940fc4d0 msgr2=0x7f02940fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:48.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 shutdown_connections 2024-08-19T01:34:48.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:48.332+0000 7f0298ba3640 1 -- 172.21.15.103:0/2117055016 wait complete. 2024-08-19T01:34:49.063 INFO:tasks.cephadm:Setup complete, yielding 2024-08-19T01:34:49.063 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-19T01:34:49.073 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:34:49.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch status' 2024-08-19T01:34:49.321 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- 172.21.15.103:0/1031717153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 msgr2=0x7f4724104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 --2- 172.21.15.103:0/1031717153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724104820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4720009900 tx=0x7f472002a410 comp rx=0 tx=0).stop 2024-08-19T01:34:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- 172.21.15.103:0/1031717153 shutdown_connections 2024-08-19T01:34:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 --2- 172.21.15.103:0/1031717153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4724107690 0x7f4724109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 --2- 172.21.15.103:0/1031717153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4724104d60 0x7f4724107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 --2- 172.21.15.103:0/1031717153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724104820 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- 172.21.15.103:0/1031717153 >> 172.21.15.103:0/1031717153 conn(0x7f47240fbfb0 msgr2=0x7f47240fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- 172.21.15.103:0/1031717153 shutdown_connections 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- 172.21.15.103:0/1031717153 wait complete. 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 Processor -- start 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- start start 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4724104d60 0x7f4724198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4724107690 0x7f472419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f472410d440 con 0x7f4724102430 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f472410d2c0 con 0x7f4724104d60 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.818+0000 7f472b28a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f472410d5c0 con 0x7f4724107690 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43920/0 (socket says 172.21.15.103:43920) 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 -- 172.21.15.103:0/586425087 learned_addr learned my addr 172.21.15.103:0/586425087 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4728825640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4724104d60 0x7f4724198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729827640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4724107690 0x7f472419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 -- 172.21.15.103:0/586425087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4724107690 msgr2=0x7f472419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4724107690 0x7f472419f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 -- 172.21.15.103:0/586425087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4724104d60 msgr2=0x7f4724198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4724104d60 0x7f4724198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 -- 172.21.15.103:0/586425087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472419f760 con 0x7f4724102430 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f4729026640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724197b30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4720004770 tx=0x7f47200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f471a7fc640 1 -- 172.21.15.103:0/586425087 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472003f030 con 0x7f4724102430 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f471a7fc640 1 -- 172.21.15.103:0/586425087 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4720003cb0 con 0x7f4724102430 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472419f9f0 con 0x7f4724102430 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f471a7fc640 1 -- 172.21.15.103:0/586425087 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720035070 con 0x7f4724102430 2024-08-19T01:34:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.819+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472419fed0 con 0x7f4724102430 2024-08-19T01:34:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.820+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47240fdaa0 con 0x7f4724102430 2024-08-19T01:34:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.821+0000 7f471a7fc640 1 -- 172.21.15.103:0/586425087 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4720042030 con 0x7f4724102430 2024-08-19T01:34:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.822+0000 7f471a7fc640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47100a2c40 0x7f47100a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.822+0000 7f4728825640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47100a2c40 0x7f47100a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.822+0000 7f471a7fc640 1 -- 172.21.15.103:0/586425087 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f47200ef890 con 0x7f4724102430 2024-08-19T01:34:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.823+0000 7f4728825640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47100a2c40 0x7f47100a5100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4714005fd0 tx=0x7f4714005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:49.827+0000 7f471a7fc640 1 -- 172.21.15.103:0/586425087 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f47200bc6c0 con 0x7f4724102430 2024-08-19T01:34:50.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.011+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4724062360 con 0x7f47100a2c40 2024-08-19T01:34:50.014 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:49 smithi103 ceph-mon[18436]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:50.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.013+0000 7f471a7fc640 1 -- 172.21.15.103:0/586425087 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4724062360 con 0x7f47100a2c40 2024-08-19T01:34:50.015 INFO:teuthology.orchestra.run.smithi103.stdout:Backend: cephadm 2024-08-19T01:34:50.015 INFO:teuthology.orchestra.run.smithi103.stdout:Available: Yes 2024-08-19T01:34:50.015 INFO:teuthology.orchestra.run.smithi103.stdout:Paused: No 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47100a2c40 msgr2=0x7f47100a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47100a2c40 0x7f47100a5100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4714005fd0 tx=0x7f4714005850 comp rx=0 tx=0).stop 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 msgr2=0x7f4724197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724197b30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4720004770 tx=0x7f47200042f0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 shutdown_connections 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47100a2c40 0x7f47100a5100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4724107690 0x7f472419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4724104d60 0x7f4724198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 --2- 172.21.15.103:0/586425087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4724102430 0x7f4724197b30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 >> 172.21.15.103:0/586425087 conn(0x7f47240fbfb0 msgr2=0x7f4724102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 shutdown_connections 2024-08-19T01:34:50.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:50.015+0000 7f472b28a640 1 -- 172.21.15.103:0/586425087 wait complete. 2024-08-19T01:34:50.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:49 smithi190 ceph-mon[22032]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:50.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:49 smithi146 ceph-mon[22098]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:34:50.772 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-19T01:34:51.022 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:51.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 -- 172.21.15.103:0/1158020970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74102b30 msgr2=0x7f7a74102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/1158020970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74102b30 0x7f7a74102fb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009900 tx=0x7f7a6002a410 comp rx=0 tx=0).stop 2024-08-19T01:34:51.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 -- 172.21.15.103:0/1158020970 shutdown_connections 2024-08-19T01:34:51.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/1158020970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a741034f0 0x7f7a74109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/1158020970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74102b30 0x7f7a74102fb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/1158020970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74101930 0x7f7a74101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 -- 172.21.15.103:0/1158020970 >> 172.21.15.103:0/1158020970 conn(0x7f7a740fd0c0 msgr2=0x7f7a740ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 -- 172.21.15.103:0/1158020970 shutdown_connections 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.477+0000 7f7a7abc9640 1 -- 172.21.15.103:0/1158020970 wait complete. 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 Processor -- start 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 -- start start 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74101930 0x7f7a74197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102b30 0x7f7a74198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a741034f0 0x7f7a7419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7410d460 con 0x7f7a74101930 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7410d2e0 con 0x7f7a74102b30 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a7abc9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7410d5e0 con 0x7f7a741034f0 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74101930 0x7f7a74197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74101930 0x7f7a74197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43950/0 (socket says 172.21.15.103:43950) 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 -- 172.21.15.103:0/3984993216 learned_addr learned my addr 172.21.15.103:0/3984993216 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:51.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a79166640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a741034f0 0x7f7a7419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a73fff640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102b30 0x7f7a74198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 -- 172.21.15.103:0/3984993216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a741034f0 msgr2=0x7f7a7419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a741034f0 0x7f7a7419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 -- 172.21.15.103:0/3984993216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102b30 msgr2=0x7f7a74198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102b30 0x7f7a74198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a78965640 1 -- 172.21.15.103:0/3984993216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7419f770 con 0x7f7a74101930 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.478+0000 7f7a73fff640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102b30 0x7f7a74198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.479+0000 7f7a78965640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74101930 0x7f7a74197b40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c970 tx=0x7f7a6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.479+0000 7f7a71ffb640 1 -- 172.21.15.103:0/3984993216 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68004220 con 0x7f7a74101930 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.479+0000 7f7a71ffb640 1 -- 172.21.15.103:0/3984993216 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a680043c0 con 0x7f7a74101930 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.479+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7419fa60 con 0x7f7a74101930 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.479+0000 7f7a71ffb640 1 -- 172.21.15.103:0/3984993216 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800f450 con 0x7f7a74101930 2024-08-19T01:34:51.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.479+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7419ffa0 con 0x7f7a74101930 2024-08-19T01:34:51.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.480+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a74108dd0 con 0x7f7a74101930 2024-08-19T01:34:51.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.481+0000 7f7a71ffb640 1 -- 172.21.15.103:0/3984993216 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7a680079b0 con 0x7f7a74101930 2024-08-19T01:34:51.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.482+0000 7f7a71ffb640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a640a2c40 0x7f7a640a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:51.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.482+0000 7f7a73fff640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a640a2c40 0x7f7a640a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:51.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.482+0000 7f7a71ffb640 1 -- 172.21.15.103:0/3984993216 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7a680cef30 con 0x7f7a74101930 2024-08-19T01:34:51.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.482+0000 7f7a73fff640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a640a2c40 0x7f7a640a5100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006010 tx=0x7f7a60000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:51.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.486+0000 7f7a71ffb640 1 -- 172.21.15.103:0/3984993216 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7a68014030 con 0x7f7a74101930 2024-08-19T01:34:51.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.662+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7a74062360 con 0x7f7a640a2c40 2024-08-19T01:34:51.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.674+0000 7f7a71ffb640 1 -- 172.21.15.103:0/3984993216 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f7a74062360 con 0x7f7a640a2c40 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.a smithi103 running (103s) 21s ago 102s 19.2M - 19.1.0-1382-gde09dd01 ea7a5189f30b a0704be86237 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.b smithi146 running (100s) 33s ago 100s 25.5M - 19.1.0-1382-gde09dd01 ea7a5189f30b 38f4334f45c1 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.c smithi190 running (115s) 33s ago 115s 17.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b d641ac39669c 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.d smithi103 running (2m) 21s ago 2m 16.6M - 19.1.0-1382-gde09dd01 ea7a5189f30b d5bbd1a5cc5e 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.e smithi146 running (2m) 33s ago 2m 17.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b 20f8936e6c78 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.f smithi190 running (118s) 33s ago 118s 19.8M - 19.1.0-1382-gde09dd01 ea7a5189f30b 151cd8713b53 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.g smithi103 running (107s) 21s ago 106s 19.1M - 19.1.0-1382-gde09dd01 ea7a5189f30b 6e180b34a10d 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.h smithi146 running (113s) 33s ago 112s 17.8M - 19.1.0-1382-gde09dd01 ea7a5189f30b 07cc3b08b067 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.i smithi190 running (110s) 33s ago 110s 19.4M - 19.1.0-1382-gde09dd01 ea7a5189f30b 1848559706c1 2024-08-19T01:34:51.676 INFO:teuthology.orchestra.run.smithi103.stdout:mds.j smithi103 running (2m) 21s ago 2m 14.5M - 19.1.0-1382-gde09dd01 ea7a5189f30b 72e1d99557ae 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mds.k smithi146 running (2m) 33s ago 2m 16.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b 28351662d1da 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mds.l smithi190 running (2m) 33s ago 2m 16.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b 9f525248d217 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.x smithi103 *:9283,8765 running (8m) 21s ago 8m 510M - 19.1.0-1382-gde09dd01 ea7a5189f30b acaf1c3cc543 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.y smithi146 *:8443,8765 running (6m) 33s ago 6m 441M - 19.1.0-1382-gde09dd01 ea7a5189f30b 9cc949ff57f8 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.z smithi190 *:8443,8765 running (6m) 33s ago 6m 441M - 19.1.0-1382-gde09dd01 ea7a5189f30b 1841274b2637 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mon.a smithi103 running (8m) 21s ago 8m 51.6M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b a8f5c18bf932 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mon.b smithi146 running (7m) 33s ago 7m 41.1M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b 1bfba6914e05 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:mon.c smithi190 running (7m) 33s ago 7m 40.5M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b 2255ac56c6eb 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 smithi103 running (6m) 21s ago 6m 58.5M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b a588ae4601b5 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 smithi146 running (6m) 33s ago 6m 45.6M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 9c47197e1574 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 smithi190 running (5m) 33s ago 5m 57.2M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b c050c8f0bcd1 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 smithi103 running (5m) 21s ago 5m 45.2M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 20d6f2926537 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 smithi146 running (5m) 33s ago 5m 42.4M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 7f161ef1fde5 2024-08-19T01:34:51.677 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 smithi190 running (4m) 33s ago 4m 55.0M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 08ee83d26a58 2024-08-19T01:34:51.678 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 smithi103 running (4m) 21s ago 4m 43.9M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 7ec074bab484 2024-08-19T01:34:51.678 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 smithi146 running (4m) 33s ago 4m 42.8M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 5a2ac26c5531 2024-08-19T01:34:51.678 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 smithi190 running (3m) 33s ago 3m 53.0M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 9d9827445358 2024-08-19T01:34:51.678 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 smithi103 running (3m) 21s ago 3m 51.6M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 5a8d15fa4ec3 2024-08-19T01:34:51.678 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 smithi146 running (2m) 33s ago 2m 42.7M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 88daebee4388 2024-08-19T01:34:51.678 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 smithi190 running (2m) 33s ago 2m 51.3M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 27496b95acd8 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a640a2c40 msgr2=0x7f7a640a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a640a2c40 0x7f7a640a5100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006010 tx=0x7f7a60000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74101930 msgr2=0x7f7a74197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74101930 0x7f7a74197b40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c970 tx=0x7f7a6800ce30 comp rx=0 tx=0).stop 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 shutdown_connections 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a640a2c40 0x7f7a640a5100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a741034f0 0x7f7a7419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102b30 0x7f7a74198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 --2- 172.21.15.103:0/3984993216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74101930 0x7f7a74197b40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 >> 172.21.15.103:0/3984993216 conn(0x7f7a740fd0c0 msgr2=0x7f7a74105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 shutdown_connections 2024-08-19T01:34:51.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:51.676+0000 7f7a7abc9640 1 -- 172.21.15.103:0/3984993216 wait complete. 2024-08-19T01:34:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:51 smithi103 ceph-mon[18436]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:52.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:51 smithi103 ceph-mon[18436]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:52.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:51 smithi190 ceph-mon[22032]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:52.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:51 smithi190 ceph-mon[22032]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:51 smithi146 ceph-mon[22098]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:51 smithi146 ceph-mon[22098]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:34:52.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-19T01:34:52.676 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:53.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:52 smithi103 ceph-mon[18436]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:53.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:52 smithi190 ceph-mon[22032]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:53.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 -- 172.21.15.103:0/2294925286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 msgr2=0x7fc31c108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 --2- 172.21.15.103:0/2294925286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c108d70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc314009ca0 tx=0x7fc31402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 -- 172.21.15.103:0/2294925286 shutdown_connections 2024-08-19T01:34:53.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 --2- 172.21.15.103:0/2294925286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c108d70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 --2- 172.21.15.103:0/2294925286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc31c100b10 0x7fc31c100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 --2- 172.21.15.103:0/2294925286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc31c101dd0 0x7fc31c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 -- 172.21.15.103:0/2294925286 >> 172.21.15.103:0/2294925286 conn(0x7fc31c0fb770 msgr2=0x7fc31c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 -- 172.21.15.103:0/2294925286 shutdown_connections 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.147+0000 7fc321a1b640 1 -- 172.21.15.103:0/2294925286 wait complete. 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 Processor -- start 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 -- start start 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc31c100b10 0x7fc31c100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc31c101dd0 0x7fc31c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c10c380 con 0x7fc31c1014d0 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c10c200 con 0x7fc31c100b10 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc321a1b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c10c500 con 0x7fc31c101dd0 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37294/0 (socket says 172.21.15.103:37294) 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 -- 172.21.15.103:0/1305512962 learned_addr learned my addr 172.21.15.103:0/1305512962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 -- 172.21.15.103:0/1305512962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc31c101dd0 msgr2=0x7fc31c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc31b7fe640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc31c101dd0 0x7fc31c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc31affd640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc31c100b10 0x7fc31c100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc31c101dd0 0x7fc31c19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 -- 172.21.15.103:0/1305512962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc31c100b10 msgr2=0x7fc31c100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc31c100b10 0x7fc31c100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 -- 172.21.15.103:0/1305512962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c19f830 con 0x7fc31c1014d0 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc31affd640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc31c100b10 0x7fc31c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc313fff640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c19c900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc30400e9f0 tx=0x7fc30400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.148+0000 7fc320a19640 1 -- 172.21.15.103:0/1305512962 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30400cd80 con 0x7fc31c1014d0 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.149+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c19fac0 con 0x7fc31c1014d0 2024-08-19T01:34:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.149+0000 7fc320a19640 1 -- 172.21.15.103:0/1305512962 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3040108e0 con 0x7fc31c1014d0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.149+0000 7fc320a19640 1 -- 172.21.15.103:0/1305512962 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304018920 con 0x7fc31c1014d0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.149+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c1a0050 con 0x7fc31c1014d0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.150+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31c107dc0 con 0x7fc31c1014d0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.151+0000 7fc320a19640 1 -- 172.21.15.103:0/1305512962 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc304010a90 con 0x7fc31c1014d0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.152+0000 7fc320a19640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc3080a2c10 0x7fc3080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.152+0000 7fc31affd640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc3080a2c10 0x7fc3080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.152+0000 7fc320a19640 1 -- 172.21.15.103:0/1305512962 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc3040cf5d0 con 0x7fc31c1014d0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.152+0000 7fc31affd640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc3080a2c10 0x7fc3080a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0058c0 tx=0x7fc30c005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.156+0000 7fc320a19640 1 -- 172.21.15.103:0/1305512962 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc30409c400 con 0x7fc31c1014d0 2024-08-19T01:34:53.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:52 smithi146 ceph-mon[22098]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:53.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.332+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fc31c062360 con 0x7fc3080a2c10 2024-08-19T01:34:53.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.339+0000 7fc320a19640 1 -- 172.21.15.103:0/1305512962 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fc31c062360 con 0x7fc3080a2c10 2024-08-19T01:34:53.340 INFO:teuthology.orchestra.run.smithi103.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-19T01:34:53.341 INFO:teuthology.orchestra.run.smithi103.stdout:mds.all 12/12 35s ago 2m smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-08-19T01:34:53.341 INFO:teuthology.orchestra.run.smithi103.stdout:mgr 3/3 35s ago 6m smithi103=x;smithi146=y;smithi190=z;count:3 2024-08-19T01:34:53.341 INFO:teuthology.orchestra.run.smithi103.stdout:mon 3/3 35s ago 7m smithi103:172.21.15.103=a;smithi146:172.21.15.146=b;smithi190:172.21.15.190=c;count:3 2024-08-19T01:34:53.341 INFO:teuthology.orchestra.run.smithi103.stdout:osd 12 35s ago - 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc3080a2c10 msgr2=0x7fc3080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc3080a2c10 0x7fc3080a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0058c0 tx=0x7fc30c005d50 comp rx=0 tx=0).stop 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 msgr2=0x7fc31c19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c19c900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc30400e9f0 tx=0x7fc30400eeb0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 shutdown_connections 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc3080a2c10 0x7fc3080a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc31c101dd0 0x7fc31c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc31c1014d0 0x7fc31c19c900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 --2- 172.21.15.103:0/1305512962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc31c100b10 0x7fc31c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 >> 172.21.15.103:0/1305512962 conn(0x7fc31c0fb770 msgr2=0x7fc31c104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 shutdown_connections 2024-08-19T01:34:53.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:53.341+0000 7fc321a1b640 1 -- 172.21.15.103:0/1305512962 wait complete. 2024-08-19T01:34:54.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:53 smithi103 ceph-mon[18436]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:54.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-19T01:34:54.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:53 smithi190 ceph-mon[22032]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:53 smithi146 ceph-mon[22098]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-19T01:34:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:54.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 -- 172.21.15.103:0/2895254522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c101930 msgr2=0x7f328c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 --2- 172.21.15.103:0/2895254522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c101930 0x7f328c101d30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3288009900 tx=0x7f328802a410 comp rx=0 tx=0).stop 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 -- 172.21.15.103:0/2895254522 shutdown_connections 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 --2- 172.21.15.103:0/2895254522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f328c1034f0 0x7f328c109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 --2- 172.21.15.103:0/2895254522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c102b30 0x7f328c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 --2- 172.21.15.103:0/2895254522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c101930 0x7f328c101d30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 -- 172.21.15.103:0/2895254522 >> 172.21.15.103:0/2895254522 conn(0x7f328c0fd0c0 msgr2=0x7f328c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 -- 172.21.15.103:0/2895254522 shutdown_connections 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.841+0000 7f329354c640 1 -- 172.21.15.103:0/2895254522 wait complete. 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 Processor -- start 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 -- start start 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c101930 0x7f328c075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:54.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f328c102b30 0x7f328c074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c1034f0 0x7f328c074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f328c10d4c0 con 0x7f328c1034f0 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f328c10d340 con 0x7f328c101930 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f329354c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f328c10d640 con 0x7f328c102b30 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c1034f0 0x7f328c074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c1034f0 0x7f328c074850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37308/0 (socket says 172.21.15.103:37308) 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 -- 172.21.15.103:0/3432445707 learned_addr learned my addr 172.21.15.103:0/3432445707 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f32912e8640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c101930 0x7f328c075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3290ae7640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f328c102b30 0x7f328c074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 -- 172.21.15.103:0/3432445707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f328c102b30 msgr2=0x7f328c074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f328c102b30 0x7f328c074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:54.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 -- 172.21.15.103:0/3432445707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c101930 msgr2=0x7f328c075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c101930 0x7f328c075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 -- 172.21.15.103:0/3432445707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c075110 con 0x7f328c1034f0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3290ae7640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f328c102b30 0x7f328c074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.842+0000 7f3291ae9640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c1034f0 0x7f328c074850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f328400b970 tx=0x7f328400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.843+0000 7f32827fc640 1 -- 172.21.15.103:0/3432445707 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284019070 con 0x7f328c1034f0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.843+0000 7f32827fc640 1 -- 172.21.15.103:0/3432445707 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3284014440 con 0x7f328c1034f0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.843+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c1a1d70 con 0x7f328c1034f0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.843+0000 7f32827fc640 1 -- 172.21.15.103:0/3432445707 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32840075d0 con 0x7f328c1034f0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.843+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c1a22a0 con 0x7f328c1034f0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.844+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328c108d80 con 0x7f328c1034f0 2024-08-19T01:34:54.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.845+0000 7f32827fc640 1 -- 172.21.15.103:0/3432445707 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f328401c070 con 0x7f328c1034f0 2024-08-19T01:34:54.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.846+0000 7f32827fc640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32780a2c10 0x7f32780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:54.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.846+0000 7f32912e8640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32780a2c10 0x7f32780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:54.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.846+0000 7f32827fc640 1 -- 172.21.15.103:0/3432445707 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f32840ceb00 con 0x7f328c1034f0 2024-08-19T01:34:54.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.846+0000 7f32912e8640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32780a2c10 0x7f32780a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3288006010 tx=0x7f3288000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:54.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:54.849+0000 7f32827fc640 1 -- 172.21.15.103:0/3432445707 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f328409b390 con 0x7f328c1034f0 2024-08-19T01:34:55.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.023+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f328c062360 con 0x7f32780a2c10 2024-08-19T01:34:55.025 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:54 smithi103 ceph-mon[18436]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.025+0000 7f32827fc640 1 -- 172.21.15.103:0/3432445707 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f328c062360 con 0x7f32780a2c10 2024-08-19T01:34:55.026 INFO:teuthology.orchestra.run.smithi103.stdout:HOST ADDR LABELS STATUS 2024-08-19T01:34:55.027 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 172.21.15.103 2024-08-19T01:34:55.027 INFO:teuthology.orchestra.run.smithi103.stdout:smithi146 172.21.15.146 2024-08-19T01:34:55.027 INFO:teuthology.orchestra.run.smithi103.stdout:smithi190 172.21.15.190 2024-08-19T01:34:55.027 INFO:teuthology.orchestra.run.smithi103.stdout:3 hosts in cluster 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32780a2c10 msgr2=0x7f32780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32780a2c10 0x7f32780a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3288006010 tx=0x7f3288000bc0 comp rx=0 tx=0).stop 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c1034f0 msgr2=0x7f328c074850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c1034f0 0x7f328c074850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f328400b970 tx=0x7f328400be30 comp rx=0 tx=0).stop 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 shutdown_connections 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32780a2c10 0x7f32780a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f328c1034f0 0x7f328c074850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f328c102b30 0x7f328c074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 --2- 172.21.15.103:0/3432445707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f328c101930 0x7f328c075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 >> 172.21.15.103:0/3432445707 conn(0x7f328c0fd0c0 msgr2=0x7f328c105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 shutdown_connections 2024-08-19T01:34:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:55.027+0000 7f329354c640 1 -- 172.21.15.103:0/3432445707 wait complete. 2024-08-19T01:34:55.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:54 smithi190 ceph-mon[22032]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:55.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:54 smithi146 ceph-mon[22098]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:55.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-19T01:34:56.014 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:56.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:55 smithi103 ceph-mon[18436]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:56.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:55 smithi190 ceph-mon[22032]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:55 smithi146 ceph-mon[22098]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-19T01:34:56.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2204342211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d81075f0 msgr2=0x7fe6d8109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:56.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2204342211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d81075f0 0x7fe6d8109a40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009ca0 tx=0x7fe6d002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2204342211 shutdown_connections 2024-08-19T01:34:56.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2204342211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d81075f0 0x7fe6d8109a40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2204342211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d8104cc0 0x7fe6d81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2204342211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6d8102390 0x7fe6d8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2204342211 >> 172.21.15.103:0/2204342211 conn(0x7fe6d80fbfb0 msgr2=0x7fe6d80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2204342211 shutdown_connections 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.511+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2204342211 wait complete. 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 Processor -- start 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 -- start start 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8102390 0x7fe6d8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d8104cc0 0x7fe6d8100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6d81075f0 0x7fe6d8100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d810d4c0 con 0x7fe6d8102390 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d810d340 con 0x7fe6d8104cc0 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6dd35a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d810d640 con 0x7fe6d81075f0 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8102390 0x7fe6d8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:56.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6d6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8102390 0x7fe6d8101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37332/0 (socket says 172.21.15.103:37332) 2024-08-19T01:34:56.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.512+0000 7fe6d6ffd640 1 -- 172.21.15.103:0/2275379775 learned_addr learned my addr 172.21.15.103:0/2275379775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:56.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d77fe640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6d81075f0 0x7fe6d8100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:56.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d67fc640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d8104cc0 0x7fe6d8100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:56.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d6ffd640 1 -- 172.21.15.103:0/2275379775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6d81075f0 msgr2=0x7fe6d8100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:56.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d6ffd640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6d81075f0 0x7fe6d8100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d6ffd640 1 -- 172.21.15.103:0/2275379775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d8104cc0 msgr2=0x7fe6d8100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d6ffd640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d8104cc0 0x7fe6d8100340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d6ffd640 1 -- 172.21.15.103:0/2275379775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d8101140 con 0x7fe6d8102390 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d77fe640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6d81075f0 0x7fe6d8100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6d6ffd640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8102390 0x7fe6d8101cf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800b4a0 tx=0x7fe6c800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6bffff640 1 -- 172.21.15.103:0/2275379775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8015020 con 0x7fe6d8102390 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81a1ca0 con 0x7fe6d8102390 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6bffff640 1 -- 172.21.15.103:0/2275379775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6c8011ce0 con 0x7fe6d8102390 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6bffff640 1 -- 172.21.15.103:0/2275379775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8010dc0 con 0x7fe6d8102390 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.513+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a21b0 con 0x7fe6d8102390 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.514+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d80fdaa0 con 0x7fe6d8102390 2024-08-19T01:34:56.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.515+0000 7fe6bffff640 1 -- 172.21.15.103:0/2275379775 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe6c8018020 con 0x7fe6d8102390 2024-08-19T01:34:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.516+0000 7fe6bffff640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6c00a2c90 0x7fe6c00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.516+0000 7fe6bffff640 1 -- 172.21.15.103:0/2275379775 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe6c80ce920 con 0x7fe6d8102390 2024-08-19T01:34:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.516+0000 7fe6d67fc640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6c00a2c90 0x7fe6c00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:56.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.517+0000 7fe6d67fc640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6c00a2c90 0x7fe6c00a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8101a70 tx=0x7fe6cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:56.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.520+0000 7fe6bffff640 1 -- 172.21.15.103:0/2275379775 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe6c801f020 con 0x7fe6d8102390 2024-08-19T01:34:56.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.693+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe6d8062360 con 0x7fe6c00a2c90 2024-08-19T01:34:56.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.695+0000 7fe6bffff640 1 -- 172.21.15.103:0/2275379775 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe6d8062360 con 0x7fe6c00a2c90 2024-08-19T01:34:56.695 INFO:teuthology.orchestra.run.smithi103.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-19T01:34:56.695 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-19T01:34:56.696 INFO:teuthology.orchestra.run.smithi103.stdout:smithi146 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-19T01:34:56.696 INFO:teuthology.orchestra.run.smithi103.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-19T01:34:56.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.696+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6c00a2c90 msgr2=0x7fe6c00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:56.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.696+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6c00a2c90 0x7fe6c00a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8101a70 tx=0x7fe6cc00a2d0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.696+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8102390 msgr2=0x7fe6d8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.696+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8102390 0x7fe6d8101cf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800b4a0 tx=0x7fe6c800b8a0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 shutdown_connections 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe6c00a2c90 0x7fe6c00a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6d81075f0 0x7fe6d8100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6d8104cc0 0x7fe6d8100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 --2- 172.21.15.103:0/2275379775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8102390 0x7fe6d8101cf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 >> 172.21.15.103:0/2275379775 conn(0x7fe6d80fbfb0 msgr2=0x7fe6d8102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 shutdown_connections 2024-08-19T01:34:56.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:56.697+0000 7fe6dd35a640 1 -- 172.21.15.103:0/2275379775 wait complete. 2024-08-19T01:34:57.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:56 smithi103 ceph-mon[18436]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:57.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:56 smithi190 ceph-mon[22032]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:57.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:56 smithi146 ceph-mon[22098]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:57.449 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-19T01:34:57.463 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:34:57.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-19T01:34:57.712 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:58.107 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:57 smithi103 ceph-mon[18436]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:58.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:57 smithi103 ceph-mon[18436]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:58.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:57 smithi190 ceph-mon[22032]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:58.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:57 smithi190 ceph-mon[22032]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:57 smithi146 ceph-mon[22098]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-19T01:34:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:57 smithi146 ceph-mon[22098]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 -- 172.21.15.103:0/15133193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 msgr2=0x7fe128100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 --2- 172.21.15.103:0/15133193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128100c00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009900 tx=0x7fe11c02a410 comp rx=0 tx=0).stop 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 -- 172.21.15.103:0/15133193 shutdown_connections 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 --2- 172.21.15.103:0/15133193 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1281023c0 0x7fe128108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 --2- 172.21.15.103:0/15133193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe128101a00 0x7fe128101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 --2- 172.21.15.103:0/15133193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128100c00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 -- 172.21.15.103:0/15133193 >> 172.21.15.103:0/15133193 conn(0x7fe1280fbfb0 msgr2=0x7fe1280fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 -- 172.21.15.103:0/15133193 shutdown_connections 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.187+0000 7fe12faca640 1 -- 172.21.15.103:0/15133193 wait complete. 2024-08-19T01:34:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 Processor -- start 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 -- start start 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe128101a00 0x7fe1281980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1281023c0 0x7fe12819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe12810c310 con 0x7fe128100800 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe12810c190 con 0x7fe1281023c0 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12faca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe12810c490 con 0x7fe128101a00 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12d866640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12d866640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37348/0 (socket says 172.21.15.103:37348) 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12d866640 1 -- 172.21.15.103:0/1688649260 learned_addr learned my addr 172.21.15.103:0/1688649260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12d065640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe128101a00 0x7fe1281980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.188+0000 7fe12e067640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1281023c0 0x7fe12819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12d866640 1 -- 172.21.15.103:0/1688649260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe128101a00 msgr2=0x7fe1281980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12d866640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe128101a00 0x7fe1281980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12d866640 1 -- 172.21.15.103:0/1688649260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1281023c0 msgr2=0x7fe12819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12d866640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1281023c0 0x7fe12819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12d866640 1 -- 172.21.15.103:0/1688649260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12819f7c0 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12e067640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1281023c0 0x7fe12819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12d866640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128197b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0314e0 tx=0x7fe11c031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe11affd640 1 -- 172.21.15.103:0/1688649260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c03f030 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe11affd640 1 -- 172.21.15.103:0/1688649260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe11c0044b0 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe11affd640 1 -- 172.21.15.103:0/1688649260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c035070 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12819fa50 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.189+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12819ff30 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.190+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1281a0160 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.191+0000 7fe11affd640 1 -- 172.21.15.103:0/1688649260 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe11c042030 con 0x7fe128100800 2024-08-19T01:34:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.192+0000 7fe11affd640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1140a2c40 0x7fe1140a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.192+0000 7fe12d065640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1140a2c40 0x7fe1140a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.192+0000 7fe11affd640 1 -- 172.21.15.103:0/1688649260 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe11c0efb10 con 0x7fe128100800 2024-08-19T01:34:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.192+0000 7fe12d065640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1140a2c40 0x7fe1140a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe1281990b0 tx=0x7fe12400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.195+0000 7fe11affd640 1 -- 172.21.15.103:0/1688649260 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe11c0bca60 con 0x7fe128100800 2024-08-19T01:34:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.398+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe1281a0330 con 0x7fe128100800 2024-08-19T01:34:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.398+0000 7fe11affd640 1 -- 172.21.15.103:0/1688649260 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3139 (secure 0 0 0) 0x7fe11c0bc360 con 0x7fe128100800 2024-08-19T01:34:58.399 INFO:teuthology.orchestra.run.smithi103.stdout:e26 2024-08-19T01:34:58.399 INFO:teuthology.orchestra.run.smithi103.stdout:btime 2024-08-19T01:33:34:962864+0000 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:legacy client fscid: 1 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:Filesystem 'cephfs' (1) 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:fs_name cephfs 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 26 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-08-19T01:33:23.991203+0000 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-08-19T01:33:34.962843+0000 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:tableserver 0 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:root 0 2024-08-19T01:34:58.400 INFO:teuthology.orchestra.run.smithi103.stdout:session_timeout 300 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:session_autoclose 300 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:max_file_size 1099511627776 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:max_xattr_size 65536 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:required_client_features {} 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure 0 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure_osd_epoch 0 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:max_mds 5 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:in 0,1,2,3,4 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:up {0=24485,1=24478,2=24466,3=24454,4=24452} 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:failed 2024-08-19T01:34:58.401 INFO:teuthology.orchestra.run.smithi103.stdout:damaged 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:stopped 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:data_pools [3] 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:metadata_pool 2 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:inline_data disabled 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:balancer 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:bal_rank_mask -1 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:standby_count_wanted 0 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:qdb_cluster leader: 24485 members: 24485,24466,24454,24478,24452 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b{0:24485} state up:active seq 5 addr [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.e{0:24425} state up:standby-replay seq 1 addr [v2:172.21.15.146:6834/3609998196,v1:172.21.15.146:6835/3609998196] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.i{1:24478} state up:active seq 9 addr [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.402 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.l{1:24415} state up:standby-replay seq 1 addr [v2:172.21.15.190:6832/1574351198,v1:172.21.15.190:6833/1574351198] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c{2:24466} state up:active seq 10 addr [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.k{2:24404} state up:standby-replay seq 1 addr [v2:172.21.15.146:6832/1682379250,v1:172.21.15.146:6833/1682379250] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.f{3:24454} state up:active seq 11 addr [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a{3:14592} state up:standby-replay seq 1 addr [v2:172.21.15.103:6840/1914348964,v1:172.21.15.103:6841/1914348964] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.h{4:24452} state up:active seq 10 addr [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.g{4:14580} state up:standby-replay seq 1 addr [v2:172.21.15.103:6838/2158943857,v1:172.21.15.103:6839/2158943857] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:Standby daemons: 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.j{-1:14514} state up:standby seq 1 addr [v2:172.21.15.103:6834/3930645278,v1:172.21.15.103:6835/3930645278] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.403 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d{-1:14526} state up:standby seq 1 addr [v2:172.21.15.103:6836/3245550347,v1:172.21.15.103:6837/3245550347] compat {c=[1],r=[1],i=[1fff]}] 2024-08-19T01:34:58.404 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-08-19T01:34:58.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1140a2c40 msgr2=0x7fe1140a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:58.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1140a2c40 0x7fe1140a5100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe1281990b0 tx=0x7fe12400b410 comp rx=0 tx=0).stop 2024-08-19T01:34:58.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 msgr2=0x7fe128197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:58.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128197b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0314e0 tx=0x7fe11c031a30 comp rx=0 tx=0).stop 2024-08-19T01:34:58.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 shutdown_connections 2024-08-19T01:34:58.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1140a2c40 0x7fe1140a5100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1281023c0 0x7fe12819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe128101a00 0x7fe1281980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 --2- 172.21.15.103:0/1688649260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe128100800 0x7fe128197b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:58.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 >> 172.21.15.103:0/1688649260 conn(0x7fe1280fbfb0 msgr2=0x7fe128069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:58.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 shutdown_connections 2024-08-19T01:34:58.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:58.400+0000 7fe12faca640 1 -- 172.21.15.103:0/1688649260 wait complete. 2024-08-19T01:34:59.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:58 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1688649260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-19T01:34:59.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-19T01:34:59.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:58 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1688649260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-19T01:34:59.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:58 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1688649260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-19T01:34:59.398 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:34:59.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 -- 172.21.15.103:0/100618904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b81023c0 msgr2=0x7f17b8108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:59.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 --2- 172.21.15.103:0/100618904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b81023c0 0x7f17b8108c50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009900 tx=0x7f17b002a410 comp rx=0 tx=0).stop 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 -- 172.21.15.103:0/100618904 shutdown_connections 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 --2- 172.21.15.103:0/100618904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b81023c0 0x7f17b8108c50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 --2- 172.21.15.103:0/100618904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b8101a00 0x7f17b8101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 --2- 172.21.15.103:0/100618904 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8100800 0x7f17b8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 -- 172.21.15.103:0/100618904 >> 172.21.15.103:0/100618904 conn(0x7f17b80fbfb0 msgr2=0x7f17b80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 -- 172.21.15.103:0/100618904 shutdown_connections 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.908+0000 7f17bf331640 1 -- 172.21.15.103:0/100618904 wait complete. 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 Processor -- start 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 -- start start 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8100800 0x7f17b8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:59.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b8101a00 0x7f17b8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b81023c0 0x7f17b819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17b810c330 con 0x7f17b8101a00 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17b810c1b0 con 0x7f17b8100800 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bf331640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17b810c4b0 con 0x7f17b81023c0 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b8101a00 0x7f17b8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b8101a00 0x7f17b8198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37380/0 (socket says 172.21.15.103:37380) 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 -- 172.21.15.103:0/3334960353 learned_addr learned my addr 172.21.15.103:0/3334960353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bd0cd640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8100800 0x7f17b8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bd8ce640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b81023c0 0x7f17b819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:59.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 -- 172.21.15.103:0/3334960353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b81023c0 msgr2=0x7f17b819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b81023c0 0x7f17b819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 -- 172.21.15.103:0/3334960353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8100800 msgr2=0x7f17b8197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8100800 0x7f17b8197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bc8cc640 1 -- 172.21.15.103:0/3334960353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b819f700 con 0x7f17b8101a00 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.909+0000 7f17bd0cd640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8100800 0x7f17b8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.910+0000 7f17bc8cc640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b8101a00 0x7f17b8198010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d770 tx=0x7f17ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.910+0000 7f17b67fc640 1 -- 172.21.15.103:0/3334960353 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac017070 con 0x7f17b8101a00 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.910+0000 7f17b67fc640 1 -- 172.21.15.103:0/3334960353 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17ac0054d0 con 0x7f17b8101a00 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.910+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b819f9f0 con 0x7f17b8101a00 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.910+0000 7f17b67fc640 1 -- 172.21.15.103:0/3334960353 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004cf0 con 0x7f17b8101a00 2024-08-19T01:34:59.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.910+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b819fee0 con 0x7f17b8101a00 2024-08-19T01:34:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.911+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b81a0110 con 0x7f17b8101a00 2024-08-19T01:34:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.912+0000 7f17b67fc640 1 -- 172.21.15.103:0/3334960353 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f17ac004e90 con 0x7f17b8101a00 2024-08-19T01:34:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.912+0000 7f17b67fc640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17a00a2c40 0x7f17a00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:34:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.913+0000 7f17bd0cd640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17a00a2c40 0x7f17a00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:34:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.913+0000 7f17b67fc640 1 -- 172.21.15.103:0/3334960353 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f17ac0cf0d0 con 0x7f17b8101a00 2024-08-19T01:34:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.913+0000 7f17bd0cd640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17a00a2c40 0x7f17a00a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f17b8101860 tx=0x7f17a8009540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:34:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:34:59.916+0000 7f17b67fc640 1 -- 172.21.15.103:0/3334960353 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f17ac09bf00 con 0x7f17b8101a00 2024-08-19T01:35:00.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.085+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f17b81087b0 con 0x7f17b8101a00 2024-08-19T01:35:00.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:34:59 smithi103 ceph-mon[18436]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:35:00.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.086+0000 7f17b67fc640 1 -- 172.21.15.103:0/3334960353 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) ==== 56+0+5294 (secure 0 0 0) 0x7f17ac09b800 con 0x7f17b8101a00 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 86 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:fsid e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-08-19T01:26:05.629697+0000 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-08-19T01:34:01.874407+0000 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:crush_version 26 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:full_ratio 0.95 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:backfillfull_ratio 0.9 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:nearfull_ratio 0.85 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:require_min_compat_client luminous 2024-08-19T01:35:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:min_compat_client luminous 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:require_osd_release squid 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:stretch_mode_enabled false 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:max_osd 12 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6802/2060644281,v1:172.21.15.103:6803/2060644281] [v2:172.21.15.103:6804/2060644281,v1:172.21.15.103:6805/2060644281] exists,up 0157e941-2ba7-4376-b473-9bcdaace0ef4 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6800/351745310,v1:172.21.15.146:6801/351745310] [v2:172.21.15.146:6802/351745310,v1:172.21.15.146:6803/351745310] exists,up bc4bc49c-cf46-48e4-98ed-60ad89a2812a 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/2941836968,v1:172.21.15.190:6801/2941836968] [v2:172.21.15.190:6802/2941836968,v1:172.21.15.190:6803/2941836968] exists,up 5253afd0-394e-4565-b605-fad40ff30406 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6810/2225524779,v1:172.21.15.103:6811/2225524779] [v2:172.21.15.103:6812/2225524779,v1:172.21.15.103:6813/2225524779] exists,up e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 up in weight 1 up_from 33 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6808/3000848263,v1:172.21.15.146:6809/3000848263] [v2:172.21.15.146:6810/3000848263,v1:172.21.15.146:6811/3000848263] exists,up 2d80a218-c821-4a4c-a133-ec98bf694692 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 up in weight 1 up_from 38 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/1607717439,v1:172.21.15.190:6809/1607717439] [v2:172.21.15.190:6810/1607717439,v1:172.21.15.190:6811/1607717439] exists,up a36b0adc-8fec-4313-8298-55fc4236a809 2024-08-19T01:35:00.088 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 up in weight 1 up_from 45 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6818/3250967398,v1:172.21.15.103:6819/3250967398] [v2:172.21.15.103:6820/3250967398,v1:172.21.15.103:6821/3250967398] exists,up 243118b8-58e2-40a3-833c-008ec861cab9 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 up in weight 1 up_from 50 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6816/2549773085,v1:172.21.15.146:6817/2549773085] [v2:172.21.15.146:6818/2549773085,v1:172.21.15.146:6819/2549773085] exists,up 01bec701-16f3-4c16-87f7-59daf836ba88 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 up in weight 1 up_from 57 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/2035395456,v1:172.21.15.190:6817/2035395456] [v2:172.21.15.190:6818/2035395456,v1:172.21.15.190:6819/2035395456] exists,up 581ea912-28e7-4dec-8e01-18301ee980cb 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 up in weight 1 up_from 65 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6826/1538313167,v1:172.21.15.103:6827/1538313167] [v2:172.21.15.103:6828/1538313167,v1:172.21.15.103:6829/1538313167] exists,up f8ae349f-b007-407a-b7e9-0f9b9ba66aec 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 up in weight 1 up_from 70 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6824/1357662473,v1:172.21.15.146:6825/1357662473] [v2:172.21.15.146:6826/1357662473,v1:172.21.15.146:6827/1357662473] exists,up 5d90c65e-e820-4352-b4bd-58204606e5ba 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 up in weight 1 up_from 78 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/2863234594,v1:172.21.15.190:6825/2863234594] [v2:172.21.15.190:6826/2863234594,v1:172.21.15.190:6827/2863234594] exists,up 22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.a [2,11] 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.b [2,4] 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.10 [8,11,2,1] 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.0 [5,9] 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.3 [4,9] 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.13 [4,9] 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.27 [4,8] 2024-08-19T01:35:00.089 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/2698864603 expires 2024-08-20T01:27:00.697852+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3003368150 expires 2024-08-20T01:27:00.697852+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/2881014674 expires 2024-08-20T01:27:00.697852+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/2881014674 expires 2024-08-20T01:27:00.697852+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/2169969902 expires 2024-08-20T01:26:36.589386+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/3969074886 expires 2024-08-20T01:26:36.589386+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/975765434 expires 2024-08-20T01:27:00.697852+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/3969074886 expires 2024-08-20T01:26:36.589386+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3082166873 expires 2024-08-20T01:26:36.589386+0000 2024-08-19T01:35:00.090 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/948089109 expires 2024-08-20T01:26:36.589386+0000 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.088+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17a00a2c40 msgr2=0x7f17a00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.088+0000 7f17bf331640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17a00a2c40 0x7f17a00a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f17b8101860 tx=0x7f17a8009540 comp rx=0 tx=0).stop 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.088+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b8101a00 msgr2=0x7f17b8198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.088+0000 7f17bf331640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b8101a00 0x7f17b8198010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d770 tx=0x7f17ac00dc30 comp rx=0 tx=0).stop 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.088+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 shutdown_connections 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.088+0000 7f17bf331640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17a00a2c40 0x7f17a00a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.089+0000 7f17bf331640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17b81023c0 0x7f17b819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.089+0000 7f17bf331640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17b8101a00 0x7f17b8198010 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.089+0000 7f17bf331640 1 --2- 172.21.15.103:0/3334960353 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17b8100800 0x7f17b8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.089+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 >> 172.21.15.103:0/3334960353 conn(0x7f17b80fbfb0 msgr2=0x7f17b80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:00.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.089+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 shutdown_connections 2024-08-19T01:35:00.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:00.089+0000 7f17bf331640 1 -- 172.21.15.103:0/3334960353 wait complete. 2024-08-19T01:35:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:34:59 smithi146 ceph-mon[22098]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:35:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:34:59 smithi190 ceph-mon[22032]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:35:00.812 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-19T01:35:00.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:35:01.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:00 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3334960353' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-19T01:35:01.086 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:01.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:00 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3334960353' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-19T01:35:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:00 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3334960353' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-19T01:35:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 -- 172.21.15.103:0/415221858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888100b50 msgr2=0x7ff888100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:01.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 --2- 172.21.15.103:0/415221858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888100b50 0x7ff888100f30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff878009900 tx=0x7ff87802a480 comp rx=0 tx=0).stop 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 -- 172.21.15.103:0/415221858 shutdown_connections 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 --2- 172.21.15.103:0/415221858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff888102c20 0x7ff88810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 --2- 172.21.15.103:0/415221858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8881022a0 0x7ff8881026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 --2- 172.21.15.103:0/415221858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888100b50 0x7ff888100f30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 -- 172.21.15.103:0/415221858 >> 172.21.15.103:0/415221858 conn(0x7ff8880fc500 msgr2=0x7ff8880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 -- 172.21.15.103:0/415221858 shutdown_connections 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.580+0000 7ff88d291640 1 -- 172.21.15.103:0/415221858 wait complete. 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.581+0000 7ff88d291640 1 Processor -- start 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.581+0000 7ff88d291640 1 -- start start 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888100b50 0x7ff888197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8881022a0 0x7ff888197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:01.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888102c20 0x7ff88819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff888110b20 con 0x7ff888102c20 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881109a0 con 0x7ff8881022a0 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff888110ca0 con 0x7ff888100b50 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888102c20 0x7ff88819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888102c20 0x7ff88819b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37394/0 (socket says 172.21.15.103:37394) 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 -- 172.21.15.103:0/1089360657 learned_addr learned my addr 172.21.15.103:0/1089360657 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff886ffd640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888100b50 0x7ff888197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff87e7fc640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8881022a0 0x7ff888197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 -- 172.21.15.103:0/1089360657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888100b50 msgr2=0x7ff888197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888100b50 0x7ff888197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 -- 172.21.15.103:0/1089360657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8881022a0 msgr2=0x7ff888197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8881022a0 0x7ff888197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 -- 172.21.15.103:0/1089360657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88819c0d0 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff8877fe640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888102c20 0x7ff88819b9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff88000eaa0 tx=0x7ff88000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff884ff9640 1 -- 172.21.15.103:0/1089360657 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88000cdb0 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88819c3c0 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff884ff9640 1 -- 172.21.15.103:0/1089360657 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff880005020 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a3ca0 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.582+0000 7ff884ff9640 1 -- 172.21.15.103:0/1089360657 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800116d0 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.583+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff888100f30 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.585+0000 7ff884ff9640 1 -- 172.21.15.103:0/1089360657 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff8800051d0 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.585+0000 7ff884ff9640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8740a2b80 0x7ff8740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.586+0000 7ff886ffd640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8740a2b80 0x7ff8740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.586+0000 7ff884ff9640 1 -- 172.21.15.103:0/1089360657 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff8800cf2a0 con 0x7ff888102c20 2024-08-19T01:35:01.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.586+0000 7ff886ffd640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8740a2b80 0x7ff8740a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff878005ab0 tx=0x7ff878005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:01.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.590+0000 7ff884ff9640 1 -- 172.21.15.103:0/1089360657 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff880016030 con 0x7ff888102c20 2024-08-19T01:35:01.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.792+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff888108dc0 con 0x7ff888102c20 2024-08-19T01:35:01.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.793+0000 7ff884ff9640 1 -- 172.21.15.103:0/1089360657 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11989 (secure 0 0 0) 0x7ff88009c1d0 con 0x7ff888102c20 2024-08-19T01:35:01.794 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:01.794 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-08-19T01:33:34:962864+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:33:34.962843+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:35:01.794 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-08-19T01:35:01.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8740a2b80 msgr2=0x7ff8740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:01.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8740a2b80 0x7ff8740a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff878005ab0 tx=0x7ff878005a20 comp rx=0 tx=0).stop 2024-08-19T01:35:01.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888102c20 msgr2=0x7ff88819b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:01.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888102c20 0x7ff88819b9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff88000eaa0 tx=0x7ff88000ef60 comp rx=0 tx=0).stop 2024-08-19T01:35:01.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 shutdown_connections 2024-08-19T01:35:01.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff8740a2b80 0x7ff8740a5040 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff888102c20 0x7ff88819b9f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8881022a0 0x7ff888197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 --2- 172.21.15.103:0/1089360657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888100b50 0x7ff888197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:01.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 >> 172.21.15.103:0/1089360657 conn(0x7ff8880fc500 msgr2=0x7ff8880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:01.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 shutdown_connections 2024-08-19T01:35:01.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:01.795+0000 7ff88d291640 1 -- 172.21.15.103:0/1089360657 wait complete. 2024-08-19T01:35:02.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:01 smithi103 ceph-mon[18436]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:35:02.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:01 smithi103 ceph-mon[18436]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-19T01:35:02.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:01 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1089360657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:01 smithi146 ceph-mon[22098]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:35:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:01 smithi146 ceph-mon[22098]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-19T01:35:02.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:01 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1089360657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:01 smithi190 ceph-mon[22032]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-19T01:35:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:01 smithi190 ceph-mon[22032]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-19T01:35:02.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:01 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1089360657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:02.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:35:02.817 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:03.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/98386970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc100860 msgr2=0x7fa3bc100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:03.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/98386970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc100860 0x7fa3bc100c40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009ca0 tx=0x7fa3b802a940 comp rx=0 tx=0).stop 2024-08-19T01:35:03.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/98386970 shutdown_connections 2024-08-19T01:35:03.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/98386970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3bc105c70 0x7fa3bc106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/98386970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3bc101180 0x7fa3bc105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/98386970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc100860 0x7fa3bc100c40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/98386970 >> 172.21.15.103:0/98386970 conn(0x7fa3bc0fc4d0 msgr2=0x7fa3bc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/98386970 shutdown_connections 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.341+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/98386970 wait complete. 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 Processor -- start 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 -- start start 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3bc100860 0x7fa3bc19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3bc101180 0x7fa3bc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc105c70 0x7fa3bc1a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc118130 con 0x7fa3bc105c70 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc117fb0 con 0x7fa3bc101180 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c3f5c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc1182b0 con 0x7fa3bc100860 2024-08-19T01:35:03.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc105c70 0x7fa3bc1a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc105c70 0x7fa3bc1a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58192/0 (socket says 172.21.15.103:58192) 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 -- 172.21.15.103:0/2982031141 learned_addr learned my addr 172.21.15.103:0/2982031141 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c1cf8640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3bc100860 0x7fa3bc19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c14f7640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3bc101180 0x7fa3bc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 -- 172.21.15.103:0/2982031141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3bc100860 msgr2=0x7fa3bc19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3bc100860 0x7fa3bc19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 -- 172.21.15.103:0/2982031141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3bc101180 msgr2=0x7fa3bc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3bc101180 0x7fa3bc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.342+0000 7fa3c24f9640 1 -- 172.21.15.103:0/2982031141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc1a3540 con 0x7fa3bc105c70 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.343+0000 7fa3c24f9640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc105c70 0x7fa3bc1a2e60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c970 tx=0x7fa3b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.343+0000 7fa3b2ffd640 1 -- 172.21.15.103:0/2982031141 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4007bb0 con 0x7fa3bc105c70 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.343+0000 7fa3b2ffd640 1 -- 172.21.15.103:0/2982031141 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b4007d50 con 0x7fa3bc105c70 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.343+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3bc1a3830 con 0x7fa3bc105c70 2024-08-19T01:35:03.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.343+0000 7fa3b2ffd640 1 -- 172.21.15.103:0/2982031141 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4018700 con 0x7fa3bc105c70 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.343+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3bc1ab110 con 0x7fa3bc105c70 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.344+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3bc105650 con 0x7fa3bc105c70 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.345+0000 7fa3b2ffd640 1 -- 172.21.15.103:0/2982031141 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa3b400fa20 con 0x7fa3bc105c70 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.345+0000 7fa3b2ffd640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa3a80a2b80 0x7fa3a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.346+0000 7fa3c1cf8640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa3a80a2b80 0x7fa3a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.346+0000 7fa3b2ffd640 1 -- 172.21.15.103:0/2982031141 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa3b40cf080 con 0x7fa3bc105c70 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.346+0000 7fa3c1cf8640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa3a80a2b80 0x7fa3a80a5040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005fe0 tx=0x7fa3b8005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:03.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.349+0000 7fa3b2ffd640 1 -- 172.21.15.103:0/2982031141 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa3b4014030 con 0x7fa3bc105c70 2024-08-19T01:35:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.549+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3bc1ab3f0 con 0x7fa3bc105c70 2024-08-19T01:35:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.549+0000 7fa3b2ffd640 1 -- 172.21.15.103:0/2982031141 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11989 (secure 0 0 0) 0x7fa3b409c010 con 0x7fa3bc105c70 2024-08-19T01:35:03.551 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:03.551 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-08-19T01:33:34:962864+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:33:34.962843+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:35:03.551 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-08-19T01:35:03.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.551+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa3a80a2b80 msgr2=0x7fa3a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:03.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa3a80a2b80 0x7fa3a80a5040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005fe0 tx=0x7fa3b8005ef0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc105c70 msgr2=0x7fa3bc1a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:03.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc105c70 0x7fa3bc1a2e60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c970 tx=0x7fa3b400ce30 comp rx=0 tx=0).stop 2024-08-19T01:35:03.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 shutdown_connections 2024-08-19T01:35:03.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa3a80a2b80 0x7fa3a80a5040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3bc105c70 0x7fa3bc1a2e60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa3bc101180 0x7fa3bc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 --2- 172.21.15.103:0/2982031141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3bc100860 0x7fa3bc19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:03.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 >> 172.21.15.103:0/2982031141 conn(0x7fa3bc0fc4d0 msgr2=0x7fa3bc103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:03.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 shutdown_connections 2024-08-19T01:35:03.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:03.552+0000 7fa3c3f5c640 1 -- 172.21.15.103:0/2982031141 wait complete. 2024-08-19T01:35:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:03 smithi146 ceph-mon[22098]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-19T01:35:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:03 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2982031141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:04.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:03 smithi103 ceph-mon[18436]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-19T01:35:04.211 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:03 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2982031141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:04.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:35:04.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:03 smithi190 ceph-mon[22032]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-19T01:35:04.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:03 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2982031141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:04.503 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.028+0000 7f58a3359640 1 -- 172.21.15.103:0/2503347297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c102280 msgr2=0x7f589c10e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.028+0000 7f58a3359640 1 --2- 172.21.15.103:0/2503347297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c102280 0x7f589c10e7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f588c009930 tx=0x7f588c02a420 comp rx=0 tx=0).stop 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.028+0000 7f58a3359640 1 -- 172.21.15.103:0/2503347297 shutdown_connections 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.028+0000 7f58a3359640 1 --2- 172.21.15.103:0/2503347297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c102280 0x7f589c10e7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.028+0000 7f58a3359640 1 --2- 172.21.15.103:0/2503347297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589c1018e0 0x7f589c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.028+0000 7f58a3359640 1 --2- 172.21.15.103:0/2503347297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589c1078e0 0x7f589c107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.028+0000 7f58a3359640 1 -- 172.21.15.103:0/2503347297 >> 172.21.15.103:0/2503347297 conn(0x7f589c0fd600 msgr2=0x7f589c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:05.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 -- 172.21.15.103:0/2503347297 shutdown_connections 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 -- 172.21.15.103:0/2503347297 wait complete. 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 Processor -- start 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 -- start start 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589c1018e0 0x7f589c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589c102280 0x7f589c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c1078e0 0x7f589c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111bb0 con 0x7f589c1078e0 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111a30 con 0x7f589c102280 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a3359640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f589c111d30 con 0x7f589c1018e0 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a18f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c1078e0 0x7f589c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a18f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c1078e0 0x7f589c19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58212/0 (socket says 172.21.15.103:58212) 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.029+0000 7f58a18f6640 1 -- 172.21.15.103:0/724294759 learned_addr learned my addr 172.21.15.103:0/724294759 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a08f4640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589c102280 0x7f589c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a10f5640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589c1018e0 0x7f589c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a18f6640 1 -- 172.21.15.103:0/724294759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589c1018e0 msgr2=0x7f589c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a18f6640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589c1018e0 0x7f589c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a18f6640 1 -- 172.21.15.103:0/724294759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589c102280 msgr2=0x7f589c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a18f6640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589c102280 0x7f589c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a18f6640 1 -- 172.21.15.103:0/724294759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c19d060 con 0x7f589c1078e0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a18f6640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c1078e0 0x7f589c19c980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f588c009900 tx=0x7f588c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f589a7fc640 1 -- 172.21.15.103:0/724294759 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c03f030 con 0x7f589c1078e0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f589a7fc640 1 -- 172.21.15.103:0/724294759 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f588c0035f0 con 0x7f589c1078e0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c19d2f0 con 0x7f589c1078e0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f589a7fc640 1 -- 172.21.15.103:0/724294759 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c035070 con 0x7f589c1078e0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.030+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1a4c30 con 0x7f589c1078e0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.031+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c103020 con 0x7f589c1078e0 2024-08-19T01:35:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.032+0000 7f589a7fc640 1 -- 172.21.15.103:0/724294759 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f588c003b10 con 0x7f589c1078e0 2024-08-19T01:35:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.033+0000 7f589a7fc640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f58840a2b80 0x7f58840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.033+0000 7f58a10f5640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f58840a2b80 0x7f58840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.033+0000 7f589a7fc640 1 -- 172.21.15.103:0/724294759 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f588c0efc80 con 0x7f589c1078e0 2024-08-19T01:35:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.033+0000 7f58a10f5640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f58840a2b80 0x7f58840a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5894009890 tx=0x7f5894008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.037+0000 7f589a7fc640 1 -- 172.21.15.103:0/724294759 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f588c0bcc10 con 0x7f589c1078e0 2024-08-19T01:35:05.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.205+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f589c109e70 con 0x7f589c1078e0 2024-08-19T01:35:05.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.207+0000 7f589a7fc640 1 -- 172.21.15.103:0/724294759 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22704 (secure 0 0 0) 0x7f588c0bc510 con 0x7f589c1078e0 2024-08-19T01:35:05.209 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:05.209 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:34:01.874407+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f58840a2b80 msgr2=0x7f58840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f58840a2b80 0x7f58840a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5894009890 tx=0x7f5894008040 comp rx=0 tx=0).stop 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c1078e0 msgr2=0x7f589c19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c1078e0 0x7f589c19c980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f588c009900 tx=0x7f588c004270 comp rx=0 tx=0).stop 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 shutdown_connections 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f58840a2b80 0x7f58840a5040 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f589c1078e0 0x7f589c19c980 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f589c102280 0x7f589c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 --2- 172.21.15.103:0/724294759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589c1018e0 0x7f589c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:05.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 >> 172.21.15.103:0/724294759 conn(0x7f589c0fd600 msgr2=0x7f589c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:05.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 shutdown_connections 2024-08-19T01:35:05.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:05.210+0000 7f58a3359640 1 -- 172.21.15.103:0/724294759 wait complete. 2024-08-19T01:35:05.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-19T01:33:23.991203+0000', 'modified': '2024-08-19T01:33:34.962843+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24485, 'mds_1': 24478, 'mds_2': 24466, 'mds_3': 24454, 'mds_4': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14580': {'gid': 14580, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/2158943857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2158943857}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2158943857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14592': {'gid': 14592, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/1914348964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 1914348964}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 1914348964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/1682379250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 1682379250}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 1682379250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/1574351198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 1574351198}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 1574351198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6835/3609998196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 3609998196}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 3609998196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.146:6837/3498679319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 3498679319}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 3498679319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.190:6835/794893065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 794893065}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 794893065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.190:6837/66432346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 66432346}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 66432346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.190:6839/639745782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 639745782}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 639745782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.146:6839/1128555603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 1128555603}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 1128555603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24485, 'qdb_cluster': [24485, 24466, 24454, 24478, 24452]} 2024-08-19T01:35:05.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-19T01:35:05.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-19T01:35:06.198 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:06.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:05 smithi146 ceph-mon[22098]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-19T01:35:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:05 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/724294759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:06.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:05 smithi103 ceph-mon[18436]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-19T01:35:06.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:05 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/724294759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:05 smithi190 ceph-mon[22032]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-08-19T01:35:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:05 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/724294759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:06.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 -- 172.21.15.103:0/1621305328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58100b50 msgr2=0x7fba58100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 --2- 172.21.15.103:0/1621305328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58100b50 0x7fba58100f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fba48009900 tx=0x7fba4802a480 comp rx=0 tx=0).stop 2024-08-19T01:35:06.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 -- 172.21.15.103:0/1621305328 shutdown_connections 2024-08-19T01:35:06.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 --2- 172.21.15.103:0/1621305328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba58102c20 0x7fba5810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 --2- 172.21.15.103:0/1621305328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba581022a0 0x7fba581026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 --2- 172.21.15.103:0/1621305328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58100b50 0x7fba58100f30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 -- 172.21.15.103:0/1621305328 >> 172.21.15.103:0/1621305328 conn(0x7fba580fc500 msgr2=0x7fba580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 -- 172.21.15.103:0/1621305328 shutdown_connections 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 -- 172.21.15.103:0/1621305328 wait complete. 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.681+0000 7fba5dca2640 1 Processor -- start 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba5dca2640 1 -- start start 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba5dca2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba58100b50 0x7fba581981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba5dca2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba581022a0 0x7fba581986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba5dca2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58102c20 0x7fba5819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba5dca2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba58111c80 con 0x7fba58102c20 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba5dca2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fba58111b00 con 0x7fba581022a0 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba5dca2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba58111e00 con 0x7fba58100b50 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58102c20 0x7fba5819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58102c20 0x7fba5819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58224/0 (socket says 172.21.15.103:58224) 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 -- 172.21.15.103:0/2542429098 learned_addr learned my addr 172.21.15.103:0/2542429098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:06.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba581022a0 0x7fba581986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba577fe640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba58100b50 0x7fba581981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 -- 172.21.15.103:0/2542429098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba58100b50 msgr2=0x7fba581981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba58100b50 0x7fba581981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 -- 172.21.15.103:0/2542429098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba581022a0 msgr2=0x7fba581986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba581022a0 0x7fba581986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 -- 172.21.15.103:0/2542429098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5819d150 con 0x7fba58102c20 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.682+0000 7fba57fff640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58102c20 0x7fba5819ca70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fba5000cad0 tx=0x7fba5000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.683+0000 7fba557fa640 1 -- 172.21.15.103:0/2542429098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5000ad60 con 0x7fba58102c20 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.683+0000 7fba557fa640 1 -- 172.21.15.103:0/2542429098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba50005020 con 0x7fba58102c20 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.683+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5819d440 con 0x7fba58102c20 2024-08-19T01:35:06.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.683+0000 7fba557fa640 1 -- 172.21.15.103:0/2542429098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5000f6b0 con 0x7fba58102c20 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.683+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba581a4d20 con 0x7fba58102c20 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba557fa640 1 -- 172.21.15.103:0/2542429098 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fba500051d0 con 0x7fba58102c20 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 Processor -- start 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 -- start start 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74d6630 0x5574a74d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba557fa640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba440a2b80 0x7fba440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74e1810 0x5574a74e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574a743b710 con 0x5574a74d6630 2024-08-19T01:35:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5574a73d3770 con 0x5574a74d6f50 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba600a0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5574a74376d0 con 0x5574a74e1810 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba4ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74d6630 0x5574a74d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba577fe640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba440a2b80 0x7fba440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.685+0000 7fba4ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74d6630 0x5574a74d6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58232/0 (socket says 172.21.15.103:58232) 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 -- 172.21.15.103:0/2037696109 learned_addr learned my addr 172.21.15.103:0/2037696109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba57fff640 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74e1810 0x5574a74e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba557fa640 1 -- 172.21.15.103:0/2542429098 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fba500cf0b0 con 0x7fba58102c20 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 -- 172.21.15.103:0/2037696109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74e1810 msgr2=0x5574a74e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74e1810 0x5574a74e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 -- 172.21.15.103:0/2037696109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 msgr2=0x5574a74e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 -- 172.21.15.103:0/2037696109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a73da3e0 con 0x5574a74d6630 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba577fe640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba440a2b80 0x7fba440a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fba48005ab0 tx=0x7fba48005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74d6630 0x5574a74d6a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba40006aa0 tx=0x7fba400095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567f0d5c8491918f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4dffb640 1 -- 172.21.15.103:0/2037696109 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40017610 con 0x5574a74d6630 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4dffb640 1 -- 172.21.15.103:0/2037696109 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba4001c070 con 0x5574a74d6630 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.686+0000 7fba4dffb640 1 -- 172.21.15.103:0/2037696109 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400158a0 con 0x5574a74d6630 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 -- 172.21.15.103:0/2037696109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74d6630 msgr2=0x5574a74d6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74d6630 0x5574a74d6a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba40006aa0 tx=0x7fba400095d0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 -- 172.21.15.103:0/2037696109 shutdown_connections 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74e1810 0x5574a74e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 --2- 172.21.15.103:0/2037696109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74d6630 0x5574a74d6a10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 -- 172.21.15.103:0/2037696109 >> 172.21.15.103:0/2037696109 conn(0x5574a74d2bc0 msgr2=0x5574a74d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 -- 172.21.15.103:0/2037696109 shutdown_connections 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 -- 172.21.15.103:0/2037696109 wait complete. 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 -- start start 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.687+0000 7fba600a0740 1 Processor -- start 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574a74e43e0 con 0x5574a74e1810 2024-08-19T01:35:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5574a73f6970 con 0x5574a74d6f50 2024-08-19T01:35:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5574a74e4560 con 0x5574a74d6630 2024-08-19T01:35:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 10 client.0 _ll_register_callbacks cb 0x5574a714a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:06.737 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:06.737 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:35:06.737 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24452, 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 92.763053768000006, 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:35:06.738 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 127.83136039 2024-08-19T01:35:06.739 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74df0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58244/0 (socket says 172.21.15.103:58244) 2024-08-19T01:35:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 -- 172.21.15.103:0/2898639241 learned_addr learned my addr 172.21.15.103:0/2898639241 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba577fe640 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 -- 172.21.15.103:0/2898639241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 msgr2=0x5574a74de670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74de670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 -- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 msgr2=0x5574a74debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 -- 172.21.15.103:0/2898639241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a71ad2b0 con 0x5574a74e1810 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba57fff640 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74df0f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fba5000a620 tx=0x7fba5001d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdc0682c4f0601a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba36ffd640 1 -- 172.21.15.103:0/2898639241 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500afe60 con 0x5574a74e1810 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba36ffd640 1 -- 172.21.15.103:0/2898639241 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fba5000c000 con 0x5574a74e1810 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.688+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba4001c070 con 0x5574a74e1810 2024-08-19T01:35:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba36ffd640 1 -- 172.21.15.103:0/2898639241 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba50097be0 con 0x5574a74e1810 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 10 client.15102 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574a74effd0 con 0x5574a74e1810 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba36ffd640 1 -- 172.21.15.103:0/2898639241 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba4001c070 con 0x5574a74e1810 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba36ffd640 1 -- 172.21.15.103:0/2898639241 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fba500c6330 con 0x5574a74e1810 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 10 client.15102 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 20 client.15102 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a74f25f0 0x5574a74f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 4 client.15102 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a7448c80 con 0x5574a74f25f0 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.689+0000 7fba577fe640 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a74f25f0 0x5574a74f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.690+0000 7fba577fe640 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a74f25f0 0x5574a74f3a60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86e582fe70626efb server_cookie=f738b7867c52b01a in_seq=0 out_seq=0 2024-08-19T01:35:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.690+0000 7fba36ffd640 10 client.15102 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.690+0000 7fba36ffd640 1 -- 172.21.15.103:0/2898639241 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574a7448c80 con 0x5574a74f25f0 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.690+0000 7fba36ffd640 10 client.15102 handle_command_reply: tid=0 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.690+0000 7fba600a0740 1 client.15102 shutdown 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 msgr2=0x5574a74df0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74df0f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fba5000a620 tx=0x7fba5001d3f0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 shutdown_connections 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a74f25f0 0x5574a74f3a60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74df0f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba36ffd640 0 client.15102 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 --2- 172.21.15.103:0/2898639241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74de670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 >> 172.21.15.103:0/2898639241 conn(0x5574a74d2bc0 msgr2=0x5574a74d88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 shutdown_connections 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 1 -- 172.21.15.103:0/2898639241 wait complete. 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.691+0000 7fba600a0740 20 client.15102 trim_cache size 0 max 16384 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 Processor -- start 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 -- start start 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74dd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574a74f1da0 con 0x5574a74e1810 2024-08-19T01:35:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5574a74f2470 con 0x5574a74d6f50 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba600a0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5574a74f2b40 con 0x5574a74d6630 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74dd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58248/0 (socket says 172.21.15.103:58248) 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.708+0000 7fba57fff640 1 -- 172.21.15.103:0/668539641 learned_addr learned my addr 172.21.15.103:0/668539641 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba4ffff640 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba577fe640 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba57fff640 1 -- 172.21.15.103:0/668539641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 msgr2=0x5574a74d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba57fff640 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba57fff640 1 -- 172.21.15.103:0/668539641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 msgr2=0x5574a74d99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba57fff640 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74d99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba57fff640 1 -- 172.21.15.103:0/668539641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a74d29f0 con 0x5574a74e1810 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba577fe640 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba57fff640 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74dd650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba5009b8d0 tx=0x7fba500c78d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbaf9645babd9db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba4d7fa640 1 -- 172.21.15.103:0/668539641 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500afe60 con 0x5574a74e1810 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba4d7fa640 1 -- 172.21.15.103:0/668539641 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba5009c040 con 0x5574a74e1810 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.709+0000 7fba4d7fa640 1 -- 172.21.15.103:0/668539641 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500860b0 con 0x5574a74e1810 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- 172.21.15.103:0/668539641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 msgr2=0x5574a74dd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74dd650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba5009b8d0 tx=0x7fba500c78d0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- 172.21.15.103:0/668539641 shutdown_connections 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74dd650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 --2- 172.21.15.103:0/668539641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- 172.21.15.103:0/668539641 >> 172.21.15.103:0/668539641 conn(0x5574a74d2bc0 msgr2=0x5574a74dcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- 172.21.15.103:0/668539641 shutdown_connections 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- 172.21.15.103:0/668539641 wait complete. 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- start start 2024-08-19T01:35:06.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 Processor -- start 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74fc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574a74f38e0 con 0x5574a74e1810 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5574a74f3210 con 0x5574a74d6f50 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5574a74f3fb0 con 0x5574a74d6630 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.710+0000 7fba600a0740 10 client.0 _ll_register_callbacks cb 0x7fba500c77d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58254/0 (socket says 172.21.15.103:58254) 2024-08-19T01:35:06.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 -- 172.21.15.103:0/2528344302 learned_addr learned my addr 172.21.15.103:0/2528344302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba577fe640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 -- 172.21.15.103:0/2528344302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 msgr2=0x5574a74fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74fc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 -- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 msgr2=0x5574a74f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 -- 172.21.15.103:0/2528344302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a74f6570 con 0x5574a74e1810 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba4ffff640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba57fff640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74f5d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba500c7940 tx=0x7fba500a45b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3304c3d4af97108e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba36ffd640 1 -- 172.21.15.103:0/2528344302 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50064070 con 0x5574a74e1810 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba36ffd640 1 -- 172.21.15.103:0/2528344302 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fba500862b0 con 0x5574a74e1810 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba5009c040 con 0x5574a74e1810 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba36ffd640 1 -- 172.21.15.103:0/2528344302 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba500afe60 con 0x5574a74e1810 2024-08-19T01:35:06.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba600a0740 10 client.15108 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5574a755cd00 con 0x5574a74e1810 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.711+0000 7fba36ffd640 1 -- 172.21.15.103:0/2528344302 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba5009c040 con 0x5574a74e1810 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba36ffd640 1 -- 172.21.15.103:0/2528344302 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fba5009a800 con 0x5574a74e1810 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba600a0740 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba600a0740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba600a0740 20 client.15108 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba600a0740 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a7561330 0x5574a7560040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba600a0740 4 client.15108 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "status"}) -- 0x5574a7443190 con 0x5574a7561330 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba577fe640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a7561330 0x5574a7560040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba577fe640 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a7561330 0x5574a7560040 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21b0a325208bbd8d server_cookie=9a46f03f65745aa6 in_seq=0 out_seq=0 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.712+0000 7fba36ffd640 10 client.15108 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba36ffd640 1 -- 172.21.15.103:0/2528344302 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5574a7443190 con 0x5574a7561330 2024-08-19T01:35:06.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba36ffd640 10 client.15108 handle_command_reply: tid=0 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 client.15108 shutdown 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 msgr2=0x5574a74f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74f5d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba500c7940 tx=0x7fba500a45b0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 shutdown_connections 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5574a7561330 0x5574a7560040 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574a74e1810 0x5574a74f5d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5574a74d6f50 0x5574a74f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba36ffd640 0 client.15108 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 --2- 172.21.15.103:0/2528344302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5574a74d6630 0x5574a74fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 >> 172.21.15.103:0/2528344302 conn(0x5574a74d2bc0 msgr2=0x5574a74dd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 shutdown_connections 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 1 -- 172.21.15.103:0/2528344302 wait complete. 2024-08-19T01:35:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.713+0000 7fba600a0740 20 client.15108 trim_cache size 0 max 16384 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba440a2b80 msgr2=0x7fba440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba440a2b80 0x7fba440a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fba48005ab0 tx=0x7fba48005a20 comp rx=0 tx=0).stop 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58102c20 msgr2=0x7fba5819ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58102c20 0x7fba5819ca70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fba5000cad0 tx=0x7fba5000ced0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 shutdown_connections 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba440a2b80 0x7fba440a5040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba58102c20 0x7fba5819ca70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba581022a0 0x7fba581986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 --2- 172.21.15.103:0/2542429098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba58100b50 0x7fba581981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 >> 172.21.15.103:0/2542429098 conn(0x7fba580fc500 msgr2=0x7fba580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 shutdown_connections 2024-08-19T01:35:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:06.714+0000 7fba5dca2640 1 -- 172.21.15.103:0/2542429098 wait complete. 2024-08-19T01:35:07.504 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.f status 2024-08-19T01:35:07.758 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:08.187 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:07 smithi103 ceph-mon[18436]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:08.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:07 smithi146 ceph-mon[22098]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 -- 172.21.15.103:0/2368166461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14102a60 msgr2=0x7f9d14102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2368166461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14102a60 0x7f9d14102e40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009930 tx=0x7f9d0402a420 comp rx=0 tx=0).stop 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 -- 172.21.15.103:0/2368166461 shutdown_connections 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2368166461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14107fd0 0x7f9d1410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2368166461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d14103410 0x7f9d141078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2368166461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14102a60 0x7f9d14102e40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 -- 172.21.15.103:0/2368166461 >> 172.21.15.103:0/2368166461 conn(0x7f9d140fc500 msgr2=0x7f9d140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 -- 172.21.15.103:0/2368166461 shutdown_connections 2024-08-19T01:35:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 -- 172.21.15.103:0/2368166461 wait complete. 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 Processor -- start 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.268+0000 7f9d19d20640 1 -- start start 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14102a60 0x7f9d14198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d14103410 0x7f9d141986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14107fd0 0x7f9d1419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14111c80 con 0x7f9d14107fd0 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14111b00 con 0x7f9d14102a60 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14111e00 con 0x7f9d14103410 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14107fd0 0x7f9d1419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14107fd0 0x7f9d1419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58274/0 (socket says 172.21.15.103:58274) 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 -- 172.21.15.103:0/2618476994 learned_addr learned my addr 172.21.15.103:0/2618476994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d137fe640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14102a60 0x7f9d14198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 -- 172.21.15.103:0/2618476994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d14103410 msgr2=0x7f9d141986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d14103410 0x7f9d141986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 -- 172.21.15.103:0/2618476994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14102a60 msgr2=0x7f9d14198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14102a60 0x7f9d14198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 -- 172.21.15.103:0/2618476994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1419d120 con 0x7f9d14107fd0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d137fe640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14102a60 0x7f9d14198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d13fff640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14107fd0 0x7f9d1419ca40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009fd0 tx=0x7f9d0c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d10ff9640 1 -- 172.21.15.103:0/2618476994 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c017070 con 0x7f9d14107fd0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d10ff9640 1 -- 172.21.15.103:0/2618476994 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d0c003e20 con 0x7f9d14107fd0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1419d410 con 0x7f9d14107fd0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d10ff9640 1 -- 172.21.15.103:0/2618476994 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c010460 con 0x7f9d14107fd0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.269+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141a4cf0 con 0x7f9d14107fd0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.271+0000 7f9d10ff9640 1 -- 172.21.15.103:0/2618476994 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9d0c003970 con 0x7f9d14107fd0 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 Processor -- start 2024-08-19T01:35:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 -- start start 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43630 0x564a0de43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de4e810 0x564a0de50c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a0dda8710 con 0x564a0de43f50 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564a0dd40770 con 0x564a0de43630 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564a0dda46d0 con 0x564a0de4e810 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58276/0 (socket says 172.21.15.103:58276) 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d137fe640 1 -- 172.21.15.103:0/1613908201 learned_addr learned my addr 172.21.15.103:0/1613908201 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d10ff9640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9d000a2b80 0x7f9d000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43630 0x564a0de43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de4e810 0x564a0de50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.272+0000 7f9d13fff640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9d000a2b80 0x7f9d000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d137fe640 1 -- 172.21.15.103:0/1613908201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de4e810 msgr2=0x564a0de50c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de4e810 0x564a0de50c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d137fe640 1 -- 172.21.15.103:0/1613908201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43630 msgr2=0x564a0de43a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43630 0x564a0de43a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d137fe640 1 -- 172.21.15.103:0/1613908201 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0dd473e0 con 0x564a0de43f50 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d10ff9640 1 -- 172.21.15.103:0/2618476994 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9d0c0ceec0 con 0x7f9d14107fd0 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4e2d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9d04031440 tx=0x7f9d04032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9deb6a751ddae1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d13fff640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9d000a2b80 0x7f9d000a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c09b450 tx=0x7f9d0c0c62e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d09ffb640 1 -- 172.21.15.103:0/1613908201 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0403f070 con 0x564a0de43f50 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d09ffb640 1 -- 172.21.15.103:0/1613908201 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d04031760 con 0x564a0de43f50 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.273+0000 7f9d09ffb640 1 -- 172.21.15.103:0/1613908201 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04035070 con 0x564a0de43f50 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1613908201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 msgr2=0x564a0de4e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4e2d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9d04031440 tx=0x7f9d04032e70 comp rx=0 tx=0).stop 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1613908201 shutdown_connections 2024-08-19T01:35:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de4e810 0x564a0de50c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4e2d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1613908201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43630 0x564a0de43a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1613908201 >> 172.21.15.103:0/1613908201 conn(0x564a0de3fbc0 msgr2=0x564a0de3ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1613908201 shutdown_connections 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1613908201 wait complete. 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 -- start start 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.274+0000 7f9d1c11e740 1 Processor -- start 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de4b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de4c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a0de513e0 con 0x564a0de43f50 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564a0dd63970 con 0x564a0de4e810 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564a0de51560 con 0x564a0de43630 2024-08-19T01:35:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 10 client.0 _ll_register_callbacks cb 0x564a0dab77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58284/0 (socket says 172.21.15.103:58284) 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 -- 172.21.15.103:0/1928768681 learned_addr learned my addr 172.21.15.103:0/1928768681 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de4b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 -- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 msgr2=0x564a0de4b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de4b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 -- 172.21.15.103:0/1928768681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 msgr2=0x564a0de4c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de4c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 -- 172.21.15.103:0/1928768681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0db1a2b0 con 0x564a0de43f50 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de4b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4bcd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9d04005450 tx=0x7f9d0403cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511e420c6e8cd6eb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1928768681 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0403f070 con 0x564a0de43f50 2024-08-19T01:35:08.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1928768681 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9d0403cd00 con 0x564a0de43f50 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.275+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d04031760 con 0x564a0de43f50 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1928768681 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d04035040 con 0x564a0de43f50 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 10 client.15120 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564a0de5d090 con 0x564a0de43f50 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1928768681 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d04031760 con 0x564a0de43f50 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1928768681 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f9d04003710 con 0x564a0de43f50 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 10 client.15120 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 10 client.15120 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 20 client.15120 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0de5f190 0x564a0de60600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 4 client.15120 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0ddb5c80 con 0x564a0de5f190 2024-08-19T01:35:08.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.276+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0de5f190 0x564a0de60600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.277+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0de5f190 0x564a0de60600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1eddea01fae50be4 server_cookie=86278df085136e2c in_seq=0 out_seq=0 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.277+0000 7f9cf2ffd640 10 client.15120 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.277+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1928768681 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a0ddb5c80 con 0x564a0de5f190 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.277+0000 7f9cf2ffd640 10 client.15120 handle_command_reply: tid=0 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 client.15120 shutdown 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 msgr2=0x564a0de4bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4bcd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9d04005450 tx=0x7f9d0403cae0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 shutdown_connections 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0de5f190 0x564a0de60600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de4c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de4bcd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9cf2ffd640 0 client.15120 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1928768681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de4b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 >> 172.21.15.103:0/1928768681 conn(0x564a0de3fbc0 msgr2=0x564a0de3ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 shutdown_connections 2024-08-19T01:35:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1928768681 wait complete. 2024-08-19T01:35:08.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.278+0000 7f9d1c11e740 20 client.15120 trim_cache size 0 max 16384 2024-08-19T01:35:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 Processor -- start 2024-08-19T01:35:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 -- start start 2024-08-19T01:35:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de5f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43f50 0x564a0de46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de4e810 0x564a0de4c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a0de5e940 con 0x564a0de4e810 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564a0de5f010 con 0x564a0de43f50 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564a0de5f6e0 con 0x564a0de43630 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de4e810 0x564a0de4c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de4e810 0x564a0de4c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58294/0 (socket says 172.21.15.103:58294) 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 -- 172.21.15.103:0/3126330551 learned_addr learned my addr 172.21.15.103:0/3126330551 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d137fe640 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43f50 0x564a0de46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 -- 172.21.15.103:0/3126330551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 msgr2=0x564a0de5f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de5f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 -- 172.21.15.103:0/3126330551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43f50 msgr2=0x564a0de46060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43f50 0x564a0de46060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.296+0000 7f9d12ffd640 1 -- 172.21.15.103:0/3126330551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0de4a740 con 0x564a0de4e810 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de4e810 0x564a0de4c060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d140feba0 tx=0x7f9cfc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f76254a8d3d9133 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d097fa640 1 -- 172.21.15.103:0/3126330551 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc005220 con 0x564a0de4e810 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d097fa640 1 -- 172.21.15.103:0/3126330551 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cfc004c50 con 0x564a0de4e810 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d097fa640 1 -- 172.21.15.103:0/3126330551 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0127c0 con 0x564a0de4e810 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d1c11e740 1 -- 172.21.15.103:0/3126330551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de4e810 msgr2=0x564a0de4c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de4e810 0x564a0de4c060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d140feba0 tx=0x7f9cfc00c990 comp rx=0 tx=0).stop 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d1c11e740 1 -- 172.21.15.103:0/3126330551 shutdown_connections 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de4e810 0x564a0de4c060 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de43f50 0x564a0de46060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/3126330551 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de5f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.297+0000 7f9d1c11e740 1 -- 172.21.15.103:0/3126330551 >> 172.21.15.103:0/3126330551 conn(0x564a0de3fbc0 msgr2=0x564a0de4cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 -- 172.21.15.103:0/3126330551 shutdown_connections 2024-08-19T01:35:08.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 -- 172.21.15.103:0/3126330551 wait complete. 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 -- start start 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 Processor -- start 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de62a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de62f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a0de60480 con 0x564a0de43f50 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564a0de5fdb0 con 0x564a0de4e810 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564a0de60b50 con 0x564a0de43630 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d1c11e740 10 client.0 _ll_register_callbacks cb 0x7f9cfc00c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de62f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de62f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58302/0 (socket says 172.21.15.103:58302) 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d137fe640 1 -- 172.21.15.103:0/1595363085 learned_addr learned my addr 172.21.15.103:0/1595363085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.298+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de62a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d137fe640 1 -- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 msgr2=0x564a0de62a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de62a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d137fe640 1 -- 172.21.15.103:0/1595363085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 msgr2=0x564a0de67210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de67210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d137fe640 1 -- 172.21.15.103:0/1595363085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0de67750 con 0x564a0de43f50 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de67210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d137fe640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de62f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9d04003ae0 tx=0x7f9d04031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96207b6c04bab3f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1595363085 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0403f070 con 0x564a0de43f50 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1595363085 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9d0403cb50 con 0x564a0de43f50 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cfc004c50 con 0x564a0de43f50 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1595363085 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d0403cd50 con 0x564a0de43f50 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d1c11e740 10 client.15126 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:08.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564a0deca000 con 0x564a0de43f50 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.299+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1595363085 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cfc004c50 con 0x564a0de43f50 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1595363085 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f9d04003fa0 con 0x564a0de43f50 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d1c11e740 10 client.15126 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d1c11e740 10 client.15126 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d1c11e740 20 client.15126 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0deceb40 0x564a0decd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d1c11e740 4 client.15126 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "status"}) -- 0x564a0ddb0190 con 0x564a0deceb40 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0deceb40 0x564a0decd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9cf2ffd640 10 client.15126 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.300+0000 7f9d12ffd640 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0deceb40 0x564a0decd850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de61f20a672ecb22 server_cookie=29bca0ca16ce8680 in_seq=0 out_seq=0 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9cf2ffd640 1 -- 172.21.15.103:0/1595363085 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564a0ddb0190 con 0x564a0deceb40 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9cf2ffd640 10 client.15126 handle_command_reply: tid=0 2024-08-19T01:35:08.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 client.15126 shutdown 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 msgr2=0x564a0de62f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de62f60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9d04003ae0 tx=0x7f9d04031810 comp rx=0 tx=0).stop 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 shutdown_connections 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x564a0deceb40 0x564a0decd850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564a0de4e810 0x564a0de67210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a0de43f50 0x564a0de62f60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 --2- 172.21.15.103:0/1595363085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a0de43630 0x564a0de62a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 >> 172.21.15.103:0/1595363085 conn(0x564a0de3fbc0 msgr2=0x564a0de59fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9cf2ffd640 0 client.15126 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 shutdown_connections 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 1 -- 172.21.15.103:0/1595363085 wait complete. 2024-08-19T01:35:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d1c11e740 20 client.15126 trim_cache size 0 max 16384 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 95.358888710000002, 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:35:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 134.82236611600001 2024-08-19T01:35:08.317 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:08.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9d000a2b80 msgr2=0x7f9d000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9d000a2b80 0x7f9d000a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c09b450 tx=0x7f9d0c0c62e0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14107fd0 msgr2=0x7f9d1419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:08.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.301+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14107fd0 0x7f9d1419ca40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009fd0 tx=0x7f9d0c00b260 comp rx=0 tx=0).stop 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 shutdown_connections 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9d000a2b80 0x7f9d000a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d14107fd0 0x7f9d1419ca40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d14103410 0x7f9d141986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 --2- 172.21.15.103:0/2618476994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d14102a60 0x7f9d14198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 >> 172.21.15.103:0/2618476994 conn(0x7f9d140fc500 msgr2=0x7f9d14105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 shutdown_connections 2024-08-19T01:35:08.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:08.302+0000 7f9d19d20640 1 -- 172.21.15.103:0/2618476994 wait complete. 2024-08-19T01:35:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:07 smithi190 ceph-mon[22032]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:09.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.c status 2024-08-19T01:35:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:09.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 -- 172.21.15.103:0/3139682958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 msgr2=0x7f022010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 --2- 172.21.15.103:0/3139682958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022010e970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0218009cd0 tx=0x7f021802a910 comp rx=0 tx=0).stop 2024-08-19T01:35:09.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 -- 172.21.15.103:0/3139682958 shutdown_connections 2024-08-19T01:35:09.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 --2- 172.21.15.103:0/3139682958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022010e970 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 --2- 172.21.15.103:0/3139682958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0220103410 0x7f02201078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 --2- 172.21.15.103:0/3139682958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0220102a60 0x7f0220102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 -- 172.21.15.103:0/3139682958 >> 172.21.15.103:0/3139682958 conn(0x7f02200fc500 msgr2=0x7f02200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:09.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 -- 172.21.15.103:0/3139682958 shutdown_connections 2024-08-19T01:35:09.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.844+0000 7f0224e6f640 1 -- 172.21.15.103:0/3139682958 wait complete. 2024-08-19T01:35:09.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 Processor -- start 2024-08-19T01:35:09.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 -- start start 2024-08-19T01:35:09.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0220102a60 0x7f0220198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0220103410 0x7f02201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0220111d40 con 0x7f0220107fd0 2024-08-19T01:35:09.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0220111bc0 con 0x7f0220103410 2024-08-19T01:35:09.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0224e6f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0220111ec0 con 0x7f0220102a60 2024-08-19T01:35:09.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58314/0 (socket says 172.21.15.103:58314) 2024-08-19T01:35:09.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 -- 172.21.15.103:0/612947702 learned_addr learned my addr 172.21.15.103:0/612947702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:09.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f0217fff640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0220103410 0x7f02201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021e59c640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0220102a60 0x7f0220198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 -- 172.21.15.103:0/612947702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0220102a60 msgr2=0x7f0220198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0220102a60 0x7f0220198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 -- 172.21.15.103:0/612947702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0220103410 msgr2=0x7f02201988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0220103410 0x7f02201988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.845+0000 7f021ed9d640 1 -- 172.21.15.103:0/612947702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022019d340 con 0x7f0220107fd0 2024-08-19T01:35:09.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.846+0000 7f021ed9d640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022019cc60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0218009df0 tx=0x7f0218003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:09.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.846+0000 7f02177fe640 1 -- 172.21.15.103:0/612947702 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218004040 con 0x7f0220107fd0 2024-08-19T01:35:09.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.846+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022019d5d0 con 0x7f0220107fd0 2024-08-19T01:35:09.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.846+0000 7f02177fe640 1 -- 172.21.15.103:0/612947702 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0218004690 con 0x7f0220107fd0 2024-08-19T01:35:09.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.846+0000 7f02177fe640 1 -- 172.21.15.103:0/612947702 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218030ea0 con 0x7f0220107fd0 2024-08-19T01:35:09.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.846+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02201a4f10 con 0x7f0220107fd0 2024-08-19T01:35:09.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.848+0000 7f02177fe640 1 -- 172.21.15.103:0/612947702 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f02180305d0 con 0x7f0220107fd0 2024-08-19T01:35:09.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 Processor -- start 2024-08-19T01:35:09.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 -- start start 2024-08-19T01:35:09.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f02177fe640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f020c0a2b80 0x7f020c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c2f550 0x55d5d5c2f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5b94710 con 0x55d5d5c2f550 2024-08-19T01:35:09.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5b2c770 con 0x55d5d5c2fe70 2024-08-19T01:35:09.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f022726d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5b906d0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c2f550 0x55d5d5c2f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c2f550 0x55d5d5c2f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58320/0 (socket says 172.21.15.103:58320) 2024-08-19T01:35:09.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f0217fff640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f020c0a2b80 0x7f020c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 -- 172.21.15.103:0/3895691990 learned_addr learned my addr 172.21.15.103:0/3895691990 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:09.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f02177fe640 1 -- 172.21.15.103:0/612947702 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f02180f0840 con 0x7f0220107fd0 2024-08-19T01:35:09.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f0217fff640 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021ed9d640 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 -- 172.21.15.103:0/3895691990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c3ec20 msgr2=0x55d5d5c41010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c41010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 -- 172.21.15.103:0/3895691990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 msgr2=0x55d5d5c3e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.849+0000 7f021e59c640 1 -- 172.21.15.103:0/3895691990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d5b333e0 con 0x55d5d5c2f550 2024-08-19T01:35:09.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.850+0000 7f0217fff640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f020c0a2b80 0x7f020c0a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0208006e10 tx=0x7f0208009490 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:09.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.850+0000 7f021e59c640 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c2f550 0x55d5d5c2f930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f021000b820 tx=0x7f021000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4069871ea05062 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:09.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.850+0000 7f0214ff9640 1 -- 172.21.15.103:0/3895691990 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021000d950 con 0x55d5d5c2f550 2024-08-19T01:35:09.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.850+0000 7f0214ff9640 1 -- 172.21.15.103:0/3895691990 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0210019070 con 0x55d5d5c2f550 2024-08-19T01:35:09.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.850+0000 7f0214ff9640 1 -- 172.21.15.103:0/3895691990 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02100123e0 con 0x55d5d5c2f550 2024-08-19T01:35:09.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.850+0000 7f022726d740 1 -- 172.21.15.103:0/3895691990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c2f550 msgr2=0x55d5d5c2f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c2f550 0x55d5d5c2f930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f021000b820 tx=0x7f021000bce0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- 172.21.15.103:0/3895691990 shutdown_connections 2024-08-19T01:35:09.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 --2- 172.21.15.103:0/3895691990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c2f550 0x55d5d5c2f930 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- 172.21.15.103:0/3895691990 >> 172.21.15.103:0/3895691990 conn(0x55d5d5c2bbc0 msgr2=0x55d5d5c2bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:09.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- 172.21.15.103:0/3895691990 shutdown_connections 2024-08-19T01:35:09.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- 172.21.15.103:0/3895691990 wait complete. 2024-08-19T01:35:09.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- start start 2024-08-19T01:35:09.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 Processor -- start 2024-08-19T01:35:09.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c3bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c3c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c41790 con 0x55d5d5c3ec20 2024-08-19T01:35:09.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5b4f970 con 0x55d5d5c2f550 2024-08-19T01:35:09.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c41910 con 0x55d5d5c2fe70 2024-08-19T01:35:09.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f022726d740 10 client.0 _ll_register_callbacks cb 0x55d5d58a37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:09.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c3c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c3c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58330/0 (socket says 172.21.15.103:58330) 2024-08-19T01:35:09.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.851+0000 7f021ed9d640 1 -- 172.21.15.103:0/2693242722 learned_addr learned my addr 172.21.15.103:0/2693242722 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:09.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f0217fff640 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f021e59c640 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c3bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f021ed9d640 1 -- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 msgr2=0x55d5d5c3c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f021ed9d640 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f021ed9d640 1 -- 172.21.15.103:0/2693242722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 msgr2=0x55d5d5c3bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f021ed9d640 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c3bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f021ed9d640 1 -- 172.21.15.103:0/2693242722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d59062b0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f021ed9d640 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c3c5a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f02180ee6e0 tx=0x7f02180bcb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752c0eedea61db3b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f0205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f0205ffb640 1 -- 172.21.15.103:0/2693242722 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180098d0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f0205ffb640 1 -- 172.21.15.103:0/2693242722 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f02180bb490 con 0x55d5d5c3ec20 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0210019070 con 0x55d5d5c3ec20 2024-08-19T01:35:09.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f0205ffb640 1 -- 172.21.15.103:0/2693242722 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02180bd890 con 0x55d5d5c3ec20 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f022726d740 10 client.15138 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5d5c4d3f0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f0205ffb640 1 -- 172.21.15.103:0/2693242722 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0210019070 con 0x55d5d5c3ec20 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f0205ffb640 1 -- 172.21.15.103:0/2693242722 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f02180b9800 con 0x55d5d5c3ec20 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.852+0000 7f022726d740 10 client.15138 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f022726d740 10 client.15138 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f022726d740 20 client.15138 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f022726d740 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5c4fad0 0x55d5d5c50f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f022726d740 4 client.15138 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5d5ba1c80 con 0x55d5d5c4fad0 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f021e59c640 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5c4fad0 0x55d5d5c50f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f021e59c640 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5c4fad0 0x55d5d5c50f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9dc4b0054127004 server_cookie=a8d0ba639bb9c75 in_seq=0 out_seq=0 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.853+0000 7f0205ffb640 10 client.15138 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f0205ffb640 1 -- 172.21.15.103:0/2693242722 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5d5ba1c80 con 0x55d5d5c4fad0 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f0205ffb640 10 client.15138 handle_command_reply: tid=0 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 client.15138 shutdown 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 msgr2=0x55d5d5c3c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c3c5a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f02180ee6e0 tx=0x7f02180bcb70 comp rx=0 tx=0).stop 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 shutdown_connections 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5c4fad0 0x55d5d5c50f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c3c5a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f0205ffb640 0 client.15138 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 --2- 172.21.15.103:0/2693242722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c3bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 >> 172.21.15.103:0/2693242722 conn(0x55d5d5c2bbc0 msgr2=0x55d5d5c2ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 shutdown_connections 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 1 -- 172.21.15.103:0/2693242722 wait complete. 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.854+0000 7f022726d740 20 client.15138 trim_cache size 0 max 16384 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 Processor -- start 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 -- start start 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2f550 0x55d5d5c50870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c38d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c4f280 con 0x55d5d5c3ec20 2024-08-19T01:35:09.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c4f950 con 0x55d5d5c2fe70 2024-08-19T01:35:09.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f022726d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c50020 con 0x55d5d5c2f550 2024-08-19T01:35:09.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c38d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c38d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58344/0 (socket says 172.21.15.103:58344) 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.872+0000 7f021ed9d640 1 -- 172.21.15.103:0/3091135940 learned_addr learned my addr 172.21.15.103:0/3091135940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f021ed9d640 1 -- 172.21.15.103:0/3091135940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2f550 msgr2=0x55d5d5c50870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f021ed9d640 1 --2- 172.21.15.103:0/3091135940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2f550 0x55d5d5c50870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f021ed9d640 1 -- 172.21.15.103:0/3091135940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 msgr2=0x55d5d5c3c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f021ed9d640 1 --2- 172.21.15.103:0/3091135940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f021ed9d640 1 -- 172.21.15.103:0/3091135940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5d5c2fca0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f021ed9d640 1 --2- 172.21.15.103:0/3091135940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c38d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0218030990 tx=0x7f021803f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb0b20fca36243 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f0207fff640 1 -- 172.21.15.103:0/3091135940 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180098d0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f0207fff640 1 -- 172.21.15.103:0/3091135940 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0218086070 con 0x55d5d5c3ec20 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f0207fff640 1 -- 172.21.15.103:0/3091135940 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180c5e00 con 0x55d5d5c3ec20 2024-08-19T01:35:09.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f022726d740 1 -- 172.21.15.103:0/3091135940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 msgr2=0x55d5d5c38d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.873+0000 7f022726d740 1 --2- 172.21.15.103:0/3091135940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c38d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0218030990 tx=0x7f021803f380 comp rx=0 tx=0).stop 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- 172.21.15.103:0/3091135940 shutdown_connections 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 --2- 172.21.15.103:0/3091135940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c38d80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 --2- 172.21.15.103:0/3091135940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c3c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 --2- 172.21.15.103:0/3091135940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2f550 0x55d5d5c50870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- 172.21.15.103:0/3091135940 >> 172.21.15.103:0/3091135940 conn(0x55d5d5c2bbc0 msgr2=0x55d5d5c3ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- 172.21.15.103:0/3091135940 shutdown_connections 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- 172.21.15.103:0/3091135940 wait complete. 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- start start 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 Processor -- start 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c52ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c53220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c50dc0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c506f0 con 0x55d5d5c2f550 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d5d5c51490 con 0x55d5d5c2fe70 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f022726d740 10 client.0 _ll_register_callbacks cb 0x7f02180bccd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c53220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f021ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c53220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58356/0 (socket says 172.21.15.103:58356) 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f021ed9d640 1 -- 172.21.15.103:0/1914688899 learned_addr learned my addr 172.21.15.103:0/1914688899 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f021e59c640 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.874+0000 7f0217fff640 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c52ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f021ed9d640 1 -- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 msgr2=0x55d5d5c52ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f021ed9d640 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c52ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f021ed9d640 1 -- 172.21.15.103:0/1914688899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 msgr2=0x55d5d5c599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f021ed9d640 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f021ed9d640 1 -- 172.21.15.103:0/1914688899 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5d5c53a50 con 0x55d5d5c3ec20 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f021ed9d640 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c53220 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f02180bd120 tx=0x7f02180bd080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f37c530503d6fe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f0205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f0205ffb640 1 -- 172.21.15.103:0/1914688899 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180098d0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f0205ffb640 1 -- 172.21.15.103:0/1914688899 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f02180c1120 con 0x55d5d5c3ec20 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0218086070 con 0x55d5d5c3ec20 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f0205ffb640 1 -- 172.21.15.103:0/1914688899 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f02180bd190 con 0x55d5d5c3ec20 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f022726d740 10 client.15144 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:09.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5d5c53a50 con 0x55d5d5c3ec20 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f0205ffb640 1 -- 172.21.15.103:0/1914688899 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0218086070 con 0x55d5d5c3ec20 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.875+0000 7f0205ffb640 1 -- 172.21.15.103:0/1914688899 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f02180bd3f0 con 0x55d5d5c3ec20 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f022726d740 10 client.15144 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f022726d740 10 client.15144 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f022726d740 20 client.15144 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f022726d740 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5cbe590 0x55d5d5cbd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f022726d740 4 client.15144 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "status"}) -- 0x55d5d5b9c190 con 0x55d5d5cbe590 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f021e59c640 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5cbe590 0x55d5d5cbd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f0205ffb640 10 client.15144 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.876+0000 7f021e59c640 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5cbe590 0x55d5d5cbd2a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dae399b6bd7684ea server_cookie=1612ac51ef96f03 in_seq=0 out_seq=0 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0205ffb640 1 -- 172.21.15.103:0/1914688899 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d5d5b9c190 con 0x55d5d5cbe590 2024-08-19T01:35:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0205ffb640 10 client.15144 handle_command_reply: tid=0 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 client.15144 shutdown 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 msgr2=0x55d5d5c53220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c53220 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f02180bd120 tx=0x7f02180bd080 comp rx=0 tx=0).stop 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 shutdown_connections 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55d5d5cbe590 0x55d5d5cbd2a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0205ffb640 0 client.15144 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5d5c3ec20 0x55d5d5c53220 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d5d5c2fe70 0x55d5d5c52ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 --2- 172.21.15.103:0/1914688899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d5d5c2f550 0x55d5d5c599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 >> 172.21.15.103:0/1914688899 conn(0x55d5d5c2bbc0 msgr2=0x55d5d5c49ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:09.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 shutdown_connections 2024-08-19T01:35:09.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 1 -- 172.21.15.103:0/1914688899 wait complete. 2024-08-19T01:35:09.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f022726d740 20 client.15144 trim_cache size 0 max 16384 2024-08-19T01:35:09.911 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:09.911 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:35:09.911 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 97.940520045, 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 133.66748874000001 2024-08-19T01:35:09.912 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f020c0a2b80 msgr2=0x7f020c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f020c0a2b80 0x7f020c0a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0208006e10 tx=0x7f0208009490 comp rx=0 tx=0).stop 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 msgr2=0x7f022019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022019cc60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0218009df0 tx=0x7f0218003ba0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 shutdown_connections 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f020c0a2b80 0x7f020c0a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0220107fd0 0x7f022019cc60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0220103410 0x7f02201988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 --2- 172.21.15.103:0/612947702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0220102a60 0x7f0220198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:09.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.877+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 >> 172.21.15.103:0/612947702 conn(0x7f02200fc500 msgr2=0x7f0220105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:09.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.878+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 shutdown_connections 2024-08-19T01:35:09.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:09.878+0000 7f0224e6f640 1 -- 172.21.15.103:0/612947702 wait complete. 2024-08-19T01:35:10.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:09 smithi103 ceph-mon[18436]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-19T01:35:10.183 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:09 smithi190 ceph-mon[22032]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-19T01:35:10.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:09 smithi146 ceph-mon[22098]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-19T01:35:10.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-19T01:35:10.985 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:11.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 -- 172.21.15.103:0/1160765195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 msgr2=0x7f5504106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 --2- 172.21.15.103:0/1160765195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f5504106b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009930 tx=0x7f54f802a420 comp rx=0 tx=0).stop 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 -- 172.21.15.103:0/1160765195 shutdown_connections 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 --2- 172.21.15.103:0/1160765195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5504101150 0x7f550410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 --2- 172.21.15.103:0/1160765195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55041007b0 0x7f5504100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 --2- 172.21.15.103:0/1160765195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f5504106b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 -- 172.21.15.103:0/1160765195 >> 172.21.15.103:0/1160765195 conn(0x7f55040fc4d0 msgr2=0x7f55040fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 -- 172.21.15.103:0/1160765195 shutdown_connections 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.513+0000 7f5508893640 1 -- 172.21.15.103:0/1160765195 wait complete. 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 Processor -- start 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 -- start start 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55041007b0 0x7f5504198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5504101150 0x7f5504198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f550419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111c10 con 0x7f55041067b0 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111a90 con 0x7f5504101150 2024-08-19T01:35:11.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5508893640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111d90 con 0x7f55041007b0 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5502d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f550419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5502d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f550419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58368/0 (socket says 172.21.15.103:58368) 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5502d9d640 1 -- 172.21.15.103:0/4067204346 learned_addr learned my addr 172.21.15.103:0/4067204346 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f5501d9b640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5504101150 0x7f5504198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.514+0000 7f550259c640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55041007b0 0x7f5504198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5502d9d640 1 -- 172.21.15.103:0/4067204346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55041007b0 msgr2=0x7f5504198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5502d9d640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55041007b0 0x7f5504198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5502d9d640 1 -- 172.21.15.103:0/4067204346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5504101150 msgr2=0x7f5504198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5502d9d640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5504101150 0x7f5504198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5502d9d640 1 -- 172.21.15.103:0/4067204346 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550419d1e0 con 0x7f55041067b0 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5502d9d640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f550419cb00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00b970 tx=0x7f54fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f54f37fe640 1 -- 172.21.15.103:0/4067204346 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc019070 con 0x7f55041067b0 2024-08-19T01:35:11.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f54f37fe640 1 -- 172.21.15.103:0/4067204346 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54fc014440 con 0x7f55041067b0 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550419d4d0 con 0x7f55041067b0 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f54f37fe640 1 -- 172.21.15.103:0/4067204346 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc0075d0 con 0x7f55041067b0 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.515+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55041a4db0 con 0x7f55041067b0 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.517+0000 7f54f37fe640 1 -- 172.21.15.103:0/4067204346 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f54fc01c070 con 0x7f55041067b0 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.517+0000 7f550ac91740 1 Processor -- start 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550ac91740 1 -- start start 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550ac91740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d6ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550ac91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550ac91740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d79810 0x558de9d7bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550ac91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558de9cd3710 con 0x558de9d6ef50 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550ac91740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558de9c6b770 con 0x558de9d79810 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550ac91740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558de9ccf6d0 con 0x558de9d6e630 2024-08-19T01:35:11.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f54f37fe640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f54ec0a2b80 0x7f54ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58382/0 (socket says 172.21.15.103:58382) 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 -- 172.21.15.103:0/2829028591 learned_addr learned my addr 172.21.15.103:0/2829028591 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f5502d9d640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f54ec0a2b80 0x7f54ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f5501d9b640 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d6ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f5501d9b640 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d79810 0x558de9d7bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f54f37fe640 1 -- 172.21.15.103:0/4067204346 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f54fc0ce580 con 0x7f55041067b0 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 -- 172.21.15.103:0/2829028591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 msgr2=0x558de9d6ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d6ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 -- 172.21.15.103:0/2829028591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d79810 msgr2=0x558de9d7bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d79810 0x558de9d7bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f550259c640 1 -- 172.21.15.103:0/2829028591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de9c723e0 con 0x558de9d6ef50 2024-08-19T01:35:11.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.518+0000 7f5501d9b640 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550259c640 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d792d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009930 tx=0x7f54f8032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f31b6f214fe7241 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f5502d9d640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f54ec0a2b80 0x7f54ec0a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0a34f0 tx=0x7f54fc0c5940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f54f0ff9640 1 -- 172.21.15.103:0/2829028591 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8035040 con 0x558de9d6ef50 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f54f0ff9640 1 -- 172.21.15.103:0/2829028591 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f80316a0 con 0x558de9d6ef50 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f54f0ff9640 1 -- 172.21.15.103:0/2829028591 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80047a0 con 0x558de9d6ef50 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550ac91740 1 -- 172.21.15.103:0/2829028591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 msgr2=0x558de9d792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550ac91740 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d792d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009930 tx=0x7f54f8032d60 comp rx=0 tx=0).stop 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550ac91740 1 -- 172.21.15.103:0/2829028591 shutdown_connections 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550ac91740 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d79810 0x558de9d7bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550ac91740 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d792d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550ac91740 1 --2- 172.21.15.103:0/2829028591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.519+0000 7f550ac91740 1 -- 172.21.15.103:0/2829028591 >> 172.21.15.103:0/2829028591 conn(0x558de9d6abc0 msgr2=0x558de9d6afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 -- 172.21.15.103:0/2829028591 shutdown_connections 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 -- 172.21.15.103:0/2829028591 wait complete. 2024-08-19T01:35:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 -- start start 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 Processor -- start 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6ef50 0x558de9d76c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d77150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558de9d7c3e0 con 0x558de9d79810 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558de9c8e970 con 0x558de9d6ef50 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558de9d7c560 con 0x558de9d6e630 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550ac91740 10 client.0 _ll_register_callbacks cb 0x558de99e27b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f5501d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d77150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f5501d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d77150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58396/0 (socket says 172.21.15.103:58396) 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f5501d9b640 1 -- 172.21.15.103:0/1052922486 learned_addr learned my addr 172.21.15.103:0/1052922486 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.520+0000 7f550259c640 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6ef50 0x558de9d76c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f5501d9b640 1 -- 172.21.15.103:0/1052922486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 msgr2=0x558de9d766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f5501d9b640 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f5501d9b640 1 -- 172.21.15.103:0/1052922486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6ef50 msgr2=0x558de9d76c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f5501d9b640 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6ef50 0x558de9d76c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f5501d9b640 1 -- 172.21.15.103:0/1052922486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de9a452b0 con 0x558de9d79810 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f5501d9b640 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d77150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009440 tx=0x7f54f4015eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53903192710da829 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f54e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f54e9ffb640 1 -- 172.21.15.103:0/1052922486 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f401b030 con 0x558de9d79810 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f54e9ffb640 1 -- 172.21.15.103:0/1052922486 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f54f400f040 con 0x558de9d79810 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f80316a0 con 0x558de9d79810 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f54e9ffb640 1 -- 172.21.15.103:0/1052922486 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54f4026020 con 0x558de9d79810 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f550ac91740 10 client.15156 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558de9d88020 con 0x558de9d79810 2024-08-19T01:35:11.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f54e9ffb640 1 -- 172.21.15.103:0/1052922486 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f80316a0 con 0x558de9d79810 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f54e9ffb640 1 -- 172.21.15.103:0/1052922486 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f54f40061b0 con 0x558de9d79810 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.521+0000 7f550ac91740 10 client.15156 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f550ac91740 10 client.15156 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f550ac91740 20 client.15156 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f550ac91740 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9d8a620 0x558de9d8ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f550ac91740 4 client.15156 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558de9ce0c80 con 0x558de9d8a620 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f550259c640 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9d8a620 0x558de9d8ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f54e9ffb640 10 client.15156 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.522+0000 7f550259c640 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9d8a620 0x558de9d8ba90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ce08d0e0118b862 server_cookie=370cd8a17c91fef3 in_seq=0 out_seq=0 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f54e9ffb640 1 -- 172.21.15.103:0/1052922486 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558de9ce0c80 con 0x558de9d8a620 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f54e9ffb640 10 client.15156 handle_command_reply: tid=0 2024-08-19T01:35:11.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 client.15156 shutdown 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 msgr2=0x558de9d77150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d77150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009440 tx=0x7f54f4015eb0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 shutdown_connections 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9d8a620 0x558de9d8ba90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d77150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6ef50 0x558de9d76c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 --2- 172.21.15.103:0/1052922486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6e630 0x558de9d766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f54e9ffb640 0 client.15156 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 >> 172.21.15.103:0/1052922486 conn(0x558de9d6abc0 msgr2=0x558de9d69d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 shutdown_connections 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 1 -- 172.21.15.103:0/1052922486 wait complete. 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.523+0000 7f550ac91740 20 client.15156 trim_cache size 0 max 16384 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 Processor -- start 2024-08-19T01:35:11.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 -- start start 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d75fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d79810 0x558de9d94160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558de9d89dd0 con 0x558de9d6ef50 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558de9d8a4a0 con 0x558de9d6e630 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550ac91740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558de9d8ab70 con 0x558de9d79810 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d75fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d75fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58410/0 (socket says 172.21.15.103:58410) 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 -- 172.21.15.103:0/4179745211 learned_addr learned my addr 172.21.15.103:0/4179745211 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 -- 172.21.15.103:0/4179745211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d79810 msgr2=0x558de9d94160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d79810 0x558de9d94160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f5501d9b640 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 -- 172.21.15.103:0/4179745211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 msgr2=0x558de9d71090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d71090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 -- 172.21.15.103:0/4179745211 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de9d6b770 con 0x558de9d6ef50 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f5501d9b640 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.541+0000 7f550259c640 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d75fb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f54f80057c0 tx=0x7f54f8049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed79b194eb1059cc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f54ebfff640 1 -- 172.21.15.103:0/4179745211 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8035040 con 0x558de9d6ef50 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f54ebfff640 1 -- 172.21.15.103:0/4179745211 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f80316a0 con 0x558de9d6ef50 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f54ebfff640 1 -- 172.21.15.103:0/4179745211 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80039e0 con 0x558de9d6ef50 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 -- 172.21.15.103:0/4179745211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 msgr2=0x558de9d75fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d75fb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f54f80057c0 tx=0x7f54f8049040 comp rx=0 tx=0).stop 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 -- 172.21.15.103:0/4179745211 shutdown_connections 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d79810 0x558de9d94160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d6ef50 0x558de9d75fb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 --2- 172.21.15.103:0/4179745211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 -- 172.21.15.103:0/4179745211 >> 172.21.15.103:0/4179745211 conn(0x558de9d6abc0 msgr2=0x558de9d84fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:11.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 -- 172.21.15.103:0/4179745211 shutdown_connections 2024-08-19T01:35:11.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.542+0000 7f550ac91740 1 -- 172.21.15.103:0/4179745211 wait complete. 2024-08-19T01:35:11.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 -- start start 2024-08-19T01:35:11.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 Processor -- start 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d74000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6ef50 0x558de9d72b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558de9d8b910 con 0x558de9d79810 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558de9d8b240 con 0x558de9d6e630 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558de9d8bfe0 con 0x558de9d6ef50 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550ac91740 10 client.0 _ll_register_callbacks cb 0x7f54f4009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58426/0 (socket says 172.21.15.103:58426) 2024-08-19T01:35:11.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 -- 172.21.15.103:0/3257136365 learned_addr learned my addr 172.21.15.103:0/3257136365 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f550259c640 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6ef50 0x558de9d72b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 -- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6ef50 msgr2=0x558de9d72b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6ef50 0x558de9d72b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 -- 172.21.15.103:0/3257136365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 msgr2=0x558de9d74000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.543+0000 7f5501d9b640 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d74000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f5501d9b640 1 -- 172.21.15.103:0/3257136365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de9d736f0 con 0x558de9d79810 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f5501d9b640 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d730b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f54f4017560 tx=0x7f54f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c390614069833596 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f54e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f54e9ffb640 1 -- 172.21.15.103:0/3257136365 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f401b040 con 0x558de9d79810 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f54e9ffb640 1 -- 172.21.15.103:0/3257136365 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f54f4014b20 con 0x558de9d79810 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f80316a0 con 0x558de9d79810 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f54e9ffb640 1 -- 172.21.15.103:0/3257136365 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54f4014d20 con 0x558de9d79810 2024-08-19T01:35:11.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f550ac91740 10 client.15162 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558de9df8b40 con 0x558de9d79810 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f54e9ffb640 1 -- 172.21.15.103:0/3257136365 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54f80316a0 con 0x558de9d79810 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f54e9ffb640 1 -- 172.21.15.103:0/3257136365 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f54f40176b0 con 0x558de9d79810 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f550ac91740 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f550ac91740 10 client.15162 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f550ac91740 20 client.15162 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.544+0000 7f550ac91740 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9dfd6e0 0x558de9dfc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 4 client.15162 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "status"}) -- 0x558de9cdb190 con 0x558de9dfd6e0 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550259c640 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9dfd6e0 0x558de9dfc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550259c640 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9dfd6e0 0x558de9dfc3f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abeea82353098250 server_cookie=132f0ecea02ecbe in_seq=0 out_seq=0 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f54e9ffb640 10 client.15162 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:35:11.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f54e9ffb640 1 -- 172.21.15.103:0/3257136365 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558de9cdb190 con 0x558de9dfd6e0 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f54e9ffb640 10 client.15162 handle_command_reply: tid=0 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 1 client.15162 shutdown 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 msgr2=0x558de9d730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d730b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f54f4017560 tx=0x7f54f40046c0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 shutdown_connections 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x558de9dfd6e0 0x558de9dfc3f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.545+0000 7f550ac91740 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558de9d79810 0x558de9d730b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f550ac91740 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558de9d6ef50 0x558de9d72b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f550ac91740 1 --2- 172.21.15.103:0/3257136365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558de9d6e630 0x558de9d74000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 >> 172.21.15.103:0/3257136365 conn(0x558de9d6abc0 msgr2=0x558de9d69520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f54e9ffb640 0 client.15162 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 shutdown_connections 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f550ac91740 1 -- 172.21.15.103:0/3257136365 wait complete. 2024-08-19T01:35:11.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f550ac91740 20 client.15162 trim_cache size 0 max 16384 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24478, 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 100.61748213200001, 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:35:11.562 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:35:11.563 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:35:11.563 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 129.98812430500001 2024-08-19T01:35:11.563 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:11.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f54ec0a2b80 msgr2=0x7f54ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f54ec0a2b80 0x7f54ec0a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0a34f0 tx=0x7f54fc0c5940 comp rx=0 tx=0).stop 2024-08-19T01:35:11.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 msgr2=0x7f550419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:11.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f550419cb00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00b970 tx=0x7f54fc00bd70 comp rx=0 tx=0).stop 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 shutdown_connections 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f54ec0a2b80 0x7f54ec0a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55041067b0 0x7f550419cb00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5504101150 0x7f5504198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 --2- 172.21.15.103:0/4067204346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55041007b0 0x7f5504198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 >> 172.21.15.103:0/4067204346 conn(0x7f55040fc4d0 msgr2=0x7f55040fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 shutdown_connections 2024-08-19T01:35:11.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:11.546+0000 7f5508893640 1 -- 172.21.15.103:0/4067204346 wait complete. 2024-08-19T01:35:12.166 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:11 smithi103 ceph-mon[18436]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:12.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:11 smithi146 ceph-mon[22098]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:12.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:11 smithi190 ceph-mon[22032]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:12.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-19T01:35:12.649 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:13.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 -- 172.21.15.103:0/3075303993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88105c70 msgr2=0x7f8d88106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3075303993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88105c70 0x7f8d88106100 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009ca0 tx=0x7f8d8002a940 comp rx=0 tx=0).stop 2024-08-19T01:35:13.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 -- 172.21.15.103:0/3075303993 shutdown_connections 2024-08-19T01:35:13.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3075303993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88105c70 0x7f8d88106100 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3075303993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d88101180 0x7f8d88105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3075303993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d88100860 0x7f8d88100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 -- 172.21.15.103:0/3075303993 >> 172.21.15.103:0/3075303993 conn(0x7f8d880fc4d0 msgr2=0x7f8d880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 -- 172.21.15.103:0/3075303993 shutdown_connections 2024-08-19T01:35:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.163+0000 7f8d9014c640 1 -- 172.21.15.103:0/3075303993 wait complete. 2024-08-19T01:35:13.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 Processor -- start 2024-08-19T01:35:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 -- start start 2024-08-19T01:35:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88100860 0x7f8d8819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d88101180 0x7f8d8819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d88105c70 0x7f8d881a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d881181a0 con 0x7f8d88100860 2024-08-19T01:35:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88118020 con 0x7f8d88101180 2024-08-19T01:35:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d9014c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8d88118320 con 0x7f8d88105c70 2024-08-19T01:35:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88100860 0x7f8d8819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88100860 0x7f8d8819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50118/0 (socket says 172.21.15.103:50118) 2024-08-19T01:35:13.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 -- 172.21.15.103:0/3078621033 learned_addr learned my addr 172.21.15.103:0/3078621033 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d88101180 0x7f8d8819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d88105c70 0x7f8d881a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 -- 172.21.15.103:0/3078621033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d88105c70 msgr2=0x7f8d881a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d88105c70 0x7f8d881a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 -- 172.21.15.103:0/3078621033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d88101180 msgr2=0x7f8d8819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d88101180 0x7f8d8819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8dee8640 1 -- 172.21.15.103:0/3078621033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d881a36d0 con 0x7f8d88100860 2024-08-19T01:35:13.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.164+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d88105c70 0x7f8d881a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:13.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.165+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88100860 0x7f8d8819e720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c890 tx=0x7f8d8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:13.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.165+0000 7f8d7effd640 1 -- 172.21.15.103:0/3078621033 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84004220 con 0x7f8d88100860 2024-08-19T01:35:13.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.165+0000 7f8d7effd640 1 -- 172.21.15.103:0/3078621033 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d840043c0 con 0x7f8d88100860 2024-08-19T01:35:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.165+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d881a39c0 con 0x7f8d88100860 2024-08-19T01:35:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.165+0000 7f8d7effd640 1 -- 172.21.15.103:0/3078621033 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84004a00 con 0x7f8d88100860 2024-08-19T01:35:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.165+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d881ab2a0 con 0x7f8d88100860 2024-08-19T01:35:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.168+0000 7f8d7effd640 1 -- 172.21.15.103:0/3078621033 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8d84004ba0 con 0x7f8d88100860 2024-08-19T01:35:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.168+0000 7f8d9254a740 1 Processor -- start 2024-08-19T01:35:13.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.168+0000 7f8d9254a740 1 -- start start 2024-08-19T01:35:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d7effd640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d740a2c00 0x7f8d740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b56fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b57062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5708c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d9254a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635b5660710 con 0x5635b56fbf50 2024-08-19T01:35:13.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d9254a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5635b55f8770 con 0x5635b5706810 2024-08-19T01:35:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d9254a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635b565c6d0 con 0x5635b56fb630 2024-08-19T01:35:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d740a2c00 0x7f8d740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b57062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d7effd640 1 -- 172.21.15.103:0/3078621033 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8d840cec50 con 0x7f8d88100860 2024-08-19T01:35:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b57062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50134/0 (socket says 172.21.15.103:50134) 2024-08-19T01:35:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/3700633110 learned_addr learned my addr 172.21.15.103:0/3700633110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b56fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5708c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/3700633110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 msgr2=0x5635b56fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b56fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/3700633110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 msgr2=0x5635b5708c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5708c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/3700633110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635b55ff3e0 con 0x5635b56fbf50 2024-08-19T01:35:13.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b57062d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009a20 tx=0x7f8d78009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5a9b17598f41a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:13.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d740a2c00 0x7f8d740a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005e30 tx=0x7f8d8002f470 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:13.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.169+0000 7f8d73fff640 1 -- 172.21.15.103:0/3700633110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78017070 con 0x5635b56fbf50 2024-08-19T01:35:13.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d73fff640 1 -- 172.21.15.103:0/3700633110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d78013470 con 0x5635b56fbf50 2024-08-19T01:35:13.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d73fff640 1 -- 172.21.15.103:0/3700633110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d780125d0 con 0x5635b56fbf50 2024-08-19T01:35:13.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 -- 172.21.15.103:0/3700633110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 msgr2=0x5635b57062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b57062d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009a20 tx=0x7f8d78009ee0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 -- 172.21.15.103:0/3700633110 shutdown_connections 2024-08-19T01:35:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5708c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b57062d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 --2- 172.21.15.103:0/3700633110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b56fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 -- 172.21.15.103:0/3700633110 >> 172.21.15.103:0/3700633110 conn(0x5635b56f7bc0 msgr2=0x5635b56f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 -- 172.21.15.103:0/3700633110 shutdown_connections 2024-08-19T01:35:13.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.170+0000 7f8d9254a740 1 -- 172.21.15.103:0/3700633110 wait complete. 2024-08-19T01:35:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 -- start start 2024-08-19T01:35:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 Processor -- start 2024-08-19T01:35:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b57036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b5703be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5704120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635b57093e0 con 0x5635b56fbf50 2024-08-19T01:35:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5635b561b970 con 0x5635b5706810 2024-08-19T01:35:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635b5709560 con 0x5635b56fb630 2024-08-19T01:35:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d9254a740 10 client.0 _ll_register_callbacks cb 0x5635b536f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d8d6e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b5703be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d8d6e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b5703be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50146/0 (socket says 172.21.15.103:50146) 2024-08-19T01:35:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.171+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/2352566825 learned_addr learned my addr 172.21.15.103:0/2352566825 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b57036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5704120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/2352566825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 msgr2=0x5635b57036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b57036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 msgr2=0x5635b5704120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5704120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/2352566825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635b53d22b0 con 0x5635b56fbf50 2024-08-19T01:35:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b57036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5704120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:13.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b5703be0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d78013e70 tx=0x7f8d78013bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd89345ebf793431 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:13.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:13.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d717fa640 1 -- 172.21.15.103:0/2352566825 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78017050 con 0x5635b56fbf50 2024-08-19T01:35:13.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d717fa640 1 -- 172.21.15.103:0/2352566825 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8d7801c450 con 0x5635b56fbf50 2024-08-19T01:35:13.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d78013470 con 0x5635b56fbf50 2024-08-19T01:35:13.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d717fa640 1 -- 172.21.15.103:0/2352566825 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d7801c650 con 0x5635b56fbf50 2024-08-19T01:35:13.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d9254a740 10 client.15174 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5635b5714f90 con 0x5635b56fbf50 2024-08-19T01:35:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.172+0000 7f8d717fa640 1 -- 172.21.15.103:0/2352566825 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d78013470 con 0x5635b56fbf50 2024-08-19T01:35:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d717fa640 1 -- 172.21.15.103:0/2352566825 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f8d78030870 con 0x5635b56fbf50 2024-08-19T01:35:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d9254a740 10 client.15174 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d9254a740 10 client.15174 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:35:13.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d9254a740 20 client.15174 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d9254a740 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5717260 0x5635b57186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d9254a740 4 client.15174 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635b566dc80 con 0x5635b5717260 2024-08-19T01:35:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.173+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5717260 0x5635b57186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.174+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5717260 0x5635b57186d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15d8c9043fba875 server_cookie=887cd700d61dbf30 in_seq=0 out_seq=0 2024-08-19T01:35:13.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.174+0000 7f8d717fa640 10 client.15174 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.174+0000 7f8d717fa640 1 -- 172.21.15.103:0/2352566825 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635b566dc80 con 0x5635b5717260 2024-08-19T01:35:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.174+0000 7f8d717fa640 10 client.15174 handle_command_reply: tid=0 2024-08-19T01:35:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.174+0000 7f8d9254a740 1 client.15174 shutdown 2024-08-19T01:35:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 msgr2=0x5635b5703be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b5703be0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d78013e70 tx=0x7f8d78013bf0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 shutdown_connections 2024-08-19T01:35:13.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5717260 0x5635b57186d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b5704120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fbf50 0x5635b5703be0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 --2- 172.21.15.103:0/2352566825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fb630 0x5635b57036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d717fa640 0 client.15174 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:13.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 >> 172.21.15.103:0/2352566825 conn(0x5635b56f7bc0 msgr2=0x5635b56f6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:13.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 shutdown_connections 2024-08-19T01:35:13.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 1 -- 172.21.15.103:0/2352566825 wait complete. 2024-08-19T01:35:13.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.175+0000 7f8d9254a740 20 client.15174 trim_cache size 0 max 16384 2024-08-19T01:35:13.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 Processor -- start 2024-08-19T01:35:13.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 -- start start 2024-08-19T01:35:13.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b56fb630 0x5635b5716b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b56fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b5706810 0x5635b5703f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635b5716a10 con 0x5635b5706810 2024-08-19T01:35:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5635b57170e0 con 0x5635b56fb630 2024-08-19T01:35:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d9254a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635b57177b0 con 0x5635b56fbf50 2024-08-19T01:35:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b5706810 0x5635b5703f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b5706810 0x5635b5703f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50148/0 (socket says 172.21.15.103:50148) 2024-08-19T01:35:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 -- 172.21.15.103:0/549410969 learned_addr learned my addr 172.21.15.103:0/549410969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:13.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b56fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b56fb630 0x5635b5716b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 -- 172.21.15.103:0/549410969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 msgr2=0x5635b56fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b56fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 -- 172.21.15.103:0/549410969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b56fb630 msgr2=0x5635b5716b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b56fb630 0x5635b5716b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.193+0000 7f8d8e6e9640 1 -- 172.21.15.103:0/549410969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635b56f8d70 con 0x5635b5706810 2024-08-19T01:35:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b5706810 0x5635b5703f70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d8002f630 tx=0x7f8d80005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa56cfa1295a2374 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d737fe640 1 -- 172.21.15.103:0/549410969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8004c070 con 0x5635b5706810 2024-08-19T01:35:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d737fe640 1 -- 172.21.15.103:0/549410969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d8002f660 con 0x5635b5706810 2024-08-19T01:35:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d737fe640 1 -- 172.21.15.103:0/549410969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80034050 con 0x5635b5706810 2024-08-19T01:35:13.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 -- 172.21.15.103:0/549410969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b5706810 msgr2=0x5635b5703f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b5706810 0x5635b5703f70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d8002f630 tx=0x7f8d80005d80 comp rx=0 tx=0).stop 2024-08-19T01:35:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 -- 172.21.15.103:0/549410969 shutdown_connections 2024-08-19T01:35:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b5706810 0x5635b5703f70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b56fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 --2- 172.21.15.103:0/549410969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b56fb630 0x5635b5716b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 -- 172.21.15.103:0/549410969 >> 172.21.15.103:0/549410969 conn(0x5635b56f7bc0 msgr2=0x5635b5702790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 -- 172.21.15.103:0/549410969 shutdown_connections 2024-08-19T01:35:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.194+0000 7f8d9254a740 1 -- 172.21.15.103:0/549410969 wait complete. 2024-08-19T01:35:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 -- start start 2024-08-19T01:35:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 Processor -- start 2024-08-19T01:35:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fb630 0x5635b5720f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b571a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b571a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635b5718550 con 0x5635b56fb630 2024-08-19T01:35:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5635b5717e80 con 0x5635b5706810 2024-08-19T01:35:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635b5718c20 con 0x5635b56fbf50 2024-08-19T01:35:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d9254a740 10 client.0 _ll_register_callbacks cb 0x7f8d78009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d8dee8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fb630 0x5635b5720f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d8dee8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fb630 0x5635b5720f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50160/0 (socket says 172.21.15.103:50160) 2024-08-19T01:35:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d8dee8640 1 -- 172.21.15.103:0/1061365367 learned_addr learned my addr 172.21.15.103:0/1061365367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:13.232 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:13.232 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "e721016a-5dc9-11ef-bccf-c7b262605968", 2024-08-19T01:35:13.232 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-08-19T01:35:13.232 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24485, 2024-08-19T01:35:13.232 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-08-19T01:35:13.232 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-08-19T01:35:13.233 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-08-19T01:35:13.233 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 109.024629295, 2024-08-19T01:35:13.233 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-08-19T01:35:13.233 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 86, 2024-08-19T01:35:13.233 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-08-19T01:35:13.234 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 121.862898234 2024-08-19T01:35:13.234 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b571a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.195+0000 7f8d8e6e9640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b571a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 msgr2=0x5635b571a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b571a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/1061365367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fb630 msgr2=0x5635b5720f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fb630 0x5635b5720f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d8d6e7640 1 -- 172.21.15.103:0/1061365367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635b571aea0 con 0x5635b56fbf50 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fb630 0x5635b5720f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d8d6e7640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b571a270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d78030e80 tx=0x7f8d7801ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f0259f70aee984 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d717fa640 1 -- 172.21.15.103:0/1061365367 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78037070 con 0x5635b56fbf50 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d717fa640 1 -- 172.21.15.103:0/1061365367 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8d78006b60 con 0x5635b56fbf50 2024-08-19T01:35:13.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8002f660 con 0x5635b56fbf50 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d717fa640 1 -- 172.21.15.103:0/1061365367 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d78017040 con 0x5635b56fbf50 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d9254a740 10 client.24820 fetch_fsmap learned FSMap version 26 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.196+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5635b5781970 con 0x5635b56fbf50 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d717fa640 1 -- 172.21.15.103:0/1061365367 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d8002f660 con 0x5635b56fbf50 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d717fa640 1 -- 172.21.15.103:0/1061365367 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f8d7803a020 con 0x5635b56fbf50 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d9254a740 10 client.24820 fetch_fsmap finished waiting for FSMap version 26 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d9254a740 10 client.24820 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-19T01:35:13.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d9254a740 20 client.24820 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d9254a740 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5786370 0x5635b5785080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d9254a740 4 client.24820 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-19T01:35:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "status"}) -- 0x5635b5668190 con 0x5635b5786370 2024-08-19T01:35:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.197+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5786370 0x5635b5785080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d717fa640 10 client.24820 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d8dee8640 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5786370 0x5635b5785080 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef9e2da93557bfc server_cookie=6d466e6f20e0716b in_seq=0 out_seq=0 2024-08-19T01:35:13.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d717fa640 1 -- 172.21.15.103:0/1061365367 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5635b5668190 con 0x5635b5786370 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d717fa640 10 client.24820 handle_command_reply: tid=0 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 client.24820 shutdown 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 msgr2=0x5635b571a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b571a270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8d78030e80 tx=0x7f8d7801ca20 comp rx=0 tx=0).stop 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 shutdown_connections 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635b5786370 0x5635b5785080 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635b5706810 0x5635b571a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635b56fbf50 0x5635b571a270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 --2- 172.21.15.103:0/1061365367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635b56fb630 0x5635b5720f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 >> 172.21.15.103:0/1061365367 conn(0x5635b56f7bc0 msgr2=0x5635b57117b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d717fa640 0 client.24820 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 shutdown_connections 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 1 -- 172.21.15.103:0/1061365367 wait complete. 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.198+0000 7f8d9254a740 20 client.24820 trim_cache size 0 max 16384 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d740a2c00 msgr2=0x7f8d740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d740a2c00 0x7f8d740a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005e30 tx=0x7f8d8002f470 comp rx=0 tx=0).stop 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88100860 msgr2=0x7f8d8819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88100860 0x7f8d8819e720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c890 tx=0x7f8d8400cd50 comp rx=0 tx=0).stop 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 shutdown_connections 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d740a2c00 0x7f8d740a50c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d88105c70 0x7f8d881a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d88101180 0x7f8d8819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 --2- 172.21.15.103:0/3078621033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d88100860 0x7f8d8819e720 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 >> 172.21.15.103:0/3078621033 conn(0x7f8d880fc4d0 msgr2=0x7f8d880fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 shutdown_connections 2024-08-19T01:35:13.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:13.199+0000 7f8d9014c640 1 -- 172.21.15.103:0/3078621033 wait complete. 2024-08-19T01:35:13.948 INFO:teuthology.run_tasks:Running task sequential... 2024-08-19T01:35:13.958 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-19T01:35:13.958 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-19T01:35:13.958 INFO:teuthology.task.print:Enabling mgr modules 2024-08-19T01:35:13.958 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-08-19T01:35:14.011 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-08-19T01:35:14.011 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-08-19T01:35:14.011 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-08-19T01:35:14.011 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-19T01:35:14.011 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:14.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:14.012 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-08-19T01:35:14.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:14.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-08-19T01:35:14.097 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2024-08-19T01:35:14.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:35:14.099 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-08-19T01:35:14.099 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-08-19T01:35:14.099 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-19T01:35:14.162 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:13 smithi103 ceph-mon[18436]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-08-19T01:35:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:13 smithi146 ceph-mon[22098]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-08-19T01:35:14.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:13 smithi190 ceph-mon[22032]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-08-19T01:35:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 -- 172.21.15.103:0/3393314169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c81078b0 msgr2=0x7fe7c8107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 --2- 172.21.15.103:0/3393314169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c81078b0 0x7fe7c8107c90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009930 tx=0x7fe7bc02a420 comp rx=0 tx=0).stop 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 -- 172.21.15.103:0/3393314169 shutdown_connections 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 --2- 172.21.15.103:0/3393314169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c8102250 0x7fe7c810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 --2- 172.21.15.103:0/3393314169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c81018b0 0x7fe7c8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 --2- 172.21.15.103:0/3393314169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c81078b0 0x7fe7c8107c90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 -- 172.21.15.103:0/3393314169 >> 172.21.15.103:0/3393314169 conn(0x7fe7c80fd5f0 msgr2=0x7fe7c80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 -- 172.21.15.103:0/3393314169 shutdown_connections 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.876+0000 7fe7ceace640 1 -- 172.21.15.103:0/3393314169 wait complete. 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 Processor -- start 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 -- start start 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c81018b0 0x7fe7c8074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c8102250 0x7fe7c8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c81078b0 0x7fe7c8074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8111b80 con 0x7fe7c8102250 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8111a00 con 0x7fe7c81018b0 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7ceace640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8111d00 con 0x7fe7c81078b0 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c8102250 0x7fe7c8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c8102250 0x7fe7c8074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50182/0 (socket says 172.21.15.103:50182) 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7c7fff640 1 -- 172.21.15.103:0/1475276560 learned_addr learned my addr 172.21.15.103:0/1475276560 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7cc86a640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c81018b0 0x7fe7c8074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.877+0000 7fe7cd06b640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c81078b0 0x7fe7c8074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c7fff640 1 -- 172.21.15.103:0/1475276560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c81078b0 msgr2=0x7fe7c8074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c7fff640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c81078b0 0x7fe7c8074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c7fff640 1 -- 172.21.15.103:0/1475276560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c81018b0 msgr2=0x7fe7c8074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c7fff640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c81018b0 0x7fe7c8074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c7fff640 1 -- 172.21.15.103:0/1475276560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c81a6620 con 0x7fe7c8102250 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7cc86a640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c81018b0 0x7fe7c8074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c7fff640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c8102250 0x7fe7c8074760 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400d950 tx=0x7fe7b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c5ffb640 1 -- 172.21.15.103:0/1475276560 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4017030 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c5ffb640 1 -- 172.21.15.103:0/1475276560 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7b4004020 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7c5ffb640 1 -- 172.21.15.103:0/1475276560 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40047a0 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c81a6910 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.878+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a6ea0 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.879+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c8101d10 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.880+0000 7fe7c5ffb640 1 -- 172.21.15.103:0/1475276560 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe7b40041d0 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.881+0000 7fe7c5ffb640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7b80a2c00 0x7fe7b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.881+0000 7fe7cc86a640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7b80a2c00 0x7fe7b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.881+0000 7fe7c5ffb640 1 -- 172.21.15.103:0/1475276560 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe7b40cea60 con 0x7fe7c8102250 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.881+0000 7fe7cc86a640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7b80a2c00 0x7fe7b80a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc000c00 tx=0x7fe7bc009610 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:14.885+0000 7fe7c5ffb640 1 -- 172.21.15.103:0/1475276560 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe7b409b9f0 con 0x7fe7c8102250 2024-08-19T01:35:15.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.089+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe7c8109e20 con 0x7fe7c8102250 2024-08-19T01:35:15.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.090+0000 7fe7c5ffb640 1 -- 172.21.15.103:0/1475276560 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fe7b409b2f0 con 0x7fe7c8102250 2024-08-19T01:35:15.091 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7b80a2c00 msgr2=0x7fe7b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7b80a2c00 0x7fe7b80a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc000c00 tx=0x7fe7bc009610 comp rx=0 tx=0).stop 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c8102250 msgr2=0x7fe7c8074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c8102250 0x7fe7c8074760 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400d950 tx=0x7fe7b400de10 comp rx=0 tx=0).stop 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 shutdown_connections 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe7b80a2c00 0x7fe7b80a50c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7c81078b0 0x7fe7c8074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7c8102250 0x7fe7c8074760 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 --2- 172.21.15.103:0/1475276560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7c81018b0 0x7fe7c8074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:15.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 >> 172.21.15.103:0/1475276560 conn(0x7fe7c80fd5f0 msgr2=0x7fe7c80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:15.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 shutdown_connections 2024-08-19T01:35:15.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:15.092+0000 7fe7ceace640 1 -- 172.21.15.103:0/1475276560 wait complete. 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-19T01:35:15.909 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-19T01:35:15.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:15.910 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-08-19T01:35:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42343sec preferred_lft 42343sec 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-08-19T01:35:15.968 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-08-19T01:35:15.968 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-19T01:35:15.969 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:35:15.969 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-08-19T01:35:15.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-08-19T01:35:15.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-08-19T01:35:15.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-08-19T01:35:15.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-19T01:35:15.969 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-08-19T01:35:16.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:35:16.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:15 smithi103 ceph-mon[18436]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-19T01:35:16.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:15 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1475276560' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-19T01:35:16.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:35:16.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:16.164 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-19T01:35:16.188 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-08-19T01:35:16.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:16.190 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-08-19T01:35:16.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:15 smithi146 ceph-mon[22098]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-19T01:35:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:15 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1475276560' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-19T01:35:16.266 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-08-19T01:35:16.266 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-19T01:35:16.266 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-19T01:35:16.266 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-19T01:35:16.266 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-19T01:35:16.267 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:35:16.267 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-08-19T01:35:16.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-19T01:35:16.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-19T01:35:16.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-19T01:35:16.267 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-08-19T01:35:16.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:35:16.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:15 smithi190 ceph-mon[22032]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-08-19T01:35:16.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:15 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1475276560' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-19T01:35:16.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:35:16.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:16.549 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-08-19T01:35:16.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:16.565 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-08-19T01:35:16.620 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:35:16.620 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-08-19T01:35:16.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:35:16.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-19T01:35:16.621 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-08-19T01:35:16.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:35:16.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:35:16.739 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-19T01:35:16.739 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:35:16.739 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-08-19T01:35:16.739 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-19T01:35:16.739 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-19T01:35:16.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-19T01:35:16.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-19T01:35:16.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-19T01:35:16.740 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-08-19T01:35:16.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:35:16.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:35:16.871 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-19T01:35:16.871 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-08-19T01:35:16.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-08-19T01:35:16.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-08-19T01:35:16.871 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-08-19T01:35:16.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-19T01:35:16.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:35:16.949 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-08-19T01:35:16.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-19T01:35:16.950 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:16.964 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-19T01:35:16.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-19T01:35:16.965 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:17.020 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-08-19T01:35:17.085 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-08-19T01:35:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-08-19T01:35:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-08-19T01:35:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-08-19T01:35:17.143 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysusers.service 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/8576e00d22ee3ebaed8472d5992ce8820a2fef41ef5d5ac1714ff3ceab16aaed/merged 2024-08-19T01:35:17.144 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/c441f5b5ae816e8b92b9467c3e1ef9f01e5a9833461385b456dcb5b8aa566b0a/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/0a23bc77b3b1eb9d0f6bdc6389a02931b2de50ebf38e9b317c4d7b7fa8d8f314/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/ac0383363d2e07cb6fab3fcb232be11a2c6bd6cbe7dc838b704bccc1389ea55b/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/ee3525aafb39724f07aab342cc7f52c83d8b93057549931680b448622bb25901/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/87285caa854028ca56604bdcca4da12d811e1066675e65ab1eb0814c7b54a1a0/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/73c4940e2a79d267ff38df1c60299cbe9f97e5658152bd155435ef3958815684/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/48b73665daf0834f8a7007b8cc5ada956aa5ebf628d09f6ba12a9c8b9e643284/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/38d73ba9e707bebdbd346e94fb09eef50333790452da9084ae4ca739a18408f9/merged 2024-08-19T01:35:17.145 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/3aaa7b7216fde50627f7b2bf666be95f710e94105aec55e3be7762efad5ae84c/merged 2024-08-19T01:35:17.146 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-08-19T01:35:17.146 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:35:17.146 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:35:17.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:17.146 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-08-19T01:35:17.198 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-19T01:35:17.198 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/a1d53bc3-7e67-4c1d-889f-ad0027be0cfc --client_fs=cephfs) 2024-08-19T01:35:17.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-08-19T01:35:17.265 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-08-19T01:35:17.322 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysusers.service 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-19T01:35:17.323 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/8576e00d22ee3ebaed8472d5992ce8820a2fef41ef5d5ac1714ff3ceab16aaed/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/c441f5b5ae816e8b92b9467c3e1ef9f01e5a9833461385b456dcb5b8aa566b0a/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/0a23bc77b3b1eb9d0f6bdc6389a02931b2de50ebf38e9b317c4d7b7fa8d8f314/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/ac0383363d2e07cb6fab3fcb232be11a2c6bd6cbe7dc838b704bccc1389ea55b/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/ee3525aafb39724f07aab342cc7f52c83d8b93057549931680b448622bb25901/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/87285caa854028ca56604bdcca4da12d811e1066675e65ab1eb0814c7b54a1a0/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/73c4940e2a79d267ff38df1c60299cbe9f97e5658152bd155435ef3958815684/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/48b73665daf0834f8a7007b8cc5ada956aa5ebf628d09f6ba12a9c8b9e643284/merged 2024-08-19T01:35:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/38d73ba9e707bebdbd346e94fb09eef50333790452da9084ae4ca739a18408f9/merged 2024-08-19T01:35:17.325 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/3aaa7b7216fde50627f7b2bf666be95f710e94105aec55e3be7762efad5ae84c/merged 2024-08-19T01:35:17.325 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-08-19T01:35:17.325 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:35:17.325 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:35:17.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:17.325 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-08-19T01:35:17.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:2024-08-19T01:35:17.350+0000 7f0b3ee7a480 -1 init, newargv = 0x559148735920 newargc=15 2024-08-19T01:35:17.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:ceph-fuse[69453]: starting ceph client 2024-08-19T01:35:17.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:ceph-fuse[69453]: starting fuse 2024-08-19T01:35:18.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:17 smithi146 ceph-mon[22098]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-19T01:35:18.235 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:17 smithi103 ceph-mon[18436]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-19T01:35:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:17 smithi190 ceph-mon[22032]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-19T01:35:18.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-08-19T01:35:18.495 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-08-19T01:35:18.553 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-08-19T01:35:18.553 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-08-19T01:35:18.554 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysusers.service 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-08-19T01:35:18.555 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/8576e00d22ee3ebaed8472d5992ce8820a2fef41ef5d5ac1714ff3ceab16aaed/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/c441f5b5ae816e8b92b9467c3e1ef9f01e5a9833461385b456dcb5b8aa566b0a/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/0a23bc77b3b1eb9d0f6bdc6389a02931b2de50ebf38e9b317c4d7b7fa8d8f314/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/ac0383363d2e07cb6fab3fcb232be11a2c6bd6cbe7dc838b704bccc1389ea55b/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/ee3525aafb39724f07aab342cc7f52c83d8b93057549931680b448622bb25901/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/87285caa854028ca56604bdcca4da12d811e1066675e65ab1eb0814c7b54a1a0/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/73c4940e2a79d267ff38df1c60299cbe9f97e5658152bd155435ef3958815684/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/48b73665daf0834f8a7007b8cc5ada956aa5ebf628d09f6ba12a9c8b9e643284/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/38d73ba9e707bebdbd346e94fb09eef50333790452da9084ae4ca739a18408f9/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/3aaa7b7216fde50627f7b2bf666be95f710e94105aec55e3be7762efad5ae84c/merged 2024-08-19T01:35:18.556 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-08-19T01:35:18.557 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:35:18.557 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:35:18.557 INFO:teuthology.orchestra.run.smithi103.stdout:/home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:18.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:18.557 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-08-19T01:35:18.610 INFO:teuthology.orchestra.run.smithi103.stdout:121 2024-08-19T01:35:18.610 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo stdin-killer -- python3 -c ' 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> import glob 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> import re 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> import os 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> import subprocess 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> def _find_admin_socket(client_name): 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> files = glob.glob(asok_path) 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> 2024-08-19T01:35:18.611 DEBUG:teuthology.orchestra.run.smithi103:> # Given a non-glob path, it better be there 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> if "*" not in asok_path: 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> assert(len(files) == 1) 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> return files[0] 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> for f in files: 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> contents = proc_f.read() 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> if mountpoint in contents: 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> return f 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> 2024-08-19T01:35:18.612 DEBUG:teuthology.orchestra.run.smithi103:> print(_find_admin_socket("client.0")) 2024-08-19T01:35:18.613 DEBUG:teuthology.orchestra.run.smithi103:> ' 2024-08-19T01:35:18.731 INFO:teuthology.orchestra.run.smithi103.stdout:/var/run/ceph/ceph-client.0.69453.asok 2024-08-19T01:35:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-19T01:35:18.742 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69453.asok 2024-08-19T01:35:18.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:18.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69453.asok status 2024-08-19T01:35:18.876 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:18.876 INFO:teuthology.orchestra.run.smithi103.stdout: "metadata": { 2024-08-19T01:35:18.876 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_sha1": "de09dd018de4baddfa6ca4d43bc8fb784cf75833", 2024-08-19T01:35:18.876 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "entity_id": "0", 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": "smithi103", 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "pid": "69453", 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "root": "/volumes/qa/sv_0/a1d53bc3-7e67-4c1d-889f-ad0027be0cfc" 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_count": 0, 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_pinned_count": 0, 2024-08-19T01:35:18.877 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 15192, 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inst": { 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": { 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client", 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "num": 15192 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 524605277 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-08-19T01:35:18.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 524605277 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "inst_str": "client.15192 192.168.0.1:0/524605277", 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "addr_str": "192.168.0.1:0/524605277", 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "inode_count": 5, 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_epoch": 26, 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch": 86, 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch_barrier": 0, 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "blocklisted": false, 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs" 2024-08-19T01:35:18.879 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:18.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:18.884 DEBUG:teuthology.orchestra.run.smithi103:> stat --file-system '--printf=%T 2024-08-19T01:35:18.885 DEBUG:teuthology.orchestra.run.smithi103:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:18.900 INFO:teuthology.orchestra.run.smithi103.stdout:fuseblk 2024-08-19T01:35:18.900 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:18.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:35:18.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:18.969 INFO:teuthology.run_tasks:Running task exec... 2024-08-19T01:35:18.982 INFO:teuthology.task.exec:Executing custom commands... 2024-08-19T01:35:18.983 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:35:18.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-08-19T01:35:19.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 -- 172.21.15.103:0/67426566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4100790 msgr2=0x7f86d4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:19.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 --2- 172.21.15.103:0/67426566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4100790 0x7f86d4100b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009900 tx=0x7f86cc02a410 comp rx=0 tx=0).stop 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 -- 172.21.15.103:0/67426566 shutdown_connections 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 --2- 172.21.15.103:0/67426566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d4102340 0x7f86d4108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 --2- 172.21.15.103:0/67426566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86d4101980 0x7f86d4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 --2- 172.21.15.103:0/67426566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4100790 0x7f86d4100b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 -- 172.21.15.103:0/67426566 >> 172.21.15.103:0/67426566 conn(0x7f86d40fbef0 msgr2=0x7f86d40fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 -- 172.21.15.103:0/67426566 shutdown_connections 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.108+0000 7f86db3d8640 1 -- 172.21.15.103:0/67426566 wait complete. 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 Processor -- start 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 -- start start 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86d4100790 0x7f86d4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4101980 0x7f86d4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d4102340 0x7f86d419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410d470 con 0x7f86d4101980 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410d2f0 con 0x7f86d4100790 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86db3d8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86d410d5f0 con 0x7f86d4102340 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86d8973640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4101980 0x7f86d4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86d8973640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4101980 0x7f86d4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50208/0 (socket says 172.21.15.103:50208) 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86d8973640 1 -- 172.21.15.103:0/1274107946 learned_addr learned my addr 172.21.15.103:0/1274107946 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86d9174640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86d4100790 0x7f86d4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.109+0000 7f86d9975640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d4102340 0x7f86d419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d8973640 1 -- 172.21.15.103:0/1274107946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d4102340 msgr2=0x7f86d419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d8973640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d4102340 0x7f86d419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d8973640 1 -- 172.21.15.103:0/1274107946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86d4100790 msgr2=0x7f86d4197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d8973640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86d4100790 0x7f86d4197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d8973640 1 -- 172.21.15.103:0/1274107946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d419f680 con 0x7f86d4101980 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d8973640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4101980 0x7f86d4197fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cce0 tx=0x7f86c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:19.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d27fc640 1 -- 172.21.15.103:0/1274107946 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8004b20 con 0x7f86d4101980 2024-08-19T01:35:19.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d27fc640 1 -- 172.21.15.103:0/1274107946 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86c8004cc0 con 0x7f86d4101980 2024-08-19T01:35:19.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d419f970 con 0x7f86d4101980 2024-08-19T01:35:19.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86d27fc640 1 -- 172.21.15.103:0/1274107946 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800f6b0 con 0x7f86d4101980 2024-08-19T01:35:19.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.110+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d419ff10 con 0x7f86d4101980 2024-08-19T01:35:19.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.111+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d4107ad0 con 0x7f86d4101980 2024-08-19T01:35:19.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.112+0000 7f86d27fc640 1 -- 172.21.15.103:0/1274107946 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f86c800f850 con 0x7f86d4101980 2024-08-19T01:35:19.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.113+0000 7f86d27fc640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86bc0a2cb0 0x7f86bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:19.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.113+0000 7f86d9174640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86bc0a2cb0 0x7f86bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:19.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.113+0000 7f86d27fc640 1 -- 172.21.15.103:0/1274107946 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f86c80cef70 con 0x7f86d4101980 2024-08-19T01:35:19.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.113+0000 7f86d9174640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86bc0a2cb0 0x7f86bc0a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005ab0 tx=0x7f86cc005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:19.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.117+0000 7f86d27fc640 1 -- 172.21.15.103:0/1274107946 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f86c8014030 con 0x7f86d4101980 2024-08-19T01:35:19.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:18 smithi103 ceph-mon[18436]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:35:19.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.321+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f86d402d7f0 con 0x7f86d4101980 2024-08-19T01:35:19.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:18 smithi190 ceph-mon[22032]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:35:19.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:18 smithi146 ceph-mon[22098]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-19T01:35:19.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.967+0000 7f86d27fc640 1 -- 172.21.15.103:0/1274107946 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f86c809b950 con 0x7f86d4101980 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86bc0a2cb0 msgr2=0x7f86bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86bc0a2cb0 0x7f86bc0a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005ab0 tx=0x7f86cc005a20 comp rx=0 tx=0).stop 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4101980 msgr2=0x7f86d4197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4101980 0x7f86d4197fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cce0 tx=0x7f86c800a620 comp rx=0 tx=0).stop 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 shutdown_connections 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86bc0a2cb0 0x7f86bc0a5170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86d4102340 0x7f86d419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86d4101980 0x7f86d4197fa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 --2- 172.21.15.103:0/1274107946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f86d4100790 0x7f86d4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 >> 172.21.15.103:0/1274107946 conn(0x7f86d40fbef0 msgr2=0x7f86d4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 shutdown_connections 2024-08-19T01:35:19.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:19.970+0000 7f86db3d8640 1 -- 172.21.15.103:0/1274107946 wait complete. 2024-08-19T01:35:19.983 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-19T01:35:19.994 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-19T01:35:20.056 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T01:35:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 -- 172.21.15.103:0/1497884185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4102310 msgr2=0x7fc4e4108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/1497884185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4102310 0x7fc4e4108ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009ca0 tx=0x7fc4dc02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 -- 172.21.15.103:0/1497884185 shutdown_connections 2024-08-19T01:35:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/1497884185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4102310 0x7fc4e4108ba0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/1497884185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4e4101950 0x7fc4e4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/1497884185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e4100760 0x7fc4e4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 -- 172.21.15.103:0/1497884185 >> 172.21.15.103:0/1497884185 conn(0x7fc4e40fbec0 msgr2=0x7fc4e40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 -- 172.21.15.103:0/1497884185 shutdown_connections 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.158+0000 7fc4ea43b640 1 -- 172.21.15.103:0/1497884185 wait complete. 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 Processor -- start 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 -- start start 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4100760 0x7fc4e4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e4101950 0x7fc4e4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4e4102310 0x7fc4e419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e410c270 con 0x7fc4e4100760 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e410c0f0 con 0x7fc4e4101950 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4ea43b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e410c3f0 con 0x7fc4e4102310 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4100760 0x7fc4e4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4100760 0x7fc4e4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50218/0 (socket says 172.21.15.103:50218) 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 -- 172.21.15.103:0/4045193318 learned_addr learned my addr 172.21.15.103:0/4045193318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e37fe640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e4101950 0x7fc4e4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e89d8640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4e4102310 0x7fc4e419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 -- 172.21.15.103:0/4045193318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4e4102310 msgr2=0x7fc4e419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4e4102310 0x7fc4e419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 -- 172.21.15.103:0/4045193318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e4101950 msgr2=0x7fc4e4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e4101950 0x7fc4e4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e3fff640 1 -- 172.21.15.103:0/4045193318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e419f6d0 con 0x7fc4e4100760 2024-08-19T01:35:20.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e89d8640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4e4102310 0x7fc4e419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.159+0000 7fc4e37fe640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e4101950 0x7fc4e4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.160+0000 7fc4e3fff640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4100760 0x7fc4e4197ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c970 tx=0x7fc4d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.160+0000 7fc4e17fa640 1 -- 172.21.15.103:0/4045193318 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004220 con 0x7fc4e4100760 2024-08-19T01:35:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.160+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e419f9c0 con 0x7fc4e4100760 2024-08-19T01:35:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.160+0000 7fc4e17fa640 1 -- 172.21.15.103:0/4045193318 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d40043c0 con 0x7fc4e4100760 2024-08-19T01:35:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.160+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e419ff60 con 0x7fc4e4100760 2024-08-19T01:35:20.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.160+0000 7fc4e17fa640 1 -- 172.21.15.103:0/4045193318 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d401d060 con 0x7fc4e4100760 2024-08-19T01:35:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.161+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e4100b60 con 0x7fc4e4100760 2024-08-19T01:35:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.162+0000 7fc4e17fa640 1 -- 172.21.15.103:0/4045193318 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc4d40079b0 con 0x7fc4e4100760 2024-08-19T01:35:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.163+0000 7fc4e17fa640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc4cc0a2b90 0x7fc4cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.163+0000 7fc4e37fe640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc4cc0a2b90 0x7fc4cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.163+0000 7fc4e17fa640 1 -- 172.21.15.103:0/4045193318 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc4d40cf1c0 con 0x7fc4e4100760 2024-08-19T01:35:20.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.163+0000 7fc4e37fe640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc4cc0a2b90 0x7fc4cc0a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4198fc0 tx=0x7fc4d800b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:20.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.167+0000 7fc4e17fa640 1 -- 172.21.15.103:0/4045193318 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc4d4014030 con 0x7fc4e4100760 2024-08-19T01:35:20.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:19 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1274107946' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-08-19T01:35:20.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.335+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4e41a09e0 con 0x7fc4e4100760 2024-08-19T01:35:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.337+0000 7fc4e17fa640 1 -- 172.21.15.103:0/4045193318 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22704 (secure 0 0 0) 0x7fc4d409c150 con 0x7fc4e4100760 2024-08-19T01:35:20.338 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:20.338 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:34:01.874407+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc4cc0a2b90 msgr2=0x7fc4cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc4cc0a2b90 0x7fc4cc0a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4198fc0 tx=0x7fc4d800b5f0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4100760 msgr2=0x7fc4e4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4100760 0x7fc4e4197ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c970 tx=0x7fc4d400ce30 comp rx=0 tx=0).stop 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 shutdown_connections 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc4cc0a2b90 0x7fc4cc0a5050 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4e4102310 0x7fc4e419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4e4101950 0x7fc4e4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 --2- 172.21.15.103:0/4045193318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4e4100760 0x7fc4e4197ab0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 >> 172.21.15.103:0/4045193318 conn(0x7fc4e40fbec0 msgr2=0x7fc4e406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 shutdown_connections 2024-08-19T01:35:20.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.339+0000 7fc4ea43b640 1 -- 172.21.15.103:0/4045193318 wait complete. 2024-08-19T01:35:20.351 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-19T01:29:05.010792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-19T01:33:19.641931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-19T01:33:22.107240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-19T01:35:20.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-19T01:35:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:19 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1274107946' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-08-19T01:35:20.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:19 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1274107946' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-08-19T01:35:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 -- 172.21.15.103:0/152435594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb801022e0 msgr2=0x7feb80108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 --2- 172.21.15.103:0/152435594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb801022e0 0x7feb80108b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feb70009ca0 tx=0x7feb7002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 -- 172.21.15.103:0/152435594 shutdown_connections 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 --2- 172.21.15.103:0/152435594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb801022e0 0x7feb80108b70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 --2- 172.21.15.103:0/152435594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb80101920 0x7feb80101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 --2- 172.21.15.103:0/152435594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb80100730 0x7feb80100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 -- 172.21.15.103:0/152435594 >> 172.21.15.103:0/152435594 conn(0x7feb800fbe70 msgr2=0x7feb800fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 -- 172.21.15.103:0/152435594 shutdown_connections 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.454+0000 7feb85425640 1 -- 172.21.15.103:0/152435594 wait complete. 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 Processor -- start 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 -- start start 2024-08-19T01:35:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb80100730 0x7feb80197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101920 0x7feb80197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb801022e0 0x7feb8019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010d3d0 con 0x7feb80101920 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010d250 con 0x7feb80100730 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb85425640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feb8010d550 con 0x7feb801022e0 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb7e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101920 0x7feb80197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb7e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101920 0x7feb80197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50228/0 (socket says 172.21.15.103:50228) 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.455+0000 7feb7e7fc640 1 -- 172.21.15.103:0/2830197430 learned_addr learned my addr 172.21.15.103:0/2830197430 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7effd640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb80100730 0x7feb80197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7f7fe640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb801022e0 0x7feb8019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7e7fc640 1 -- 172.21.15.103:0/2830197430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb801022e0 msgr2=0x7feb8019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7e7fc640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb801022e0 0x7feb8019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7e7fc640 1 -- 172.21.15.103:0/2830197430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb80100730 msgr2=0x7feb80197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7e7fc640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb80100730 0x7feb80197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7e7fc640 1 -- 172.21.15.103:0/2830197430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8019f6a0 con 0x7feb80101920 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7f7fe640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb801022e0 0x7feb8019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb7e7fc640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101920 0x7feb80197fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c890 tx=0x7feb6800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb77fff640 1 -- 172.21.15.103:0/2830197430 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68004220 con 0x7feb80101920 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8019f990 con 0x7feb80101920 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb77fff640 1 -- 172.21.15.103:0/2830197430 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb680043c0 con 0x7feb80101920 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb77fff640 1 -- 172.21.15.103:0/2830197430 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800f450 con 0x7feb80101920 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.456+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8019ff30 con 0x7feb80101920 2024-08-19T01:35:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.457+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb80107b70 con 0x7feb80101920 2024-08-19T01:35:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.458+0000 7feb77fff640 1 -- 172.21.15.103:0/2830197430 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feb68003680 con 0x7feb80101920 2024-08-19T01:35:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.459+0000 7feb77fff640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb6c0a2b90 0x7feb6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.459+0000 7feb7effd640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb6c0a2b90 0x7feb6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.459+0000 7feb77fff640 1 -- 172.21.15.103:0/2830197430 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7feb680cee00 con 0x7feb80101920 2024-08-19T01:35:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.459+0000 7feb7effd640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb6c0a2b90 0x7feb6c0a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb80101780 tx=0x7feb78005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.463+0000 7feb77fff640 1 -- 172.21.15.103:0/2830197430 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feb68014030 con 0x7feb80101920 2024-08-19T01:35:20.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.626+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb80062b50 con 0x7feb80101920 2024-08-19T01:35:20.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.626+0000 7feb77fff640 1 -- 172.21.15.103:0/2830197430 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7feb6809bd90 con 0x7feb80101920 2024-08-19T01:35:20.627 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-08-19T01:35:20.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb6c0a2b90 msgr2=0x7feb6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb6c0a2b90 0x7feb6c0a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb80101780 tx=0x7feb78005f90 comp rx=0 tx=0).stop 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101920 msgr2=0x7feb80197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101920 0x7feb80197fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c890 tx=0x7feb6800cd50 comp rx=0 tx=0).stop 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 shutdown_connections 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb6c0a2b90 0x7feb6c0a5050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb801022e0 0x7feb8019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb80101920 0x7feb80197fc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 --2- 172.21.15.103:0/2830197430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb80100730 0x7feb80197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.628+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 >> 172.21.15.103:0/2830197430 conn(0x7feb800fbe70 msgr2=0x7feb80104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.629+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 shutdown_connections 2024-08-19T01:35:20.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.629+0000 7feb85425640 1 -- 172.21.15.103:0/2830197430 wait complete. 2024-08-19T01:35:20.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-19T01:35:20.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 -- 172.21.15.103:0/2994812296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130102300 msgr2=0x7f21301046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 --2- 172.21.15.103:0/2994812296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130102300 0x7f21301046f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2120009900 tx=0x7f212002a410 comp rx=0 tx=0).stop 2024-08-19T01:35:20.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 -- 172.21.15.103:0/2994812296 shutdown_connections 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 --2- 172.21.15.103:0/2994812296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2130107560 0x7f2130109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 --2- 172.21.15.103:0/2994812296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2130104c30 0x7f2130107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 --2- 172.21.15.103:0/2994812296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130102300 0x7f21301046f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 -- 172.21.15.103:0/2994812296 >> 172.21.15.103:0/2994812296 conn(0x7f21300fbea0 msgr2=0x7f21300fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 -- 172.21.15.103:0/2994812296 shutdown_connections 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.742+0000 7f2134e25640 1 -- 172.21.15.103:0/2994812296 wait complete. 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 Processor -- start 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 -- start start 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2130102300 0x7f2130197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2130104c30 0x7f2130198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130107560 0x7f213019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f213010d3a0 con 0x7f2130107560 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f213010d220 con 0x7f2130104c30 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f2134e25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f213010d520 con 0x7f2130102300 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130107560 0x7f213019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130107560 0x7f213019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50240/0 (socket says 172.21.15.103:50240) 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 -- 172.21.15.103:0/1167135819 learned_addr learned my addr 172.21.15.103:0/1167135819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212dd9b640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2130104c30 0x7f2130198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212e59c640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2130102300 0x7f2130197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 -- 172.21.15.103:0/1167135819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2130102300 msgr2=0x7f2130197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2130102300 0x7f2130197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 -- 172.21.15.103:0/1167135819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2130104c30 msgr2=0x7f2130198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2130104c30 0x7f2130198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212ed9d640 1 -- 172.21.15.103:0/1167135819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213019f710 con 0x7f2130107560 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212dd9b640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2130104c30 0x7f2130198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.743+0000 7f212e59c640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2130102300 0x7f2130197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.744+0000 7f212ed9d640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130107560 0x7f213019f0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f212800e9d0 tx=0x7f212800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.744+0000 7f21277fe640 1 -- 172.21.15.103:0/1167135819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212800cd80 con 0x7f2130107560 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.744+0000 7f21277fe640 1 -- 172.21.15.103:0/1167135819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2128005020 con 0x7f2130107560 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.744+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213019fa00 con 0x7f2130107560 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.744+0000 7f21277fe640 1 -- 172.21.15.103:0/1167135819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21280116d0 con 0x7f2130107560 2024-08-19T01:35:20.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.744+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213019ff00 con 0x7f2130107560 2024-08-19T01:35:20.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.745+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21300fd990 con 0x7f2130107560 2024-08-19T01:35:20.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.746+0000 7f21277fe640 1 -- 172.21.15.103:0/1167135819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f21280051d0 con 0x7f2130107560 2024-08-19T01:35:20.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.746+0000 7f21277fe640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f211c0a2b90 0x7f211c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:20.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.746+0000 7f212e59c640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f211c0a2b90 0x7f211c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:20.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.747+0000 7f21277fe640 1 -- 172.21.15.103:0/1167135819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f21280cf2f0 con 0x7f2130107560 2024-08-19T01:35:20.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.747+0000 7f212e59c640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f211c0a2b90 0x7f211c0a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2120006010 tx=0x7f2120000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:20.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.750+0000 7f21277fe640 1 -- 172.21.15.103:0/1167135819 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2128016030 con 0x7f2130107560 2024-08-19T01:35:20.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.919+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2130062b20 con 0x7f2130107560 2024-08-19T01:35:20.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.919+0000 7f21277fe640 1 -- 172.21.15.103:0/1167135819 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f212809c280 con 0x7f2130107560 2024-08-19T01:35:20.921 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-08-19T01:35:20.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f211c0a2b90 msgr2=0x7f211c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f211c0a2b90 0x7f211c0a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2120006010 tx=0x7f2120000bc0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130107560 msgr2=0x7f213019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130107560 0x7f213019f0a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f212800e9d0 tx=0x7f212800ee90 comp rx=0 tx=0).stop 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 shutdown_connections 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f211c0a2b90 0x7f211c0a5050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2130107560 0x7f213019f0a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2130104c30 0x7f2130198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 --2- 172.21.15.103:0/1167135819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2130102300 0x7f2130197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 >> 172.21.15.103:0/1167135819 conn(0x7f21300fbea0 msgr2=0x7f2130102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 shutdown_connections 2024-08-19T01:35:20.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:20.921+0000 7f2134e25640 1 -- 172.21.15.103:0/1167135819 wait complete. 2024-08-19T01:35:20.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-19T01:35:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 -- 172.21.15.103:0/1116852317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0100740 msgr2=0x7f6de0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 --2- 172.21.15.103:0/1116852317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0100740 0x7f6de0100b40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009900 tx=0x7f6dd002a410 comp rx=0 tx=0).stop 2024-08-19T01:35:21.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 -- 172.21.15.103:0/1116852317 shutdown_connections 2024-08-19T01:35:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 --2- 172.21.15.103:0/1116852317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de01022f0 0x7f6de0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 --2- 172.21.15.103:0/1116852317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0101930 0x7f6de0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 --2- 172.21.15.103:0/1116852317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0100740 0x7f6de0100b40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 -- 172.21.15.103:0/1116852317 >> 172.21.15.103:0/1116852317 conn(0x7f6de00fbee0 msgr2=0x7f6de00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 -- 172.21.15.103:0/1116852317 shutdown_connections 2024-08-19T01:35:21.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.037+0000 7f6de4efc640 1 -- 172.21.15.103:0/1116852317 wait complete. 2024-08-19T01:35:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 Processor -- start 2024-08-19T01:35:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 -- start start 2024-08-19T01:35:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0100740 0x7f6de0197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0101930 0x7f6de0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:21.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de01022f0 0x7f6de019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010c270 con 0x7f6de0101930 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010c0f0 con 0x7f6de0100740 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6de4efc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010c3f0 con 0x7f6de01022f0 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.038+0000 7f6dd5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0101930 0x7f6de0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0101930 0x7f6de0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50246/0 (socket says 172.21.15.103:50246) 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 -- 172.21.15.103:0/3311027085 learned_addr learned my addr 172.21.15.103:0/3311027085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dde59c640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0100740 0x7f6de0197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dded9d640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de01022f0 0x7f6de019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 -- 172.21.15.103:0/3311027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de01022f0 msgr2=0x7f6de019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de01022f0 0x7f6de019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 -- 172.21.15.103:0/3311027085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0100740 msgr2=0x7f6de0197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0100740 0x7f6de0197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 -- 172.21.15.103:0/3311027085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de019f660 con 0x7f6de0101930 2024-08-19T01:35:21.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd5d9b640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0101930 0x7f6de0197f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c920 tx=0x7f6dc800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd7fff640 1 -- 172.21.15.103:0/3311027085 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8007c70 con 0x7f6de0101930 2024-08-19T01:35:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd7fff640 1 -- 172.21.15.103:0/3311027085 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dc8007e10 con 0x7f6de0101930 2024-08-19T01:35:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de019f950 con 0x7f6de0101930 2024-08-19T01:35:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6dd7fff640 1 -- 172.21.15.103:0/3311027085 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8018820 con 0x7f6de0101930 2024-08-19T01:35:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.039+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de019fea0 con 0x7f6de0101930 2024-08-19T01:35:21.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.040+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de01a0130 con 0x7f6de0101930 2024-08-19T01:35:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.041+0000 7f6dd7fff640 1 -- 172.21.15.103:0/3311027085 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6dc80189c0 con 0x7f6de0101930 2024-08-19T01:35:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.042+0000 7f6dd7fff640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6dcc0a2b90 0x7f6dcc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.043+0000 7f6dde59c640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6dcc0a2b90 0x7f6dcc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.043+0000 7f6dd7fff640 1 -- 172.21.15.103:0/3311027085 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6dc80cf140 con 0x7f6de0101930 2024-08-19T01:35:21.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.043+0000 7f6dde59c640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6dcc0a2b90 0x7f6dcc0a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0004770 tx=0x7f6dd0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:21.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.047+0000 7f6dd7fff640 1 -- 172.21.15.103:0/3311027085 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6dc8014030 con 0x7f6de0101930 2024-08-19T01:35:21.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.217+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6de01a0300 con 0x7f6de0101930 2024-08-19T01:35:21.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.217+0000 7f6dd7fff640 1 -- 172.21.15.103:0/3311027085 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f6dc809c0d0 con 0x7f6de0101930 2024-08-19T01:35:21.219 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6dcc0a2b90 msgr2=0x7f6dcc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6dcc0a2b90 0x7f6dcc0a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0004770 tx=0x7f6dd0000bc0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0101930 msgr2=0x7f6de0197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0101930 0x7f6de0197f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c920 tx=0x7f6dc800cde0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 shutdown_connections 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6dcc0a2b90 0x7f6dcc0a5050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6de01022f0 0x7f6de019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de0101930 0x7f6de0197f80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 --2- 172.21.15.103:0/3311027085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6de0100740 0x7f6de0197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:21.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 >> 172.21.15.103:0/3311027085 conn(0x7f6de00fbee0 msgr2=0x7f6de006a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:21.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 shutdown_connections 2024-08-19T01:35:21.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:21.220+0000 7f6de4efc640 1 -- 172.21.15.103:0/3311027085 wait complete. 2024-08-19T01:35:21.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:20 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1274107946' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-19T01:35:21.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:20 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:35:21.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:20 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/4045193318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:21.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:20 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2830197430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:35:21.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:20 smithi103 ceph-mon[18436]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:35:21.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:20 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1167135819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:35:21.233 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-19T01:35:21.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:35:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:20 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1274107946' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-19T01:35:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:20 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:35:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:20 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/4045193318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:20 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2830197430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:35:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:20 smithi190 ceph-mon[22032]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:35:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:20 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1167135819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:35:21.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:20 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1274107946' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-19T01:35:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:20 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:35:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:20 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/4045193318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:20 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2830197430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:35:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:20 smithi146 ceph-mon[22098]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:35:21.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:20 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1167135819' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:35:21.487 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 -- 172.21.15.103:0/1571958518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 msgr2=0x7f1e3410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 --2- 172.21.15.103:0/1571958518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e3410d770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009ca0 tx=0x7f1e2c02a950 comp rx=0 tx=0).stop 2024-08-19T01:35:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 -- 172.21.15.103:0/1571958518 shutdown_connections 2024-08-19T01:35:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 --2- 172.21.15.103:0/1571958518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e3410d770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 --2- 172.21.15.103:0/1571958518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e341022a0 0x7f1e341026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 --2- 172.21.15.103:0/1571958518 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e34100b50 0x7f1e34100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 -- 172.21.15.103:0/1571958518 >> 172.21.15.103:0/1571958518 conn(0x7f1e340fc500 msgr2=0x7f1e340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 -- 172.21.15.103:0/1571958518 shutdown_connections 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.009+0000 7f1e3c304640 1 -- 172.21.15.103:0/1571958518 wait complete. 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 Processor -- start 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 -- start start 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e34100b50 0x7f1e34198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e341022a0 0x7f1e34199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e341a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34110b60 con 0x7f1e34102c20 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341109e0 con 0x7f1e34100b50 2024-08-19T01:35:22.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3c304640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34110ce0 con 0x7f1e341022a0 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3a8a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e341a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3a8a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e341a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51272/0 (socket says 172.21.15.103:51272) 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3a8a1640 1 -- 172.21.15.103:0/2440694859 learned_addr learned my addr 172.21.15.103:0/2440694859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3a0a0640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e34100b50 0x7f1e34198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.010+0000 7f1e3989f640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e341022a0 0x7f1e34199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3a8a1640 1 -- 172.21.15.103:0/2440694859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e341022a0 msgr2=0x7f1e34199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3a8a1640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e341022a0 0x7f1e34199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3a8a1640 1 -- 172.21.15.103:0/2440694859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e34100b50 msgr2=0x7f1e34198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3a8a1640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e34100b50 0x7f1e34198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3a8a1640 1 -- 172.21.15.103:0/2440694859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e341a4590 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3a8a1640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e341a4050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c02ae70 tx=0x7f1e2c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/2440694859 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c034040 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e341a4760 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/2440694859 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e2c0308f0 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/2440694859 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c004780 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.011+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e341a4c10 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.012+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e34100f30 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.013+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/2440694859 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1e2c037020 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.014+0000 7f1e2b7fe640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1e200a2b80 0x7f1e200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.014+0000 7f1e3a0a0640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1e200a2b80 0x7f1e200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.014+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/2440694859 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1e2c0f0100 con 0x7f1e34102c20 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.014+0000 7f1e3a0a0640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1e200a2b80 0x7f1e200a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1e240058c0 tx=0x7f1e2400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:22.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.018+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/2440694859 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1e2c0bd090 con 0x7f1e34102c20 2024-08-19T01:35:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.220+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e34108dc0 con 0x7f1e34102c20 2024-08-19T01:35:22.223 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:21 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3311027085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:35:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.221+0000 7f1e2b7fe640 1 -- 172.21.15.103:0/2440694859 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7f1e2c0bc990 con 0x7f1e34102c20 2024-08-19T01:35:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-08-19T01:35:22.224 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:22.224 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-08-19T01:35:19:962857+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:35:19.321960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:35:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1e200a2b80 msgr2=0x7f1e200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1e200a2b80 0x7f1e200a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1e240058c0 tx=0x7f1e2400a2d0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 msgr2=0x7f1e341a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e341a4050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c02ae70 tx=0x7f1e2c02f5d0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 shutdown_connections 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1e200a2b80 0x7f1e200a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e34102c20 0x7f1e341a4050 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e341022a0 0x7f1e34199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 --2- 172.21.15.103:0/2440694859 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e34100b50 0x7f1e34198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 >> 172.21.15.103:0/2440694859 conn(0x7f1e340fc500 msgr2=0x7f1e340fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 shutdown_connections 2024-08-19T01:35:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:22.223+0000 7f1e3c304640 1 -- 172.21.15.103:0/2440694859 wait complete. 2024-08-19T01:35:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:21 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3311027085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:35:22.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:21 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3311027085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:35:23.228 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-19T01:35:23.228 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-19T01:35:23.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T01:35:23.253 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:22 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2440694859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:23.253 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:22 smithi103 ceph-mon[18436]: pgmap v327: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-08-19T01:35:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 -- 172.21.15.103:0/3230998915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 msgr2=0x7f9b68102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 --2- 172.21.15.103:0/3230998915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68102ed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009900 tx=0x7f9b6402a410 comp rx=0 tx=0).stop 2024-08-19T01:35:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 -- 172.21.15.103:0/3230998915 shutdown_connections 2024-08-19T01:35:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 --2- 172.21.15.103:0/3230998915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b68103410 0x7f9b68109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 --2- 172.21.15.103:0/3230998915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68102ed0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 --2- 172.21.15.103:0/3230998915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b68101860 0x7f9b68101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 -- 172.21.15.103:0/3230998915 >> 172.21.15.103:0/3230998915 conn(0x7f9b680fcfe0 msgr2=0x7f9b680ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 -- 172.21.15.103:0/3230998915 shutdown_connections 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.333+0000 7f9b70017640 1 -- 172.21.15.103:0/3230998915 wait complete. 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 Processor -- start 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 -- start start 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b68101860 0x7f9b68197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b68103410 0x7f9b6819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810d3e0 con 0x7f9b68102a50 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810d260 con 0x7f9b68101860 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b70017640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810d560 con 0x7f9b68103410 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b6d5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b6d5b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51284/0 (socket says 172.21.15.103:51284) 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b6d5b2640 1 -- 172.21.15.103:0/1913709700 learned_addr learned my addr 172.21.15.103:0/1913709700 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b6ddb3640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b68101860 0x7f9b68197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.334+0000 7f9b6e5b4640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b68103410 0x7f9b6819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b6d5b2640 1 -- 172.21.15.103:0/1913709700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b68103410 msgr2=0x7f9b6819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b6d5b2640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b68103410 0x7f9b6819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b6d5b2640 1 -- 172.21.15.103:0/1913709700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b68101860 msgr2=0x7f9b68197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b6d5b2640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b68101860 0x7f9b68197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b6d5b2640 1 -- 172.21.15.103:0/1913709700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6819f6b0 con 0x7f9b68102a50 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b6d5b2640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68197fd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b640314e0 tx=0x7f9b64003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b5effd640 1 -- 172.21.15.103:0/1913709700 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6403f070 con 0x7f9b68102a50 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b5effd640 1 -- 172.21.15.103:0/1913709700 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b64031e40 con 0x7f9b68102a50 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6819f940 con 0x7f9b68102a50 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b5effd640 1 -- 172.21.15.103:0/1913709700 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64035030 con 0x7f9b68102a50 2024-08-19T01:35:23.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.335+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6819fe30 con 0x7f9b68102a50 2024-08-19T01:35:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.336+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b68108c20 con 0x7f9b68102a50 2024-08-19T01:35:23.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.337+0000 7f9b5effd640 1 -- 172.21.15.103:0/1913709700 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9b64042070 con 0x7f9b68102a50 2024-08-19T01:35:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.338+0000 7f9b5effd640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9b540a2b90 0x7f9b540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.338+0000 7f9b6ddb3640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9b540a2b90 0x7f9b540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.338+0000 7f9b5effd640 1 -- 172.21.15.103:0/1913709700 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9b640ef960 con 0x7f9b68102a50 2024-08-19T01:35:23.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.338+0000 7f9b6ddb3640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9b540a2b90 0x7f9b540a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b5800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:23.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.342+0000 7f9b5effd640 1 -- 172.21.15.103:0/1913709700 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9b640bc8f0 con 0x7f9b68102a50 2024-08-19T01:35:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:22 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2440694859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:22 smithi190 ceph-mon[22032]: pgmap v327: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-08-19T01:35:23.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:22 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2440694859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:23.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:22 smithi146 ceph-mon[22098]: pgmap v327: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-08-19T01:35:23.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.507+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b68062360 con 0x7f9b540a2b90 2024-08-19T01:35:23.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.513+0000 7f9b5effd640 1 -- 172.21.15.103:0/1913709700 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259116 (secure 0 0 0) 0x7f9b68062360 con 0x7f9b540a2b90 2024-08-19T01:35:23.517 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:23.519 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T01:35:23.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9b540a2b90 msgr2=0x7f9b540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9b540a2b90 0x7f9b540a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b58005fd0 tx=0x7f9b5800b390 comp rx=0 tx=0).stop 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 msgr2=0x7f9b68197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68197fd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9b640314e0 tx=0x7f9b64003c80 comp rx=0 tx=0).stop 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 shutdown_connections 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f9b540a2b90 0x7f9b540a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b68103410 0x7f9b6819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b68102a50 0x7f9b68197fd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 --2- 172.21.15.103:0/1913709700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b68101860 0x7f9b68197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 >> 172.21.15.103:0/1913709700 conn(0x7f9b680fcfe0 msgr2=0x7f9b68105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 shutdown_connections 2024-08-19T01:35:23.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.516+0000 7f9b70017640 1 -- 172.21.15.103:0/1913709700 wait complete. 2024-08-19T01:35:23.531 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-08-19T01:35:22.800223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666256,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1239,"num_read_kb":744,"num_write":459,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364872,"kb_used_data":11560,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576952,"statfs":{"total":1151940427776,"available":1151566798848,"internally_reserved":0,"allocated":11837440,"data_stored":7422996,"data_compressed":112947,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6774,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002958"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804705+0000","last_change":"2024-08-19T01:33:23.202035+0000","last_active":"2024-08-19T01:35:00.804705+0000","last_peered":"2024-08-19T01:35:00.804705+0000","last_clean":"2024-08-19T01:35:00.804705+0000","last_became_active":"2024-08-19T01:33:23.201418+0000","last_became_peered":"2024-08-19T01:33:23.201418+0000","last_unstale":"2024-08-19T01:35:00.804705+0000","last_undegraded":"2024-08-19T01:35:00.804705+0000","last_fullsized":"2024-08-19T01:35:00.804705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:54:57.927424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802387+0000","last_change":"2024-08-19T01:33:23.202047+0000","last_active":"2024-08-19T01:35:00.802387+0000","last_peered":"2024-08-19T01:35:00.802387+0000","last_clean":"2024-08-19T01:35:00.802387+0000","last_became_active":"2024-08-19T01:33:23.201762+0000","last_became_peered":"2024-08-19T01:33:23.201762+0000","last_unstale":"2024-08-19T01:35:00.802387+0000","last_undegraded":"2024-08-19T01:35:00.802387+0000","last_fullsized":"2024-08-19T01:35:00.802387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:23:57.252284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804290+0000","last_change":"2024-08-19T01:33:23.202876+0000","last_active":"2024-08-19T01:35:00.804290+0000","last_peered":"2024-08-19T01:35:00.804290+0000","last_clean":"2024-08-19T01:35:00.804290+0000","last_became_active":"2024-08-19T01:33:23.202528+0000","last_became_peered":"2024-08-19T01:33:23.202528+0000","last_unstale":"2024-08-19T01:35:00.804290+0000","last_undegraded":"2024-08-19T01:35:00.804290+0000","last_fullsized":"2024-08-19T01:35:00.804290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:20.514390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790002+0000","last_change":"2024-08-19T01:33:23.199443+0000","last_active":"2024-08-19T01:35:00.790002+0000","last_peered":"2024-08-19T01:35:00.790002+0000","last_clean":"2024-08-19T01:35:00.790002+0000","last_became_active":"2024-08-19T01:33:23.198344+0000","last_became_peered":"2024-08-19T01:33:23.198344+0000","last_unstale":"2024-08-19T01:35:00.790002+0000","last_undegraded":"2024-08-19T01:35:00.790002+0000","last_fullsized":"2024-08-19T01:35:00.790002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:43:43.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800162+0000","last_change":"2024-08-19T01:33:23.191185+0000","last_active":"2024-08-19T01:35:00.800162+0000","last_peered":"2024-08-19T01:35:00.800162+0000","last_clean":"2024-08-19T01:35:00.800162+0000","last_became_active":"2024-08-19T01:33:23.190892+0000","last_became_peered":"2024-08-19T01:33:23.190892+0000","last_unstale":"2024-08-19T01:35:00.800162+0000","last_undegraded":"2024-08-19T01:35:00.800162+0000","last_fullsized":"2024-08-19T01:35:00.800162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:28.384609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.795199+0000","last_change":"2024-08-19T01:33:23.193036+0000","last_active":"2024-08-19T01:35:00.795199+0000","last_peered":"2024-08-19T01:35:00.795199+0000","last_clean":"2024-08-19T01:35:00.795199+0000","last_became_active":"2024-08-19T01:33:23.192686+0000","last_became_peered":"2024-08-19T01:33:23.192686+0000","last_unstale":"2024-08-19T01:35:00.795199+0000","last_undegraded":"2024-08-19T01:35:00.795199+0000","last_fullsized":"2024-08-19T01:35:00.795199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:46.235112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801472+0000","last_change":"2024-08-19T01:33:23.197636+0000","last_active":"2024-08-19T01:35:00.801472+0000","last_peered":"2024-08-19T01:35:00.801472+0000","last_clean":"2024-08-19T01:35:00.801472+0000","last_became_active":"2024-08-19T01:33:23.197323+0000","last_became_peered":"2024-08-19T01:33:23.197323+0000","last_unstale":"2024-08-19T01:35:00.801472+0000","last_undegraded":"2024-08-19T01:35:00.801472+0000","last_fullsized":"2024-08-19T01:35:00.801472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:46:36.168786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802002+0000","last_change":"2024-08-19T01:33:23.193717+0000","last_active":"2024-08-19T01:35:00.802002+0000","last_peered":"2024-08-19T01:35:00.802002+0000","last_clean":"2024-08-19T01:35:00.802002+0000","last_became_active":"2024-08-19T01:33:23.193376+0000","last_became_peered":"2024-08-19T01:33:23.193376+0000","last_unstale":"2024-08-19T01:35:00.802002+0000","last_undegraded":"2024-08-19T01:35:00.802002+0000","last_fullsized":"2024-08-19T01:35:00.802002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:16.290081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796833+0000","last_change":"2024-08-19T01:33:23.197956+0000","last_active":"2024-08-19T01:35:00.796833+0000","last_peered":"2024-08-19T01:35:00.796833+0000","last_clean":"2024-08-19T01:35:00.796833+0000","last_became_active":"2024-08-19T01:33:23.197667+0000","last_became_peered":"2024-08-19T01:33:23.197667+0000","last_unstale":"2024-08-19T01:35:00.796833+0000","last_undegraded":"2024-08-19T01:35:00.796833+0000","last_fullsized":"2024-08-19T01:35:00.796833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:16.841046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792010+0000","last_change":"2024-08-19T01:33:23.194569+0000","last_active":"2024-08-19T01:35:00.792010+0000","last_peered":"2024-08-19T01:35:00.792010+0000","last_clean":"2024-08-19T01:35:00.792010+0000","last_became_active":"2024-08-19T01:33:23.194223+0000","last_became_peered":"2024-08-19T01:33:23.194223+0000","last_unstale":"2024-08-19T01:35:00.792010+0000","last_undegraded":"2024-08-19T01:35:00.792010+0000","last_fullsized":"2024-08-19T01:35:00.792010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:14:30.184738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800081+0000","last_change":"2024-08-19T01:33:23.202396+0000","last_active":"2024-08-19T01:35:00.800081+0000","last_peered":"2024-08-19T01:35:00.800081+0000","last_clean":"2024-08-19T01:35:00.800081+0000","last_became_active":"2024-08-19T01:33:23.202063+0000","last_became_peered":"2024-08-19T01:33:23.202063+0000","last_unstale":"2024-08-19T01:35:00.800081+0000","last_undegraded":"2024-08-19T01:35:00.800081+0000","last_fullsized":"2024-08-19T01:35:00.800081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:57.513639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.789724+0000","last_change":"2024-08-19T01:33:23.200610+0000","last_active":"2024-08-19T01:35:00.789724+0000","last_peered":"2024-08-19T01:35:00.789724+0000","last_clean":"2024-08-19T01:35:00.789724+0000","last_became_active":"2024-08-19T01:33:23.200136+0000","last_became_peered":"2024-08-19T01:33:23.200136+0000","last_unstale":"2024-08-19T01:35:00.789724+0000","last_undegraded":"2024-08-19T01:35:00.789724+0000","last_fullsized":"2024-08-19T01:35:00.789724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:47:39.252702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792991+0000","last_change":"2024-08-19T01:33:23.196062+0000","last_active":"2024-08-19T01:35:00.792991+0000","last_peered":"2024-08-19T01:35:00.792991+0000","last_clean":"2024-08-19T01:35:00.792991+0000","last_became_active":"2024-08-19T01:33:23.195577+0000","last_became_peered":"2024-08-19T01:33:23.195577+0000","last_unstale":"2024-08-19T01:35:00.792991+0000","last_undegraded":"2024-08-19T01:35:00.792991+0000","last_fullsized":"2024-08-19T01:35:00.792991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:46:09.111323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804514+0000","last_change":"2024-08-19T01:33:23.201664+0000","last_active":"2024-08-19T01:35:00.804514+0000","last_peered":"2024-08-19T01:35:00.804514+0000","last_clean":"2024-08-19T01:35:00.804514+0000","last_became_active":"2024-08-19T01:33:23.200241+0000","last_became_peered":"2024-08-19T01:33:23.200241+0000","last_unstale":"2024-08-19T01:35:00.804514+0000","last_undegraded":"2024-08-19T01:35:00.804514+0000","last_fullsized":"2024-08-19T01:35:00.804514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:17.199776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803104+0000","last_change":"2024-08-19T01:33:23.198102+0000","last_active":"2024-08-19T01:35:00.803104+0000","last_peered":"2024-08-19T01:35:00.803104+0000","last_clean":"2024-08-19T01:35:00.803104+0000","last_became_active":"2024-08-19T01:33:23.197778+0000","last_became_peered":"2024-08-19T01:33:23.197778+0000","last_unstale":"2024-08-19T01:35:00.803104+0000","last_undegraded":"2024-08-19T01:35:00.803104+0000","last_fullsized":"2024-08-19T01:35:00.803104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:03.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804605+0000","last_change":"2024-08-19T01:33:23.194581+0000","last_active":"2024-08-19T01:35:00.804605+0000","last_peered":"2024-08-19T01:35:00.804605+0000","last_clean":"2024-08-19T01:35:00.804605+0000","last_became_active":"2024-08-19T01:33:23.194256+0000","last_became_peered":"2024-08-19T01:33:23.194256+0000","last_unstale":"2024-08-19T01:35:00.804605+0000","last_undegraded":"2024-08-19T01:35:00.804605+0000","last_fullsized":"2024-08-19T01:35:00.804605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:17:20.954190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804738+0000","last_change":"2024-08-19T01:33:23.201242+0000","last_active":"2024-08-19T01:35:00.804738+0000","last_peered":"2024-08-19T01:35:00.804738+0000","last_clean":"2024-08-19T01:35:00.804738+0000","last_became_active":"2024-08-19T01:33:23.200835+0000","last_became_peered":"2024-08-19T01:33:23.200835+0000","last_unstale":"2024-08-19T01:35:00.804738+0000","last_undegraded":"2024-08-19T01:35:00.804738+0000","last_fullsized":"2024-08-19T01:35:00.804738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:35:54.312266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.799770+0000","last_change":"2024-08-19T01:33:23.187728+0000","last_active":"2024-08-19T01:35:00.799770+0000","last_peered":"2024-08-19T01:35:00.799770+0000","last_clean":"2024-08-19T01:35:00.799770+0000","last_became_active":"2024-08-19T01:33:23.187352+0000","last_became_peered":"2024-08-19T01:33:23.187352+0000","last_unstale":"2024-08-19T01:35:00.799770+0000","last_undegraded":"2024-08-19T01:35:00.799770+0000","last_fullsized":"2024-08-19T01:35:00.799770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:06:03.041328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797919+0000","last_change":"2024-08-19T01:33:23.193146+0000","last_active":"2024-08-19T01:35:00.797919+0000","last_peered":"2024-08-19T01:35:00.797919+0000","last_clean":"2024-08-19T01:35:00.797919+0000","last_became_active":"2024-08-19T01:33:23.192810+0000","last_became_peered":"2024-08-19T01:33:23.192810+0000","last_unstale":"2024-08-19T01:35:00.797919+0000","last_undegraded":"2024-08-19T01:35:00.797919+0000","last_fullsized":"2024-08-19T01:35:00.797919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:12.044175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.791343+0000","last_change":"2024-08-19T01:33:23.194635+0000","last_active":"2024-08-19T01:35:00.791343+0000","last_peered":"2024-08-19T01:35:00.791343+0000","last_clean":"2024-08-19T01:35:00.791343+0000","last_became_active":"2024-08-19T01:33:23.194308+0000","last_became_peered":"2024-08-19T01:33:23.194308+0000","last_unstale":"2024-08-19T01:35:00.791343+0000","last_undegraded":"2024-08-19T01:35:00.791343+0000","last_fullsized":"2024-08-19T01:35:00.791343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:12.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802081+0000","last_change":"2024-08-19T01:33:23.193108+0000","last_active":"2024-08-19T01:35:00.802081+0000","last_peered":"2024-08-19T01:35:00.802081+0000","last_clean":"2024-08-19T01:35:00.802081+0000","last_became_active":"2024-08-19T01:33:23.192830+0000","last_became_peered":"2024-08-19T01:33:23.192830+0000","last_unstale":"2024-08-19T01:35:00.802081+0000","last_undegraded":"2024-08-19T01:35:00.802081+0000","last_fullsized":"2024-08-19T01:35:00.802081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:33.355498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803102+0000","last_change":"2024-08-19T01:33:23.195789+0000","last_active":"2024-08-19T01:35:00.803102+0000","last_peered":"2024-08-19T01:35:00.803102+0000","last_clean":"2024-08-19T01:35:00.803102+0000","last_became_active":"2024-08-19T01:33:23.195064+0000","last_became_peered":"2024-08-19T01:33:23.195064+0000","last_unstale":"2024-08-19T01:35:00.803102+0000","last_undegraded":"2024-08-19T01:35:00.803102+0000","last_fullsized":"2024-08-19T01:35:00.803102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:05:33.848953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802554+0000","last_change":"2024-08-19T01:33:23.200968+0000","last_active":"2024-08-19T01:35:00.802554+0000","last_peered":"2024-08-19T01:35:00.802554+0000","last_clean":"2024-08-19T01:35:00.802554+0000","last_became_active":"2024-08-19T01:33:23.200606+0000","last_became_peered":"2024-08-19T01:33:23.200606+0000","last_unstale":"2024-08-19T01:35:00.802554+0000","last_undegraded":"2024-08-19T01:35:00.802554+0000","last_fullsized":"2024-08-19T01:35:00.802554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:44.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.798216+0000","last_change":"2024-08-19T01:33:23.191204+0000","last_active":"2024-08-19T01:35:00.798216+0000","last_peered":"2024-08-19T01:35:00.798216+0000","last_clean":"2024-08-19T01:35:00.798216+0000","last_became_active":"2024-08-19T01:33:23.190709+0000","last_became_peered":"2024-08-19T01:33:23.190709+0000","last_unstale":"2024-08-19T01:35:00.798216+0000","last_undegraded":"2024-08-19T01:35:00.798216+0000","last_fullsized":"2024-08-19T01:35:00.798216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:56:10.988967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793707+0000","last_change":"2024-08-19T01:34:01.885389+0000","last_active":"2024-08-19T01:35:00.793707+0000","last_peered":"2024-08-19T01:35:00.793707+0000","last_clean":"2024-08-19T01:35:00.793707+0000","last_became_active":"2024-08-19T01:34:01.885100+0000","last_became_peered":"2024-08-19T01:34:01.885100+0000","last_unstale":"2024-08-19T01:35:00.793707+0000","last_undegraded":"2024-08-19T01:35:00.793707+0000","last_fullsized":"2024-08-19T01:35:00.793707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:21.616876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.782764+0000","last_change":"2024-08-19T01:33:23.187309+0000","last_active":"2024-08-19T01:35:00.782764+0000","last_peered":"2024-08-19T01:35:00.782764+0000","last_clean":"2024-08-19T01:35:00.782764+0000","last_became_active":"2024-08-19T01:33:23.186976+0000","last_became_peered":"2024-08-19T01:33:23.186976+0000","last_unstale":"2024-08-19T01:35:00.782764+0000","last_undegraded":"2024-08-19T01:35:00.782764+0000","last_fullsized":"2024-08-19T01:35:00.782764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:46:02.852000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790722+0000","last_change":"2024-08-19T01:33:23.190292+0000","last_active":"2024-08-19T01:35:00.790722+0000","last_peered":"2024-08-19T01:35:00.790722+0000","last_clean":"2024-08-19T01:35:00.790722+0000","last_became_active":"2024-08-19T01:33:23.189969+0000","last_became_peered":"2024-08-19T01:33:23.189969+0000","last_unstale":"2024-08-19T01:35:00.790722+0000","last_undegraded":"2024-08-19T01:35:00.790722+0000","last_fullsized":"2024-08-19T01:35:00.790722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:35:51.388150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796348+0000","last_change":"2024-08-19T01:33:23.192148+0000","last_active":"2024-08-19T01:35:00.796348+0000","last_peered":"2024-08-19T01:35:00.796348+0000","last_clean":"2024-08-19T01:35:00.796348+0000","last_became_active":"2024-08-19T01:33:23.191845+0000","last_became_peered":"2024-08-19T01:33:23.191845+0000","last_unstale":"2024-08-19T01:35:00.796348+0000","last_undegraded":"2024-08-19T01:35:00.796348+0000","last_fullsized":"2024-08-19T01:35:00.796348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:51.308393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803032+0000","last_change":"2024-08-19T01:33:23.195033+0000","last_active":"2024-08-19T01:35:00.803032+0000","last_peered":"2024-08-19T01:35:00.803032+0000","last_clean":"2024-08-19T01:35:00.803032+0000","last_became_active":"2024-08-19T01:33:23.194745+0000","last_became_peered":"2024-08-19T01:33:23.194745+0000","last_unstale":"2024-08-19T01:35:00.803032+0000","last_undegraded":"2024-08-19T01:35:00.803032+0000","last_fullsized":"2024-08-19T01:35:00.803032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:33:40.748311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797892+0000","last_change":"2024-08-19T01:33:23.195817+0000","last_active":"2024-08-19T01:35:00.797892+0000","last_peered":"2024-08-19T01:35:00.797892+0000","last_clean":"2024-08-19T01:35:00.797892+0000","last_became_active":"2024-08-19T01:33:23.194688+0000","last_became_peered":"2024-08-19T01:33:23.194688+0000","last_unstale":"2024-08-19T01:35:00.797892+0000","last_undegraded":"2024-08-19T01:35:00.797892+0000","last_fullsized":"2024-08-19T01:35:00.797892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:13:04.913921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793529+0000","last_change":"2024-08-19T01:33:23.187604+0000","last_active":"2024-08-19T01:35:00.793529+0000","last_peered":"2024-08-19T01:35:00.793529+0000","last_clean":"2024-08-19T01:35:00.793529+0000","last_became_active":"2024-08-19T01:33:23.187242+0000","last_became_peered":"2024-08-19T01:33:23.187242+0000","last_unstale":"2024-08-19T01:35:00.793529+0000","last_undegraded":"2024-08-19T01:35:00.793529+0000","last_fullsized":"2024-08-19T01:35:00.793529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:28.592525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792374+0000","last_change":"2024-08-19T01:33:23.191737+0000","last_active":"2024-08-19T01:35:00.792374+0000","last_peered":"2024-08-19T01:35:00.792374+0000","last_clean":"2024-08-19T01:35:00.792374+0000","last_became_active":"2024-08-19T01:33:23.191358+0000","last_became_peered":"2024-08-19T01:33:23.191358+0000","last_unstale":"2024-08-19T01:35:00.792374+0000","last_undegraded":"2024-08-19T01:35:00.792374+0000","last_fullsized":"2024-08-19T01:35:00.792374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:28:14.694157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.785064+0000","last_change":"2024-08-19T01:33:23.193110+0000","last_active":"2024-08-19T01:35:00.785064+0000","last_peered":"2024-08-19T01:35:00.785064+0000","last_clean":"2024-08-19T01:35:00.785064+0000","last_became_active":"2024-08-19T01:33:23.192789+0000","last_became_peered":"2024-08-19T01:33:23.192789+0000","last_unstale":"2024-08-19T01:35:00.785064+0000","last_undegraded":"2024-08-19T01:35:00.785064+0000","last_fullsized":"2024-08-19T01:35:00.785064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:45:48.141403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:20.124936+0000","last_change":"2024-08-19T01:33:21.173518+0000","last_active":"2024-08-19T01:35:20.124936+0000","last_peered":"2024-08-19T01:35:20.124936+0000","last_clean":"2024-08-19T01:35:20.124936+0000","last_became_active":"2024-08-19T01:33:21.172288+0000","last_became_peered":"2024-08-19T01:33:21.172288+0000","last_unstale":"2024-08-19T01:35:20.124936+0000","last_undegraded":"2024-08-19T01:35:20.124936+0000","last_fullsized":"2024-08-19T01:35:20.124936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:59:42.864724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1645,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801261+0000","last_change":"2024-08-19T01:33:23.190160+0000","last_active":"2024-08-19T01:35:00.801261+0000","last_peered":"2024-08-19T01:35:00.801261+0000","last_clean":"2024-08-19T01:35:00.801261+0000","last_became_active":"2024-08-19T01:33:23.189802+0000","last_became_peered":"2024-08-19T01:33:23.189802+0000","last_unstale":"2024-08-19T01:35:00.801261+0000","last_undegraded":"2024-08-19T01:35:00.801261+0000","last_fullsized":"2024-08-19T01:35:00.801261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:53:10.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880543+0000","last_change":"2024-08-19T01:33:21.167443+0000","last_active":"2024-08-19T01:34:01.880543+0000","last_peered":"2024-08-19T01:34:01.880543+0000","last_clean":"2024-08-19T01:34:01.880543+0000","last_became_active":"2024-08-19T01:33:21.167054+0000","last_became_peered":"2024-08-19T01:33:21.167054+0000","last_unstale":"2024-08-19T01:34:01.880543+0000","last_undegraded":"2024-08-19T01:34:01.880543+0000","last_fullsized":"2024-08-19T01:34:01.880543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:00:38.358807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928860+0000","last_change":"2024-08-19T01:33:21.163097+0000","last_active":"2024-08-19T01:34:01.928860+0000","last_peered":"2024-08-19T01:34:01.928860+0000","last_clean":"2024-08-19T01:34:01.928860+0000","last_became_active":"2024-08-19T01:33:21.162802+0000","last_became_peered":"2024-08-19T01:33:21.162802+0000","last_unstale":"2024-08-19T01:34:01.928860+0000","last_undegraded":"2024-08-19T01:34:01.928860+0000","last_fullsized":"2024-08-19T01:34:01.928860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:15:41.244104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803050+0000","last_change":"2024-08-19T01:33:23.200975+0000","last_active":"2024-08-19T01:35:00.803050+0000","last_peered":"2024-08-19T01:35:00.803050+0000","last_clean":"2024-08-19T01:35:00.803050+0000","last_became_active":"2024-08-19T01:33:23.200667+0000","last_became_peered":"2024-08-19T01:33:23.200667+0000","last_unstale":"2024-08-19T01:35:00.803050+0000","last_undegraded":"2024-08-19T01:35:00.803050+0000","last_fullsized":"2024-08-19T01:35:00.803050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:35.105521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879336+0000","last_change":"2024-08-19T01:33:21.171252+0000","last_active":"2024-08-19T01:34:01.879336+0000","last_peered":"2024-08-19T01:34:01.879336+0000","last_clean":"2024-08-19T01:34:01.879336+0000","last_became_active":"2024-08-19T01:33:21.170944+0000","last_became_peered":"2024-08-19T01:33:21.170944+0000","last_unstale":"2024-08-19T01:34:01.879336+0000","last_undegraded":"2024-08-19T01:34:01.879336+0000","last_fullsized":"2024-08-19T01:34:01.879336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:10:15.869452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801852+0000","last_change":"2024-08-19T01:33:23.191931+0000","last_active":"2024-08-19T01:35:00.801852+0000","last_peered":"2024-08-19T01:35:00.801852+0000","last_clean":"2024-08-19T01:35:00.801852+0000","last_became_active":"2024-08-19T01:33:23.191601+0000","last_became_peered":"2024-08-19T01:33:23.191601+0000","last_unstale":"2024-08-19T01:35:00.801852+0000","last_undegraded":"2024-08-19T01:35:00.801852+0000","last_fullsized":"2024-08-19T01:35:00.801852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:52.874546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":243,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.125745+0000","last_change":"2024-08-19T01:33:21.170714+0000","last_active":"2024-08-19T01:35:21.125745+0000","last_peered":"2024-08-19T01:35:21.125745+0000","last_clean":"2024-08-19T01:35:21.125745+0000","last_became_active":"2024-08-19T01:33:21.170379+0000","last_became_peered":"2024-08-19T01:33:21.170379+0000","last_unstale":"2024-08-19T01:35:21.125745+0000","last_undegraded":"2024-08-19T01:35:21.125745+0000","last_fullsized":"2024-08-19T01:35:21.125745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:27:27.417634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.791818+0000","last_change":"2024-08-19T01:33:23.189680+0000","last_active":"2024-08-19T01:35:00.791818+0000","last_peered":"2024-08-19T01:35:00.791818+0000","last_clean":"2024-08-19T01:35:00.791818+0000","last_became_active":"2024-08-19T01:33:23.189029+0000","last_became_peered":"2024-08-19T01:33:23.189029+0000","last_unstale":"2024-08-19T01:35:00.791818+0000","last_undegraded":"2024-08-19T01:35:00.791818+0000","last_fullsized":"2024-08-19T01:35:00.791818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:39.667675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881530+0000","last_change":"2024-08-19T01:33:21.169233+0000","last_active":"2024-08-19T01:34:01.881530+0000","last_peered":"2024-08-19T01:34:01.881530+0000","last_clean":"2024-08-19T01:34:01.881530+0000","last_became_active":"2024-08-19T01:33:21.168761+0000","last_became_peered":"2024-08-19T01:33:21.168761+0000","last_unstale":"2024-08-19T01:34:01.881530+0000","last_undegraded":"2024-08-19T01:34:01.881530+0000","last_fullsized":"2024-08-19T01:34:01.881530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:31:56.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804526+0000","last_change":"2024-08-19T01:33:23.194250+0000","last_active":"2024-08-19T01:35:00.804526+0000","last_peered":"2024-08-19T01:35:00.804526+0000","last_clean":"2024-08-19T01:35:00.804526+0000","last_became_active":"2024-08-19T01:33:23.193939+0000","last_became_peered":"2024-08-19T01:33:23.193939+0000","last_unstale":"2024-08-19T01:35:00.804526+0000","last_undegraded":"2024-08-19T01:35:00.804526+0000","last_fullsized":"2024-08-19T01:35:00.804526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:19:09.703701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879370+0000","last_change":"2024-08-19T01:33:21.287684+0000","last_active":"2024-08-19T01:34:01.879370+0000","last_peered":"2024-08-19T01:34:01.879370+0000","last_clean":"2024-08-19T01:34:01.879370+0000","last_became_active":"2024-08-19T01:33:21.287252+0000","last_became_peered":"2024-08-19T01:33:21.287252+0000","last_unstale":"2024-08-19T01:34:01.879370+0000","last_undegraded":"2024-08-19T01:34:01.879370+0000","last_fullsized":"2024-08-19T01:34:01.879370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:08.801345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.788353+0000","last_change":"2024-08-19T01:33:23.195077+0000","last_active":"2024-08-19T01:35:00.788353+0000","last_peered":"2024-08-19T01:35:00.788353+0000","last_clean":"2024-08-19T01:35:00.788353+0000","last_became_active":"2024-08-19T01:33:23.194753+0000","last_became_peered":"2024-08-19T01:33:23.194753+0000","last_unstale":"2024-08-19T01:35:00.788353+0000","last_undegraded":"2024-08-19T01:35:00.788353+0000","last_fullsized":"2024-08-19T01:35:00.788353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:12:09.186261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":282,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.125848+0000","last_change":"2024-08-19T01:33:21.168845+0000","last_active":"2024-08-19T01:35:21.125848+0000","last_peered":"2024-08-19T01:35:21.125848+0000","last_clean":"2024-08-19T01:35:21.125848+0000","last_became_active":"2024-08-19T01:33:21.168479+0000","last_became_peered":"2024-08-19T01:33:21.168479+0000","last_unstale":"2024-08-19T01:35:21.125848+0000","last_undegraded":"2024-08-19T01:35:21.125848+0000","last_fullsized":"2024-08-19T01:35:21.125848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:59:50.296329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65671,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":224,"num_read_kb":171,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.785907+0000","last_change":"2024-08-19T01:33:23.192252+0000","last_active":"2024-08-19T01:35:00.785907+0000","last_peered":"2024-08-19T01:35:00.785907+0000","last_clean":"2024-08-19T01:35:00.785907+0000","last_became_active":"2024-08-19T01:33:23.191934+0000","last_became_peered":"2024-08-19T01:33:23.191934+0000","last_unstale":"2024-08-19T01:35:00.785907+0000","last_undegraded":"2024-08-19T01:35:00.785907+0000","last_fullsized":"2024-08-19T01:35:00.785907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:31.134825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882306+0000","last_change":"2024-08-19T01:33:21.168541+0000","last_active":"2024-08-19T01:34:01.882306+0000","last_peered":"2024-08-19T01:34:01.882306+0000","last_clean":"2024-08-19T01:34:01.882306+0000","last_became_active":"2024-08-19T01:33:21.168205+0000","last_became_peered":"2024-08-19T01:33:21.168205+0000","last_unstale":"2024-08-19T01:34:01.882306+0000","last_undegraded":"2024-08-19T01:34:01.882306+0000","last_fullsized":"2024-08-19T01:34:01.882306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:19:52.078947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.799749+0000","last_change":"2024-08-19T01:33:23.193013+0000","last_active":"2024-08-19T01:35:00.799749+0000","last_peered":"2024-08-19T01:35:00.799749+0000","last_clean":"2024-08-19T01:35:00.799749+0000","last_became_active":"2024-08-19T01:33:23.192728+0000","last_became_peered":"2024-08-19T01:33:23.192728+0000","last_unstale":"2024-08-19T01:35:00.799749+0000","last_undegraded":"2024-08-19T01:35:00.799749+0000","last_fullsized":"2024-08-19T01:35:00.799749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:23:17.587527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879101+0000","last_change":"2024-08-19T01:33:21.170118+0000","last_active":"2024-08-19T01:34:01.879101+0000","last_peered":"2024-08-19T01:34:01.879101+0000","last_clean":"2024-08-19T01:34:01.879101+0000","last_became_active":"2024-08-19T01:33:21.169757+0000","last_became_peered":"2024-08-19T01:33:21.169757+0000","last_unstale":"2024-08-19T01:34:01.879101+0000","last_undegraded":"2024-08-19T01:34:01.879101+0000","last_fullsized":"2024-08-19T01:34:01.879101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:09:26.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804979+0000","last_change":"2024-08-19T01:34:01.929944+0000","last_active":"2024-08-19T01:35:00.804979+0000","last_peered":"2024-08-19T01:35:00.804979+0000","last_clean":"2024-08-19T01:35:00.804979+0000","last_became_active":"2024-08-19T01:34:01.929679+0000","last_became_peered":"2024-08-19T01:34:01.929679+0000","last_unstale":"2024-08-19T01:35:00.804979+0000","last_undegraded":"2024-08-19T01:35:00.804979+0000","last_fullsized":"2024-08-19T01:35:00.804979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:25:24.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.128678+0000","last_change":"2024-08-19T01:33:21.165204+0000","last_active":"2024-08-19T01:35:18.128678+0000","last_peered":"2024-08-19T01:35:18.128678+0000","last_clean":"2024-08-19T01:35:18.128678+0000","last_became_active":"2024-08-19T01:33:21.164826+0000","last_became_peered":"2024-08-19T01:33:21.164826+0000","last_unstale":"2024-08-19T01:35:18.128678+0000","last_undegraded":"2024-08-19T01:35:18.128678+0000","last_fullsized":"2024-08-19T01:35:18.128678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:45:23.212064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800773+0000","last_change":"2024-08-19T01:33:23.198176+0000","last_active":"2024-08-19T01:35:00.800773+0000","last_peered":"2024-08-19T01:35:00.800773+0000","last_clean":"2024-08-19T01:35:00.800773+0000","last_became_active":"2024-08-19T01:33:23.196187+0000","last_became_peered":"2024-08-19T01:33:23.196187+0000","last_unstale":"2024-08-19T01:35:00.800773+0000","last_undegraded":"2024-08-19T01:35:00.800773+0000","last_fullsized":"2024-08-19T01:35:00.800773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:53:05.943467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.079412+0000","last_change":"2024-08-19T01:34:02.079028+0000","last_active":"2024-08-19T01:34:02.079412+0000","last_peered":"2024-08-19T01:34:02.079412+0000","last_clean":"2024-08-19T01:34:02.079412+0000","last_became_active":"2024-08-19T01:34:02.049890+0000","last_became_peered":"2024-08-19T01:34:02.049890+0000","last_unstale":"2024-08-19T01:34:02.079412+0000","last_undegraded":"2024-08-19T01:34:02.079412+0000","last_fullsized":"2024-08-19T01:34:02.079412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:58.595161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.789928+0000","last_change":"2024-08-19T01:33:23.201104+0000","last_active":"2024-08-19T01:35:00.789928+0000","last_peered":"2024-08-19T01:35:00.789928+0000","last_clean":"2024-08-19T01:35:00.789928+0000","last_became_active":"2024-08-19T01:33:23.200809+0000","last_became_peered":"2024-08-19T01:33:23.200809+0000","last_unstale":"2024-08-19T01:35:00.789928+0000","last_undegraded":"2024-08-19T01:35:00.789928+0000","last_fullsized":"2024-08-19T01:35:00.789928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:54.901412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.884029+0000","last_change":"2024-08-19T01:33:21.172423+0000","last_active":"2024-08-19T01:34:01.884029+0000","last_peered":"2024-08-19T01:34:01.884029+0000","last_clean":"2024-08-19T01:34:01.884029+0000","last_became_active":"2024-08-19T01:33:21.171950+0000","last_became_peered":"2024-08-19T01:33:21.171950+0000","last_unstale":"2024-08-19T01:34:01.884029+0000","last_undegraded":"2024-08-19T01:34:01.884029+0000","last_fullsized":"2024-08-19T01:34:01.884029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:44:11.446875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790970+0000","last_change":"2024-08-19T01:33:23.192933+0000","last_active":"2024-08-19T01:35:00.790970+0000","last_peered":"2024-08-19T01:35:00.790970+0000","last_clean":"2024-08-19T01:35:00.790970+0000","last_became_active":"2024-08-19T01:33:23.192601+0000","last_became_peered":"2024-08-19T01:33:23.192601+0000","last_unstale":"2024-08-19T01:35:00.790970+0000","last_undegraded":"2024-08-19T01:35:00.790970+0000","last_fullsized":"2024-08-19T01:35:00.790970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:09.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929233+0000","last_change":"2024-08-19T01:33:21.168225+0000","last_active":"2024-08-19T01:34:01.929233+0000","last_peered":"2024-08-19T01:34:01.929233+0000","last_clean":"2024-08-19T01:34:01.929233+0000","last_became_active":"2024-08-19T01:33:21.167881+0000","last_became_peered":"2024-08-19T01:33:21.167881+0000","last_unstale":"2024-08-19T01:34:01.929233+0000","last_undegraded":"2024-08-19T01:34:01.929233+0000","last_fullsized":"2024-08-19T01:34:01.929233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:53:38.037036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793586+0000","last_change":"2024-08-19T01:33:23.194079+0000","last_active":"2024-08-19T01:35:00.793586+0000","last_peered":"2024-08-19T01:35:00.793586+0000","last_clean":"2024-08-19T01:35:00.793586+0000","last_became_active":"2024-08-19T01:33:23.193749+0000","last_became_peered":"2024-08-19T01:33:23.193749+0000","last_unstale":"2024-08-19T01:35:00.793586+0000","last_undegraded":"2024-08-19T01:35:00.793586+0000","last_fullsized":"2024-08-19T01:35:00.793586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:53:41.102163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879491+0000","last_change":"2024-08-19T01:33:21.168916+0000","last_active":"2024-08-19T01:34:01.879491+0000","last_peered":"2024-08-19T01:34:01.879491+0000","last_clean":"2024-08-19T01:34:01.879491+0000","last_became_active":"2024-08-19T01:33:21.168618+0000","last_became_peered":"2024-08-19T01:33:21.168618+0000","last_unstale":"2024-08-19T01:34:01.879491+0000","last_undegraded":"2024-08-19T01:34:01.879491+0000","last_fullsized":"2024-08-19T01:34:01.879491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:45:11.567779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.798985+0000","last_change":"2024-08-19T01:33:23.195546+0000","last_active":"2024-08-19T01:35:00.798985+0000","last_peered":"2024-08-19T01:35:00.798985+0000","last_clean":"2024-08-19T01:35:00.798985+0000","last_became_active":"2024-08-19T01:33:23.195221+0000","last_became_peered":"2024-08-19T01:33:23.195221+0000","last_unstale":"2024-08-19T01:35:00.798985+0000","last_undegraded":"2024-08-19T01:35:00.798985+0000","last_fullsized":"2024-08-19T01:35:00.798985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:46:10.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.123644+0000","last_change":"2024-08-19T01:33:21.287946+0000","last_active":"2024-08-19T01:35:18.123644+0000","last_peered":"2024-08-19T01:35:18.123644+0000","last_clean":"2024-08-19T01:35:18.123644+0000","last_became_active":"2024-08-19T01:33:21.287594+0000","last_became_peered":"2024-08-19T01:33:21.287594+0000","last_unstale":"2024-08-19T01:35:18.123644+0000","last_undegraded":"2024-08-19T01:35:18.123644+0000","last_fullsized":"2024-08-19T01:35:18.123644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:10:06.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1662,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800058+0000","last_change":"2024-08-19T01:33:23.200914+0000","last_active":"2024-08-19T01:35:00.800058+0000","last_peered":"2024-08-19T01:35:00.800058+0000","last_clean":"2024-08-19T01:35:00.800058+0000","last_became_active":"2024-08-19T01:33:23.200611+0000","last_became_peered":"2024-08-19T01:33:23.200611+0000","last_unstale":"2024-08-19T01:35:00.800058+0000","last_undegraded":"2024-08-19T01:35:00.800058+0000","last_fullsized":"2024-08-19T01:35:00.800058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:22:00.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880899+0000","last_change":"2024-08-19T01:32:09.926979+0000","last_active":"2024-08-19T01:34:01.880899+0000","last_peered":"2024-08-19T01:34:01.880899+0000","last_clean":"2024-08-19T01:34:01.880899+0000","last_became_active":"2024-08-19T01:32:09.926560+0000","last_became_peered":"2024-08-19T01:32:09.926560+0000","last_unstale":"2024-08-19T01:34:01.880899+0000","last_undegraded":"2024-08-19T01:34:01.880899+0000","last_fullsized":"2024-08-19T01:34:01.880899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_clean_scrub_stamp":"2024-08-19T01:29:05.356703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:12:23.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878743+0000","last_change":"2024-08-19T01:33:21.165099+0000","last_active":"2024-08-19T01:34:01.878743+0000","last_peered":"2024-08-19T01:34:01.878743+0000","last_clean":"2024-08-19T01:34:01.878743+0000","last_became_active":"2024-08-19T01:33:21.164756+0000","last_became_peered":"2024-08-19T01:33:21.164756+0000","last_unstale":"2024-08-19T01:34:01.878743+0000","last_undegraded":"2024-08-19T01:34:01.878743+0000","last_fullsized":"2024-08-19T01:34:01.878743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:11:54.480263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793548+0000","last_change":"2024-08-19T01:33:23.200185+0000","last_active":"2024-08-19T01:35:00.793548+0000","last_peered":"2024-08-19T01:35:00.793548+0000","last_clean":"2024-08-19T01:35:00.793548+0000","last_became_active":"2024-08-19T01:33:23.199885+0000","last_became_peered":"2024-08-19T01:33:23.199885+0000","last_unstale":"2024-08-19T01:35:00.793548+0000","last_undegraded":"2024-08-19T01:35:00.793548+0000","last_fullsized":"2024-08-19T01:35:00.793548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:17.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880804+0000","last_change":"2024-08-19T01:33:21.172587+0000","last_active":"2024-08-19T01:34:01.880804+0000","last_peered":"2024-08-19T01:34:01.880804+0000","last_clean":"2024-08-19T01:34:01.880804+0000","last_became_active":"2024-08-19T01:33:21.172233+0000","last_became_peered":"2024-08-19T01:33:21.172233+0000","last_unstale":"2024-08-19T01:34:01.880804+0000","last_undegraded":"2024-08-19T01:34:01.880804+0000","last_fullsized":"2024-08-19T01:34:01.880804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:26.690403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.795949+0000","last_change":"2024-08-19T01:33:23.197272+0000","last_active":"2024-08-19T01:35:00.795949+0000","last_peered":"2024-08-19T01:35:00.795949+0000","last_clean":"2024-08-19T01:35:00.795949+0000","last_became_active":"2024-08-19T01:33:23.196869+0000","last_became_peered":"2024-08-19T01:33:23.196869+0000","last_unstale":"2024-08-19T01:35:00.795949+0000","last_undegraded":"2024-08-19T01:35:00.795949+0000","last_fullsized":"2024-08-19T01:35:00.795949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:30.020184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:19.129363+0000","last_change":"2024-08-19T01:33:21.287366+0000","last_active":"2024-08-19T01:35:19.129363+0000","last_peered":"2024-08-19T01:35:19.129363+0000","last_clean":"2024-08-19T01:35:19.129363+0000","last_became_active":"2024-08-19T01:33:21.287044+0000","last_became_peered":"2024-08-19T01:33:21.287044+0000","last_unstale":"2024-08-19T01:35:19.129363+0000","last_undegraded":"2024-08-19T01:35:19.129363+0000","last_fullsized":"2024-08-19T01:35:19.129363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:19:04.194284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792226+0000","last_change":"2024-08-19T01:34:01.885001+0000","last_active":"2024-08-19T01:35:00.792226+0000","last_peered":"2024-08-19T01:35:00.792226+0000","last_clean":"2024-08-19T01:35:00.792226+0000","last_became_active":"2024-08-19T01:34:01.884680+0000","last_became_peered":"2024-08-19T01:34:01.884680+0000","last_unstale":"2024-08-19T01:35:00.792226+0000","last_undegraded":"2024-08-19T01:35:00.792226+0000","last_fullsized":"2024-08-19T01:35:00.792226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:08.251298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"86'8","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.148454+0000","last_change":"2024-08-19T01:33:21.164639+0000","last_active":"2024-08-19T01:35:21.148454+0000","last_peered":"2024-08-19T01:35:21.148454+0000","last_clean":"2024-08-19T01:35:21.148454+0000","last_became_active":"2024-08-19T01:33:21.164330+0000","last_became_peered":"2024-08-19T01:33:21.164330+0000","last_unstale":"2024-08-19T01:35:21.148454+0000","last_undegraded":"2024-08-19T01:35:21.148454+0000","last_fullsized":"2024-08-19T01:35:21.148454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:26:04.035130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801157+0000","last_change":"2024-08-19T01:34:01.883666+0000","last_active":"2024-08-19T01:35:00.801157+0000","last_peered":"2024-08-19T01:35:00.801157+0000","last_clean":"2024-08-19T01:35:00.801157+0000","last_became_active":"2024-08-19T01:34:01.883264+0000","last_became_peered":"2024-08-19T01:34:01.883264+0000","last_unstale":"2024-08-19T01:35:00.801157+0000","last_undegraded":"2024-08-19T01:35:00.801157+0000","last_fullsized":"2024-08-19T01:35:00.801157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:41:52.266751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879931+0000","last_change":"2024-08-19T01:33:21.287906+0000","last_active":"2024-08-19T01:34:01.879931+0000","last_peered":"2024-08-19T01:34:01.879931+0000","last_clean":"2024-08-19T01:34:01.879931+0000","last_became_active":"2024-08-19T01:33:21.287597+0000","last_became_peered":"2024-08-19T01:33:21.287597+0000","last_unstale":"2024-08-19T01:34:01.879931+0000","last_undegraded":"2024-08-19T01:34:01.879931+0000","last_fullsized":"2024-08-19T01:34:01.879931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:30.605948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801131+0000","last_change":"2024-08-19T01:33:23.191026+0000","last_active":"2024-08-19T01:35:00.801131+0000","last_peered":"2024-08-19T01:35:00.801131+0000","last_clean":"2024-08-19T01:35:00.801131+0000","last_became_active":"2024-08-19T01:33:23.190382+0000","last_became_peered":"2024-08-19T01:33:23.190382+0000","last_unstale":"2024-08-19T01:35:00.801131+0000","last_undegraded":"2024-08-19T01:35:00.801131+0000","last_fullsized":"2024-08-19T01:35:00.801131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:33:45.065441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883898+0000","last_change":"2024-08-19T01:33:21.288008+0000","last_active":"2024-08-19T01:34:01.883898+0000","last_peered":"2024-08-19T01:34:01.883898+0000","last_clean":"2024-08-19T01:34:01.883898+0000","last_became_active":"2024-08-19T01:33:21.287613+0000","last_became_peered":"2024-08-19T01:33:21.287613+0000","last_unstale":"2024-08-19T01:34:01.883898+0000","last_undegraded":"2024-08-19T01:34:01.883898+0000","last_fullsized":"2024-08-19T01:34:01.883898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:35:18.169564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796936+0000","last_change":"2024-08-19T01:33:23.197201+0000","last_active":"2024-08-19T01:35:00.796936+0000","last_peered":"2024-08-19T01:35:00.796936+0000","last_clean":"2024-08-19T01:35:00.796936+0000","last_became_active":"2024-08-19T01:33:23.196946+0000","last_became_peered":"2024-08-19T01:33:23.196946+0000","last_unstale":"2024-08-19T01:35:00.796936+0000","last_undegraded":"2024-08-19T01:35:00.796936+0000","last_fullsized":"2024-08-19T01:35:00.796936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:14:31.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'18","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.195232+0000","last_change":"2024-08-19T01:33:21.167109+0000","last_active":"2024-08-19T01:35:18.195232+0000","last_peered":"2024-08-19T01:35:18.195232+0000","last_clean":"2024-08-19T01:35:18.195232+0000","last_became_active":"2024-08-19T01:33:21.166788+0000","last_became_peered":"2024-08-19T01:33:21.166788+0000","last_unstale":"2024-08-19T01:35:18.195232+0000","last_undegraded":"2024-08-19T01:35:18.195232+0000","last_fullsized":"2024-08-19T01:35:18.195232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:44.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803153+0000","last_change":"2024-08-19T01:33:23.199699+0000","last_active":"2024-08-19T01:35:00.803153+0000","last_peered":"2024-08-19T01:35:00.803153+0000","last_clean":"2024-08-19T01:35:00.803153+0000","last_became_active":"2024-08-19T01:33:23.199417+0000","last_became_peered":"2024-08-19T01:33:23.199417+0000","last_unstale":"2024-08-19T01:35:00.803153+0000","last_undegraded":"2024-08-19T01:35:00.803153+0000","last_fullsized":"2024-08-19T01:35:00.803153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:36.519135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878888+0000","last_change":"2024-08-19T01:33:21.164177+0000","last_active":"2024-08-19T01:34:01.878888+0000","last_peered":"2024-08-19T01:34:01.878888+0000","last_clean":"2024-08-19T01:34:01.878888+0000","last_became_active":"2024-08-19T01:33:21.163832+0000","last_became_peered":"2024-08-19T01:33:21.163832+0000","last_unstale":"2024-08-19T01:34:01.878888+0000","last_undegraded":"2024-08-19T01:34:01.878888+0000","last_fullsized":"2024-08-19T01:34:01.878888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:31.907677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804059+0000","last_change":"2024-08-19T01:33:23.195102+0000","last_active":"2024-08-19T01:35:00.804059+0000","last_peered":"2024-08-19T01:35:00.804059+0000","last_clean":"2024-08-19T01:35:00.804059+0000","last_became_active":"2024-08-19T01:33:23.194802+0000","last_became_peered":"2024-08-19T01:33:23.194802+0000","last_unstale":"2024-08-19T01:35:00.804059+0000","last_undegraded":"2024-08-19T01:35:00.804059+0000","last_fullsized":"2024-08-19T01:35:00.804059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:33.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048628+0000","last_change":"2024-08-19T01:33:21.287933+0000","last_active":"2024-08-19T01:34:02.048628+0000","last_peered":"2024-08-19T01:34:02.048628+0000","last_clean":"2024-08-19T01:34:02.048628+0000","last_became_active":"2024-08-19T01:33:21.287568+0000","last_became_peered":"2024-08-19T01:33:21.287568+0000","last_unstale":"2024-08-19T01:34:02.048628+0000","last_undegraded":"2024-08-19T01:34:02.048628+0000","last_fullsized":"2024-08-19T01:34:02.048628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:23.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.780229+0000","last_change":"2024-08-19T01:33:23.194320+0000","last_active":"2024-08-19T01:35:00.780229+0000","last_peered":"2024-08-19T01:35:00.780229+0000","last_clean":"2024-08-19T01:35:00.780229+0000","last_became_active":"2024-08-19T01:33:23.194002+0000","last_became_peered":"2024-08-19T01:33:23.194002+0000","last_unstale":"2024-08-19T01:35:00.780229+0000","last_undegraded":"2024-08-19T01:35:00.780229+0000","last_fullsized":"2024-08-19T01:35:00.780229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:59.198997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877938+0000","last_change":"2024-08-19T01:33:21.165421+0000","last_active":"2024-08-19T01:34:01.877938+0000","last_peered":"2024-08-19T01:34:01.877938+0000","last_clean":"2024-08-19T01:34:01.877938+0000","last_became_active":"2024-08-19T01:33:21.165013+0000","last_became_peered":"2024-08-19T01:33:21.165013+0000","last_unstale":"2024-08-19T01:34:01.877938+0000","last_undegraded":"2024-08-19T01:34:01.877938+0000","last_fullsized":"2024-08-19T01:34:01.877938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:07:18.546099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803714+0000","last_change":"2024-08-19T01:33:23.193544+0000","last_active":"2024-08-19T01:35:00.803714+0000","last_peered":"2024-08-19T01:35:00.803714+0000","last_clean":"2024-08-19T01:35:00.803714+0000","last_became_active":"2024-08-19T01:33:23.193213+0000","last_became_peered":"2024-08-19T01:33:23.193213+0000","last_unstale":"2024-08-19T01:35:00.803714+0000","last_undegraded":"2024-08-19T01:35:00.803714+0000","last_fullsized":"2024-08-19T01:35:00.803714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:20:54.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:17.948248+0000","last_change":"2024-08-19T01:34:01.889627+0000","last_active":"2024-08-19T01:35:17.948248+0000","last_peered":"2024-08-19T01:35:17.948248+0000","last_clean":"2024-08-19T01:35:17.948248+0000","last_became_active":"2024-08-19T01:34:01.885181+0000","last_became_peered":"2024-08-19T01:34:01.885181+0000","last_unstale":"2024-08-19T01:35:17.948248+0000","last_undegraded":"2024-08-19T01:35:17.948248+0000","last_fullsized":"2024-08-19T01:35:17.948248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:51.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801718+0000","last_change":"2024-08-19T01:33:23.196266+0000","last_active":"2024-08-19T01:35:00.801718+0000","last_peered":"2024-08-19T01:35:00.801718+0000","last_clean":"2024-08-19T01:35:00.801718+0000","last_became_active":"2024-08-19T01:33:23.194268+0000","last_became_peered":"2024-08-19T01:33:23.194268+0000","last_unstale":"2024-08-19T01:35:00.801718+0000","last_undegraded":"2024-08-19T01:35:00.801718+0000","last_fullsized":"2024-08-19T01:35:00.801718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:15:38.628746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.890210+0000","last_change":"2024-08-19T01:34:01.890210+0000","last_active":"2024-08-19T01:34:01.890210+0000","last_peered":"2024-08-19T01:34:01.890210+0000","last_clean":"2024-08-19T01:34:01.890210+0000","last_became_active":"2024-08-19T01:34:01.885955+0000","last_became_peered":"2024-08-19T01:34:01.885955+0000","last_unstale":"2024-08-19T01:34:01.890210+0000","last_undegraded":"2024-08-19T01:34:01.890210+0000","last_fullsized":"2024-08-19T01:34:01.890210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.176530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804239+0000","last_change":"2024-08-19T01:33:23.202534+0000","last_active":"2024-08-19T01:35:00.804239+0000","last_peered":"2024-08-19T01:35:00.804239+0000","last_clean":"2024-08-19T01:35:00.804239+0000","last_became_active":"2024-08-19T01:33:23.202191+0000","last_became_peered":"2024-08-19T01:33:23.202191+0000","last_unstale":"2024-08-19T01:35:00.804239+0000","last_undegraded":"2024-08-19T01:35:00.804239+0000","last_fullsized":"2024-08-19T01:35:00.804239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:03.478905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797711+0000","last_change":"2024-08-19T01:33:23.201136+0000","last_active":"2024-08-19T01:35:00.797711+0000","last_peered":"2024-08-19T01:35:00.797711+0000","last_clean":"2024-08-19T01:35:00.797711+0000","last_became_active":"2024-08-19T01:33:23.200864+0000","last_became_peered":"2024-08-19T01:33:23.200864+0000","last_unstale":"2024-08-19T01:35:00.797711+0000","last_undegraded":"2024-08-19T01:35:00.797711+0000","last_fullsized":"2024-08-19T01:35:00.797711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:32:42.091880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883696+0000","last_change":"2024-08-19T01:33:21.172149+0000","last_active":"2024-08-19T01:34:01.883696+0000","last_peered":"2024-08-19T01:34:01.883696+0000","last_clean":"2024-08-19T01:34:01.883696+0000","last_became_active":"2024-08-19T01:33:21.171833+0000","last_became_peered":"2024-08-19T01:33:21.171833+0000","last_unstale":"2024-08-19T01:34:01.883696+0000","last_undegraded":"2024-08-19T01:34:01.883696+0000","last_fullsized":"2024-08-19T01:34:01.883696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:25:11.097212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793188+0000","last_change":"2024-08-19T01:33:23.193675+0000","last_active":"2024-08-19T01:35:00.793188+0000","last_peered":"2024-08-19T01:35:00.793188+0000","last_clean":"2024-08-19T01:35:00.793188+0000","last_became_active":"2024-08-19T01:33:23.188719+0000","last_became_peered":"2024-08-19T01:33:23.188719+0000","last_unstale":"2024-08-19T01:35:00.793188+0000","last_undegraded":"2024-08-19T01:35:00.793188+0000","last_fullsized":"2024-08-19T01:35:00.793188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:54:12.698211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883781+0000","last_change":"2024-08-19T01:33:21.172675+0000","last_active":"2024-08-19T01:34:01.883781+0000","last_peered":"2024-08-19T01:34:01.883781+0000","last_clean":"2024-08-19T01:34:01.883781+0000","last_became_active":"2024-08-19T01:33:21.172320+0000","last_became_peered":"2024-08-19T01:33:21.172320+0000","last_unstale":"2024-08-19T01:34:01.883781+0000","last_undegraded":"2024-08-19T01:34:01.883781+0000","last_fullsized":"2024-08-19T01:34:01.883781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:29:41.511220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804015+0000","last_change":"2024-08-19T01:33:23.203113+0000","last_active":"2024-08-19T01:35:00.804015+0000","last_peered":"2024-08-19T01:35:00.804015+0000","last_clean":"2024-08-19T01:35:00.804015+0000","last_became_active":"2024-08-19T01:33:23.202807+0000","last_became_peered":"2024-08-19T01:33:23.202807+0000","last_unstale":"2024-08-19T01:35:00.804015+0000","last_undegraded":"2024-08-19T01:35:00.804015+0000","last_fullsized":"2024-08-19T01:35:00.804015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:24:21.570929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878245+0000","last_change":"2024-08-19T01:33:21.287382+0000","last_active":"2024-08-19T01:34:01.878245+0000","last_peered":"2024-08-19T01:34:01.878245+0000","last_clean":"2024-08-19T01:34:01.878245+0000","last_became_active":"2024-08-19T01:33:21.287040+0000","last_became_peered":"2024-08-19T01:33:21.287040+0000","last_unstale":"2024-08-19T01:34:01.878245+0000","last_undegraded":"2024-08-19T01:34:01.878245+0000","last_fullsized":"2024-08-19T01:34:01.878245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:01:49.973969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792352+0000","last_change":"2024-08-19T01:33:23.197869+0000","last_active":"2024-08-19T01:35:00.792352+0000","last_peered":"2024-08-19T01:35:00.792352+0000","last_clean":"2024-08-19T01:35:00.792352+0000","last_became_active":"2024-08-19T01:33:23.197529+0000","last_became_peered":"2024-08-19T01:33:23.197529+0000","last_unstale":"2024-08-19T01:35:00.792352+0000","last_undegraded":"2024-08-19T01:35:00.792352+0000","last_fullsized":"2024-08-19T01:35:00.792352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:14.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'15","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.143334+0000","last_change":"2024-08-19T01:33:21.172180+0000","last_active":"2024-08-19T01:35:18.143334+0000","last_peered":"2024-08-19T01:35:18.143334+0000","last_clean":"2024-08-19T01:35:18.143334+0000","last_became_active":"2024-08-19T01:33:21.171895+0000","last_became_peered":"2024-08-19T01:33:21.171895+0000","last_unstale":"2024-08-19T01:35:18.143334+0000","last_undegraded":"2024-08-19T01:35:18.143334+0000","last_fullsized":"2024-08-19T01:35:18.143334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:51:38.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75654,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1113,"num_read_kb":635,"num_write":276,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":218334,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":454575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1111694,"data_compressed":37649,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":453901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":454167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1049475,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":454641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1047107,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":454502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":453443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":517511,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":517023,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":454957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64244,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64732,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":68430,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:35:23.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T01:35:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 -- 172.21.15.103:0/2325234262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064101070 msgr2=0x7f10641014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 --2- 172.21.15.103:0/2325234262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064101070 0x7f10641014f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f104c009900 tx=0x7f104c02a410 comp rx=0 tx=0).stop 2024-08-19T01:35:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 -- 172.21.15.103:0/2325234262 shutdown_connections 2024-08-19T01:35:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 --2- 172.21.15.103:0/2325234262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064101a30 0x7f1064108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 --2- 172.21.15.103:0/2325234262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064101070 0x7f10641014f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 --2- 172.21.15.103:0/2325234262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10640ffe90 0x7f1064100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 -- 172.21.15.103:0/2325234262 >> 172.21.15.103:0/2325234262 conn(0x7f10640fb640 msgr2=0x7f10640fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 -- 172.21.15.103:0/2325234262 shutdown_connections 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.670+0000 7f10699b2640 1 -- 172.21.15.103:0/2325234262 wait complete. 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 Processor -- start 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 -- start start 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10640ffe90 0x7f10640fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064101070 0x7f106419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1064101a30 0x7f106419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f106410ba30 con 0x7f10640ffe90 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f106410b8b0 con 0x7f1064101070 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10699b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f106410bbb0 con 0x7f1064101a30 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10640ffe90 0x7f10640fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10640ffe90 0x7f10640fd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51304/0 (socket says 172.21.15.103:51304) 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 -- 172.21.15.103:0/4282335203 learned_addr learned my addr 172.21.15.103:0/4282335203 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f10637fe640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1064101a30 0x7f106419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f105a7fc640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064101070 0x7f106419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 -- 172.21.15.103:0/4282335203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1064101a30 msgr2=0x7f106419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1064101a30 0x7f106419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 -- 172.21.15.103:0/4282335203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064101070 msgr2=0x7f106419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064101070 0x7f106419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.671+0000 7f1062ffd640 1 -- 172.21.15.103:0/4282335203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106419f720 con 0x7f10640ffe90 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.672+0000 7f105a7fc640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064101070 0x7f106419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.672+0000 7f1062ffd640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10640ffe90 0x7f10640fd830 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f105400c970 tx=0x7f105400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.672+0000 7f10689b0640 1 -- 172.21.15.103:0/4282335203 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054004220 con 0x7f10640ffe90 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.672+0000 7f10689b0640 1 -- 172.21.15.103:0/4282335203 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10540043c0 con 0x7f10640ffe90 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.672+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106419f9b0 con 0x7f10640ffe90 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.672+0000 7f10689b0640 1 -- 172.21.15.103:0/4282335203 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054004a00 con 0x7f10640ffe90 2024-08-19T01:35:23.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.672+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106419feb0 con 0x7f10640ffe90 2024-08-19T01:35:23.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.673+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10641a0140 con 0x7f10640ffe90 2024-08-19T01:35:23.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.674+0000 7f10689b0640 1 -- 172.21.15.103:0/4282335203 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f10540079b0 con 0x7f10640ffe90 2024-08-19T01:35:23.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.675+0000 7f10689b0640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10500a2b90 0x7f10500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:23.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.675+0000 7f105a7fc640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10500a2b90 0x7f10500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:23.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.675+0000 7f10689b0640 1 -- 172.21.15.103:0/4282335203 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f10540cf290 con 0x7f10640ffe90 2024-08-19T01:35:23.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.675+0000 7f105a7fc640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10500a2b90 0x7f10500a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f104c006010 tx=0x7f104c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:23.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.678+0000 7f10689b0640 1 -- 172.21.15.103:0/4282335203 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1054014030 con 0x7f10640ffe90 2024-08-19T01:35:23.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.842+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1064062360 con 0x7f10500a2b90 2024-08-19T01:35:23.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.848+0000 7f10689b0640 1 -- 172.21.15.103:0/4282335203 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259116 (secure 0 0 0) 0x7f1064062360 con 0x7f10500a2b90 2024-08-19T01:35:23.852 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:23.854 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T01:35:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10500a2b90 msgr2=0x7f10500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10500a2b90 0x7f10500a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f104c006010 tx=0x7f104c000bc0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10640ffe90 msgr2=0x7f10640fd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10640ffe90 0x7f10640fd830 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f105400c970 tx=0x7f105400ce30 comp rx=0 tx=0).stop 2024-08-19T01:35:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 shutdown_connections 2024-08-19T01:35:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10500a2b90 0x7f10500a5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1064101a30 0x7f106419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1064101070 0x7f106419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 --2- 172.21.15.103:0/4282335203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10640ffe90 0x7f10640fd830 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:23.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 >> 172.21.15.103:0/4282335203 conn(0x7f10640fb640 msgr2=0x7f1064104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:23.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 shutdown_connections 2024-08-19T01:35:23.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:23.851+0000 7f10699b2640 1 -- 172.21.15.103:0/4282335203 wait complete. 2024-08-19T01:35:23.863 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-08-19T01:35:22.800223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666256,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1239,"num_read_kb":744,"num_write":459,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364872,"kb_used_data":11560,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576952,"statfs":{"total":1151940427776,"available":1151566798848,"internally_reserved":0,"allocated":11837440,"data_stored":7422996,"data_compressed":112947,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6774,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002958"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804705+0000","last_change":"2024-08-19T01:33:23.202035+0000","last_active":"2024-08-19T01:35:00.804705+0000","last_peered":"2024-08-19T01:35:00.804705+0000","last_clean":"2024-08-19T01:35:00.804705+0000","last_became_active":"2024-08-19T01:33:23.201418+0000","last_became_peered":"2024-08-19T01:33:23.201418+0000","last_unstale":"2024-08-19T01:35:00.804705+0000","last_undegraded":"2024-08-19T01:35:00.804705+0000","last_fullsized":"2024-08-19T01:35:00.804705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:54:57.927424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802387+0000","last_change":"2024-08-19T01:33:23.202047+0000","last_active":"2024-08-19T01:35:00.802387+0000","last_peered":"2024-08-19T01:35:00.802387+0000","last_clean":"2024-08-19T01:35:00.802387+0000","last_became_active":"2024-08-19T01:33:23.201762+0000","last_became_peered":"2024-08-19T01:33:23.201762+0000","last_unstale":"2024-08-19T01:35:00.802387+0000","last_undegraded":"2024-08-19T01:35:00.802387+0000","last_fullsized":"2024-08-19T01:35:00.802387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:23:57.252284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804290+0000","last_change":"2024-08-19T01:33:23.202876+0000","last_active":"2024-08-19T01:35:00.804290+0000","last_peered":"2024-08-19T01:35:00.804290+0000","last_clean":"2024-08-19T01:35:00.804290+0000","last_became_active":"2024-08-19T01:33:23.202528+0000","last_became_peered":"2024-08-19T01:33:23.202528+0000","last_unstale":"2024-08-19T01:35:00.804290+0000","last_undegraded":"2024-08-19T01:35:00.804290+0000","last_fullsized":"2024-08-19T01:35:00.804290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:20.514390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790002+0000","last_change":"2024-08-19T01:33:23.199443+0000","last_active":"2024-08-19T01:35:00.790002+0000","last_peered":"2024-08-19T01:35:00.790002+0000","last_clean":"2024-08-19T01:35:00.790002+0000","last_became_active":"2024-08-19T01:33:23.198344+0000","last_became_peered":"2024-08-19T01:33:23.198344+0000","last_unstale":"2024-08-19T01:35:00.790002+0000","last_undegraded":"2024-08-19T01:35:00.790002+0000","last_fullsized":"2024-08-19T01:35:00.790002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:43:43.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800162+0000","last_change":"2024-08-19T01:33:23.191185+0000","last_active":"2024-08-19T01:35:00.800162+0000","last_peered":"2024-08-19T01:35:00.800162+0000","last_clean":"2024-08-19T01:35:00.800162+0000","last_became_active":"2024-08-19T01:33:23.190892+0000","last_became_peered":"2024-08-19T01:33:23.190892+0000","last_unstale":"2024-08-19T01:35:00.800162+0000","last_undegraded":"2024-08-19T01:35:00.800162+0000","last_fullsized":"2024-08-19T01:35:00.800162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:28.384609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.795199+0000","last_change":"2024-08-19T01:33:23.193036+0000","last_active":"2024-08-19T01:35:00.795199+0000","last_peered":"2024-08-19T01:35:00.795199+0000","last_clean":"2024-08-19T01:35:00.795199+0000","last_became_active":"2024-08-19T01:33:23.192686+0000","last_became_peered":"2024-08-19T01:33:23.192686+0000","last_unstale":"2024-08-19T01:35:00.795199+0000","last_undegraded":"2024-08-19T01:35:00.795199+0000","last_fullsized":"2024-08-19T01:35:00.795199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:46.235112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801472+0000","last_change":"2024-08-19T01:33:23.197636+0000","last_active":"2024-08-19T01:35:00.801472+0000","last_peered":"2024-08-19T01:35:00.801472+0000","last_clean":"2024-08-19T01:35:00.801472+0000","last_became_active":"2024-08-19T01:33:23.197323+0000","last_became_peered":"2024-08-19T01:33:23.197323+0000","last_unstale":"2024-08-19T01:35:00.801472+0000","last_undegraded":"2024-08-19T01:35:00.801472+0000","last_fullsized":"2024-08-19T01:35:00.801472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:46:36.168786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802002+0000","last_change":"2024-08-19T01:33:23.193717+0000","last_active":"2024-08-19T01:35:00.802002+0000","last_peered":"2024-08-19T01:35:00.802002+0000","last_clean":"2024-08-19T01:35:00.802002+0000","last_became_active":"2024-08-19T01:33:23.193376+0000","last_became_peered":"2024-08-19T01:33:23.193376+0000","last_unstale":"2024-08-19T01:35:00.802002+0000","last_undegraded":"2024-08-19T01:35:00.802002+0000","last_fullsized":"2024-08-19T01:35:00.802002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:16.290081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796833+0000","last_change":"2024-08-19T01:33:23.197956+0000","last_active":"2024-08-19T01:35:00.796833+0000","last_peered":"2024-08-19T01:35:00.796833+0000","last_clean":"2024-08-19T01:35:00.796833+0000","last_became_active":"2024-08-19T01:33:23.197667+0000","last_became_peered":"2024-08-19T01:33:23.197667+0000","last_unstale":"2024-08-19T01:35:00.796833+0000","last_undegraded":"2024-08-19T01:35:00.796833+0000","last_fullsized":"2024-08-19T01:35:00.796833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:16.841046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792010+0000","last_change":"2024-08-19T01:33:23.194569+0000","last_active":"2024-08-19T01:35:00.792010+0000","last_peered":"2024-08-19T01:35:00.792010+0000","last_clean":"2024-08-19T01:35:00.792010+0000","last_became_active":"2024-08-19T01:33:23.194223+0000","last_became_peered":"2024-08-19T01:33:23.194223+0000","last_unstale":"2024-08-19T01:35:00.792010+0000","last_undegraded":"2024-08-19T01:35:00.792010+0000","last_fullsized":"2024-08-19T01:35:00.792010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:14:30.184738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800081+0000","last_change":"2024-08-19T01:33:23.202396+0000","last_active":"2024-08-19T01:35:00.800081+0000","last_peered":"2024-08-19T01:35:00.800081+0000","last_clean":"2024-08-19T01:35:00.800081+0000","last_became_active":"2024-08-19T01:33:23.202063+0000","last_became_peered":"2024-08-19T01:33:23.202063+0000","last_unstale":"2024-08-19T01:35:00.800081+0000","last_undegraded":"2024-08-19T01:35:00.800081+0000","last_fullsized":"2024-08-19T01:35:00.800081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:57.513639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.789724+0000","last_change":"2024-08-19T01:33:23.200610+0000","last_active":"2024-08-19T01:35:00.789724+0000","last_peered":"2024-08-19T01:35:00.789724+0000","last_clean":"2024-08-19T01:35:00.789724+0000","last_became_active":"2024-08-19T01:33:23.200136+0000","last_became_peered":"2024-08-19T01:33:23.200136+0000","last_unstale":"2024-08-19T01:35:00.789724+0000","last_undegraded":"2024-08-19T01:35:00.789724+0000","last_fullsized":"2024-08-19T01:35:00.789724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:47:39.252702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792991+0000","last_change":"2024-08-19T01:33:23.196062+0000","last_active":"2024-08-19T01:35:00.792991+0000","last_peered":"2024-08-19T01:35:00.792991+0000","last_clean":"2024-08-19T01:35:00.792991+0000","last_became_active":"2024-08-19T01:33:23.195577+0000","last_became_peered":"2024-08-19T01:33:23.195577+0000","last_unstale":"2024-08-19T01:35:00.792991+0000","last_undegraded":"2024-08-19T01:35:00.792991+0000","last_fullsized":"2024-08-19T01:35:00.792991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:46:09.111323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804514+0000","last_change":"2024-08-19T01:33:23.201664+0000","last_active":"2024-08-19T01:35:00.804514+0000","last_peered":"2024-08-19T01:35:00.804514+0000","last_clean":"2024-08-19T01:35:00.804514+0000","last_became_active":"2024-08-19T01:33:23.200241+0000","last_became_peered":"2024-08-19T01:33:23.200241+0000","last_unstale":"2024-08-19T01:35:00.804514+0000","last_undegraded":"2024-08-19T01:35:00.804514+0000","last_fullsized":"2024-08-19T01:35:00.804514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:17.199776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803104+0000","last_change":"2024-08-19T01:33:23.198102+0000","last_active":"2024-08-19T01:35:00.803104+0000","last_peered":"2024-08-19T01:35:00.803104+0000","last_clean":"2024-08-19T01:35:00.803104+0000","last_became_active":"2024-08-19T01:33:23.197778+0000","last_became_peered":"2024-08-19T01:33:23.197778+0000","last_unstale":"2024-08-19T01:35:00.803104+0000","last_undegraded":"2024-08-19T01:35:00.803104+0000","last_fullsized":"2024-08-19T01:35:00.803104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:03.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804605+0000","last_change":"2024-08-19T01:33:23.194581+0000","last_active":"2024-08-19T01:35:00.804605+0000","last_peered":"2024-08-19T01:35:00.804605+0000","last_clean":"2024-08-19T01:35:00.804605+0000","last_became_active":"2024-08-19T01:33:23.194256+0000","last_became_peered":"2024-08-19T01:33:23.194256+0000","last_unstale":"2024-08-19T01:35:00.804605+0000","last_undegraded":"2024-08-19T01:35:00.804605+0000","last_fullsized":"2024-08-19T01:35:00.804605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:17:20.954190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804738+0000","last_change":"2024-08-19T01:33:23.201242+0000","last_active":"2024-08-19T01:35:00.804738+0000","last_peered":"2024-08-19T01:35:00.804738+0000","last_clean":"2024-08-19T01:35:00.804738+0000","last_became_active":"2024-08-19T01:33:23.200835+0000","last_became_peered":"2024-08-19T01:33:23.200835+0000","last_unstale":"2024-08-19T01:35:00.804738+0000","last_undegraded":"2024-08-19T01:35:00.804738+0000","last_fullsized":"2024-08-19T01:35:00.804738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:35:54.312266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.799770+0000","last_change":"2024-08-19T01:33:23.187728+0000","last_active":"2024-08-19T01:35:00.799770+0000","last_peered":"2024-08-19T01:35:00.799770+0000","last_clean":"2024-08-19T01:35:00.799770+0000","last_became_active":"2024-08-19T01:33:23.187352+0000","last_became_peered":"2024-08-19T01:33:23.187352+0000","last_unstale":"2024-08-19T01:35:00.799770+0000","last_undegraded":"2024-08-19T01:35:00.799770+0000","last_fullsized":"2024-08-19T01:35:00.799770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:06:03.041328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797919+0000","last_change":"2024-08-19T01:33:23.193146+0000","last_active":"2024-08-19T01:35:00.797919+0000","last_peered":"2024-08-19T01:35:00.797919+0000","last_clean":"2024-08-19T01:35:00.797919+0000","last_became_active":"2024-08-19T01:33:23.192810+0000","last_became_peered":"2024-08-19T01:33:23.192810+0000","last_unstale":"2024-08-19T01:35:00.797919+0000","last_undegraded":"2024-08-19T01:35:00.797919+0000","last_fullsized":"2024-08-19T01:35:00.797919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:12.044175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.791343+0000","last_change":"2024-08-19T01:33:23.194635+0000","last_active":"2024-08-19T01:35:00.791343+0000","last_peered":"2024-08-19T01:35:00.791343+0000","last_clean":"2024-08-19T01:35:00.791343+0000","last_became_active":"2024-08-19T01:33:23.194308+0000","last_became_peered":"2024-08-19T01:33:23.194308+0000","last_unstale":"2024-08-19T01:35:00.791343+0000","last_undegraded":"2024-08-19T01:35:00.791343+0000","last_fullsized":"2024-08-19T01:35:00.791343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:12.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802081+0000","last_change":"2024-08-19T01:33:23.193108+0000","last_active":"2024-08-19T01:35:00.802081+0000","last_peered":"2024-08-19T01:35:00.802081+0000","last_clean":"2024-08-19T01:35:00.802081+0000","last_became_active":"2024-08-19T01:33:23.192830+0000","last_became_peered":"2024-08-19T01:33:23.192830+0000","last_unstale":"2024-08-19T01:35:00.802081+0000","last_undegraded":"2024-08-19T01:35:00.802081+0000","last_fullsized":"2024-08-19T01:35:00.802081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:33.355498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803102+0000","last_change":"2024-08-19T01:33:23.195789+0000","last_active":"2024-08-19T01:35:00.803102+0000","last_peered":"2024-08-19T01:35:00.803102+0000","last_clean":"2024-08-19T01:35:00.803102+0000","last_became_active":"2024-08-19T01:33:23.195064+0000","last_became_peered":"2024-08-19T01:33:23.195064+0000","last_unstale":"2024-08-19T01:35:00.803102+0000","last_undegraded":"2024-08-19T01:35:00.803102+0000","last_fullsized":"2024-08-19T01:35:00.803102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:05:33.848953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802554+0000","last_change":"2024-08-19T01:33:23.200968+0000","last_active":"2024-08-19T01:35:00.802554+0000","last_peered":"2024-08-19T01:35:00.802554+0000","last_clean":"2024-08-19T01:35:00.802554+0000","last_became_active":"2024-08-19T01:33:23.200606+0000","last_became_peered":"2024-08-19T01:33:23.200606+0000","last_unstale":"2024-08-19T01:35:00.802554+0000","last_undegraded":"2024-08-19T01:35:00.802554+0000","last_fullsized":"2024-08-19T01:35:00.802554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:44.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.798216+0000","last_change":"2024-08-19T01:33:23.191204+0000","last_active":"2024-08-19T01:35:00.798216+0000","last_peered":"2024-08-19T01:35:00.798216+0000","last_clean":"2024-08-19T01:35:00.798216+0000","last_became_active":"2024-08-19T01:33:23.190709+0000","last_became_peered":"2024-08-19T01:33:23.190709+0000","last_unstale":"2024-08-19T01:35:00.798216+0000","last_undegraded":"2024-08-19T01:35:00.798216+0000","last_fullsized":"2024-08-19T01:35:00.798216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:56:10.988967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793707+0000","last_change":"2024-08-19T01:34:01.885389+0000","last_active":"2024-08-19T01:35:00.793707+0000","last_peered":"2024-08-19T01:35:00.793707+0000","last_clean":"2024-08-19T01:35:00.793707+0000","last_became_active":"2024-08-19T01:34:01.885100+0000","last_became_peered":"2024-08-19T01:34:01.885100+0000","last_unstale":"2024-08-19T01:35:00.793707+0000","last_undegraded":"2024-08-19T01:35:00.793707+0000","last_fullsized":"2024-08-19T01:35:00.793707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:21.616876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.782764+0000","last_change":"2024-08-19T01:33:23.187309+0000","last_active":"2024-08-19T01:35:00.782764+0000","last_peered":"2024-08-19T01:35:00.782764+0000","last_clean":"2024-08-19T01:35:00.782764+0000","last_became_active":"2024-08-19T01:33:23.186976+0000","last_became_peered":"2024-08-19T01:33:23.186976+0000","last_unstale":"2024-08-19T01:35:00.782764+0000","last_undegraded":"2024-08-19T01:35:00.782764+0000","last_fullsized":"2024-08-19T01:35:00.782764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:46:02.852000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790722+0000","last_change":"2024-08-19T01:33:23.190292+0000","last_active":"2024-08-19T01:35:00.790722+0000","last_peered":"2024-08-19T01:35:00.790722+0000","last_clean":"2024-08-19T01:35:00.790722+0000","last_became_active":"2024-08-19T01:33:23.189969+0000","last_became_peered":"2024-08-19T01:33:23.189969+0000","last_unstale":"2024-08-19T01:35:00.790722+0000","last_undegraded":"2024-08-19T01:35:00.790722+0000","last_fullsized":"2024-08-19T01:35:00.790722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:35:51.388150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796348+0000","last_change":"2024-08-19T01:33:23.192148+0000","last_active":"2024-08-19T01:35:00.796348+0000","last_peered":"2024-08-19T01:35:00.796348+0000","last_clean":"2024-08-19T01:35:00.796348+0000","last_became_active":"2024-08-19T01:33:23.191845+0000","last_became_peered":"2024-08-19T01:33:23.191845+0000","last_unstale":"2024-08-19T01:35:00.796348+0000","last_undegraded":"2024-08-19T01:35:00.796348+0000","last_fullsized":"2024-08-19T01:35:00.796348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:51.308393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803032+0000","last_change":"2024-08-19T01:33:23.195033+0000","last_active":"2024-08-19T01:35:00.803032+0000","last_peered":"2024-08-19T01:35:00.803032+0000","last_clean":"2024-08-19T01:35:00.803032+0000","last_became_active":"2024-08-19T01:33:23.194745+0000","last_became_peered":"2024-08-19T01:33:23.194745+0000","last_unstale":"2024-08-19T01:35:00.803032+0000","last_undegraded":"2024-08-19T01:35:00.803032+0000","last_fullsized":"2024-08-19T01:35:00.803032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:33:40.748311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797892+0000","last_change":"2024-08-19T01:33:23.195817+0000","last_active":"2024-08-19T01:35:00.797892+0000","last_peered":"2024-08-19T01:35:00.797892+0000","last_clean":"2024-08-19T01:35:00.797892+0000","last_became_active":"2024-08-19T01:33:23.194688+0000","last_became_peered":"2024-08-19T01:33:23.194688+0000","last_unstale":"2024-08-19T01:35:00.797892+0000","last_undegraded":"2024-08-19T01:35:00.797892+0000","last_fullsized":"2024-08-19T01:35:00.797892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:13:04.913921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793529+0000","last_change":"2024-08-19T01:33:23.187604+0000","last_active":"2024-08-19T01:35:00.793529+0000","last_peered":"2024-08-19T01:35:00.793529+0000","last_clean":"2024-08-19T01:35:00.793529+0000","last_became_active":"2024-08-19T01:33:23.187242+0000","last_became_peered":"2024-08-19T01:33:23.187242+0000","last_unstale":"2024-08-19T01:35:00.793529+0000","last_undegraded":"2024-08-19T01:35:00.793529+0000","last_fullsized":"2024-08-19T01:35:00.793529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:28.592525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792374+0000","last_change":"2024-08-19T01:33:23.191737+0000","last_active":"2024-08-19T01:35:00.792374+0000","last_peered":"2024-08-19T01:35:00.792374+0000","last_clean":"2024-08-19T01:35:00.792374+0000","last_became_active":"2024-08-19T01:33:23.191358+0000","last_became_peered":"2024-08-19T01:33:23.191358+0000","last_unstale":"2024-08-19T01:35:00.792374+0000","last_undegraded":"2024-08-19T01:35:00.792374+0000","last_fullsized":"2024-08-19T01:35:00.792374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:28:14.694157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.785064+0000","last_change":"2024-08-19T01:33:23.193110+0000","last_active":"2024-08-19T01:35:00.785064+0000","last_peered":"2024-08-19T01:35:00.785064+0000","last_clean":"2024-08-19T01:35:00.785064+0000","last_became_active":"2024-08-19T01:33:23.192789+0000","last_became_peered":"2024-08-19T01:33:23.192789+0000","last_unstale":"2024-08-19T01:35:00.785064+0000","last_undegraded":"2024-08-19T01:35:00.785064+0000","last_fullsized":"2024-08-19T01:35:00.785064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:45:48.141403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:20.124936+0000","last_change":"2024-08-19T01:33:21.173518+0000","last_active":"2024-08-19T01:35:20.124936+0000","last_peered":"2024-08-19T01:35:20.124936+0000","last_clean":"2024-08-19T01:35:20.124936+0000","last_became_active":"2024-08-19T01:33:21.172288+0000","last_became_peered":"2024-08-19T01:33:21.172288+0000","last_unstale":"2024-08-19T01:35:20.124936+0000","last_undegraded":"2024-08-19T01:35:20.124936+0000","last_fullsized":"2024-08-19T01:35:20.124936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:59:42.864724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1645,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801261+0000","last_change":"2024-08-19T01:33:23.190160+0000","last_active":"2024-08-19T01:35:00.801261+0000","last_peered":"2024-08-19T01:35:00.801261+0000","last_clean":"2024-08-19T01:35:00.801261+0000","last_became_active":"2024-08-19T01:33:23.189802+0000","last_became_peered":"2024-08-19T01:33:23.189802+0000","last_unstale":"2024-08-19T01:35:00.801261+0000","last_undegraded":"2024-08-19T01:35:00.801261+0000","last_fullsized":"2024-08-19T01:35:00.801261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:53:10.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880543+0000","last_change":"2024-08-19T01:33:21.167443+0000","last_active":"2024-08-19T01:34:01.880543+0000","last_peered":"2024-08-19T01:34:01.880543+0000","last_clean":"2024-08-19T01:34:01.880543+0000","last_became_active":"2024-08-19T01:33:21.167054+0000","last_became_peered":"2024-08-19T01:33:21.167054+0000","last_unstale":"2024-08-19T01:34:01.880543+0000","last_undegraded":"2024-08-19T01:34:01.880543+0000","last_fullsized":"2024-08-19T01:34:01.880543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:00:38.358807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928860+0000","last_change":"2024-08-19T01:33:21.163097+0000","last_active":"2024-08-19T01:34:01.928860+0000","last_peered":"2024-08-19T01:34:01.928860+0000","last_clean":"2024-08-19T01:34:01.928860+0000","last_became_active":"2024-08-19T01:33:21.162802+0000","last_became_peered":"2024-08-19T01:33:21.162802+0000","last_unstale":"2024-08-19T01:34:01.928860+0000","last_undegraded":"2024-08-19T01:34:01.928860+0000","last_fullsized":"2024-08-19T01:34:01.928860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:15:41.244104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803050+0000","last_change":"2024-08-19T01:33:23.200975+0000","last_active":"2024-08-19T01:35:00.803050+0000","last_peered":"2024-08-19T01:35:00.803050+0000","last_clean":"2024-08-19T01:35:00.803050+0000","last_became_active":"2024-08-19T01:33:23.200667+0000","last_became_peered":"2024-08-19T01:33:23.200667+0000","last_unstale":"2024-08-19T01:35:00.803050+0000","last_undegraded":"2024-08-19T01:35:00.803050+0000","last_fullsized":"2024-08-19T01:35:00.803050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:35.105521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879336+0000","last_change":"2024-08-19T01:33:21.171252+0000","last_active":"2024-08-19T01:34:01.879336+0000","last_peered":"2024-08-19T01:34:01.879336+0000","last_clean":"2024-08-19T01:34:01.879336+0000","last_became_active":"2024-08-19T01:33:21.170944+0000","last_became_peered":"2024-08-19T01:33:21.170944+0000","last_unstale":"2024-08-19T01:34:01.879336+0000","last_undegraded":"2024-08-19T01:34:01.879336+0000","last_fullsized":"2024-08-19T01:34:01.879336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:10:15.869452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801852+0000","last_change":"2024-08-19T01:33:23.191931+0000","last_active":"2024-08-19T01:35:00.801852+0000","last_peered":"2024-08-19T01:35:00.801852+0000","last_clean":"2024-08-19T01:35:00.801852+0000","last_became_active":"2024-08-19T01:33:23.191601+0000","last_became_peered":"2024-08-19T01:33:23.191601+0000","last_unstale":"2024-08-19T01:35:00.801852+0000","last_undegraded":"2024-08-19T01:35:00.801852+0000","last_fullsized":"2024-08-19T01:35:00.801852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:52.874546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":243,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.125745+0000","last_change":"2024-08-19T01:33:21.170714+0000","last_active":"2024-08-19T01:35:21.125745+0000","last_peered":"2024-08-19T01:35:21.125745+0000","last_clean":"2024-08-19T01:35:21.125745+0000","last_became_active":"2024-08-19T01:33:21.170379+0000","last_became_peered":"2024-08-19T01:33:21.170379+0000","last_unstale":"2024-08-19T01:35:21.125745+0000","last_undegraded":"2024-08-19T01:35:21.125745+0000","last_fullsized":"2024-08-19T01:35:21.125745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:27:27.417634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.791818+0000","last_change":"2024-08-19T01:33:23.189680+0000","last_active":"2024-08-19T01:35:00.791818+0000","last_peered":"2024-08-19T01:35:00.791818+0000","last_clean":"2024-08-19T01:35:00.791818+0000","last_became_active":"2024-08-19T01:33:23.189029+0000","last_became_peered":"2024-08-19T01:33:23.189029+0000","last_unstale":"2024-08-19T01:35:00.791818+0000","last_undegraded":"2024-08-19T01:35:00.791818+0000","last_fullsized":"2024-08-19T01:35:00.791818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:39.667675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881530+0000","last_change":"2024-08-19T01:33:21.169233+0000","last_active":"2024-08-19T01:34:01.881530+0000","last_peered":"2024-08-19T01:34:01.881530+0000","last_clean":"2024-08-19T01:34:01.881530+0000","last_became_active":"2024-08-19T01:33:21.168761+0000","last_became_peered":"2024-08-19T01:33:21.168761+0000","last_unstale":"2024-08-19T01:34:01.881530+0000","last_undegraded":"2024-08-19T01:34:01.881530+0000","last_fullsized":"2024-08-19T01:34:01.881530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:31:56.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804526+0000","last_change":"2024-08-19T01:33:23.194250+0000","last_active":"2024-08-19T01:35:00.804526+0000","last_peered":"2024-08-19T01:35:00.804526+0000","last_clean":"2024-08-19T01:35:00.804526+0000","last_became_active":"2024-08-19T01:33:23.193939+0000","last_became_peered":"2024-08-19T01:33:23.193939+0000","last_unstale":"2024-08-19T01:35:00.804526+0000","last_undegraded":"2024-08-19T01:35:00.804526+0000","last_fullsized":"2024-08-19T01:35:00.804526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:19:09.703701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879370+0000","last_change":"2024-08-19T01:33:21.287684+0000","last_active":"2024-08-19T01:34:01.879370+0000","last_peered":"2024-08-19T01:34:01.879370+0000","last_clean":"2024-08-19T01:34:01.879370+0000","last_became_active":"2024-08-19T01:33:21.287252+0000","last_became_peered":"2024-08-19T01:33:21.287252+0000","last_unstale":"2024-08-19T01:34:01.879370+0000","last_undegraded":"2024-08-19T01:34:01.879370+0000","last_fullsized":"2024-08-19T01:34:01.879370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:08.801345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.788353+0000","last_change":"2024-08-19T01:33:23.195077+0000","last_active":"2024-08-19T01:35:00.788353+0000","last_peered":"2024-08-19T01:35:00.788353+0000","last_clean":"2024-08-19T01:35:00.788353+0000","last_became_active":"2024-08-19T01:33:23.194753+0000","last_became_peered":"2024-08-19T01:33:23.194753+0000","last_unstale":"2024-08-19T01:35:00.788353+0000","last_undegraded":"2024-08-19T01:35:00.788353+0000","last_fullsized":"2024-08-19T01:35:00.788353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:12:09.186261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":282,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.125848+0000","last_change":"2024-08-19T01:33:21.168845+0000","last_active":"2024-08-19T01:35:21.125848+0000","last_peered":"2024-08-19T01:35:21.125848+0000","last_clean":"2024-08-19T01:35:21.125848+0000","last_became_active":"2024-08-19T01:33:21.168479+0000","last_became_peered":"2024-08-19T01:33:21.168479+0000","last_unstale":"2024-08-19T01:35:21.125848+0000","last_undegraded":"2024-08-19T01:35:21.125848+0000","last_fullsized":"2024-08-19T01:35:21.125848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:59:50.296329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65671,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":224,"num_read_kb":171,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.785907+0000","last_change":"2024-08-19T01:33:23.192252+0000","last_active":"2024-08-19T01:35:00.785907+0000","last_peered":"2024-08-19T01:35:00.785907+0000","last_clean":"2024-08-19T01:35:00.785907+0000","last_became_active":"2024-08-19T01:33:23.191934+0000","last_became_peered":"2024-08-19T01:33:23.191934+0000","last_unstale":"2024-08-19T01:35:00.785907+0000","last_undegraded":"2024-08-19T01:35:00.785907+0000","last_fullsized":"2024-08-19T01:35:00.785907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:31.134825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882306+0000","last_change":"2024-08-19T01:33:21.168541+0000","last_active":"2024-08-19T01:34:01.882306+0000","last_peered":"2024-08-19T01:34:01.882306+0000","last_clean":"2024-08-19T01:34:01.882306+0000","last_became_active":"2024-08-19T01:33:21.168205+0000","last_became_peered":"2024-08-19T01:33:21.168205+0000","last_unstale":"2024-08-19T01:34:01.882306+0000","last_undegraded":"2024-08-19T01:34:01.882306+0000","last_fullsized":"2024-08-19T01:34:01.882306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:19:52.078947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.799749+0000","last_change":"2024-08-19T01:33:23.193013+0000","last_active":"2024-08-19T01:35:00.799749+0000","last_peered":"2024-08-19T01:35:00.799749+0000","last_clean":"2024-08-19T01:35:00.799749+0000","last_became_active":"2024-08-19T01:33:23.192728+0000","last_became_peered":"2024-08-19T01:33:23.192728+0000","last_unstale":"2024-08-19T01:35:00.799749+0000","last_undegraded":"2024-08-19T01:35:00.799749+0000","last_fullsized":"2024-08-19T01:35:00.799749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:23:17.587527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879101+0000","last_change":"2024-08-19T01:33:21.170118+0000","last_active":"2024-08-19T01:34:01.879101+0000","last_peered":"2024-08-19T01:34:01.879101+0000","last_clean":"2024-08-19T01:34:01.879101+0000","last_became_active":"2024-08-19T01:33:21.169757+0000","last_became_peered":"2024-08-19T01:33:21.169757+0000","last_unstale":"2024-08-19T01:34:01.879101+0000","last_undegraded":"2024-08-19T01:34:01.879101+0000","last_fullsized":"2024-08-19T01:34:01.879101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:09:26.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804979+0000","last_change":"2024-08-19T01:34:01.929944+0000","last_active":"2024-08-19T01:35:00.804979+0000","last_peered":"2024-08-19T01:35:00.804979+0000","last_clean":"2024-08-19T01:35:00.804979+0000","last_became_active":"2024-08-19T01:34:01.929679+0000","last_became_peered":"2024-08-19T01:34:01.929679+0000","last_unstale":"2024-08-19T01:35:00.804979+0000","last_undegraded":"2024-08-19T01:35:00.804979+0000","last_fullsized":"2024-08-19T01:35:00.804979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:25:24.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.128678+0000","last_change":"2024-08-19T01:33:21.165204+0000","last_active":"2024-08-19T01:35:18.128678+0000","last_peered":"2024-08-19T01:35:18.128678+0000","last_clean":"2024-08-19T01:35:18.128678+0000","last_became_active":"2024-08-19T01:33:21.164826+0000","last_became_peered":"2024-08-19T01:33:21.164826+0000","last_unstale":"2024-08-19T01:35:18.128678+0000","last_undegraded":"2024-08-19T01:35:18.128678+0000","last_fullsized":"2024-08-19T01:35:18.128678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:45:23.212064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800773+0000","last_change":"2024-08-19T01:33:23.198176+0000","last_active":"2024-08-19T01:35:00.800773+0000","last_peered":"2024-08-19T01:35:00.800773+0000","last_clean":"2024-08-19T01:35:00.800773+0000","last_became_active":"2024-08-19T01:33:23.196187+0000","last_became_peered":"2024-08-19T01:33:23.196187+0000","last_unstale":"2024-08-19T01:35:00.800773+0000","last_undegraded":"2024-08-19T01:35:00.800773+0000","last_fullsized":"2024-08-19T01:35:00.800773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:53:05.943467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.079412+0000","last_change":"2024-08-19T01:34:02.079028+0000","last_active":"2024-08-19T01:34:02.079412+0000","last_peered":"2024-08-19T01:34:02.079412+0000","last_clean":"2024-08-19T01:34:02.079412+0000","last_became_active":"2024-08-19T01:34:02.049890+0000","last_became_peered":"2024-08-19T01:34:02.049890+0000","last_unstale":"2024-08-19T01:34:02.079412+0000","last_undegraded":"2024-08-19T01:34:02.079412+0000","last_fullsized":"2024-08-19T01:34:02.079412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:58.595161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.789928+0000","last_change":"2024-08-19T01:33:23.201104+0000","last_active":"2024-08-19T01:35:00.789928+0000","last_peered":"2024-08-19T01:35:00.789928+0000","last_clean":"2024-08-19T01:35:00.789928+0000","last_became_active":"2024-08-19T01:33:23.200809+0000","last_became_peered":"2024-08-19T01:33:23.200809+0000","last_unstale":"2024-08-19T01:35:00.789928+0000","last_undegraded":"2024-08-19T01:35:00.789928+0000","last_fullsized":"2024-08-19T01:35:00.789928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:54.901412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.884029+0000","last_change":"2024-08-19T01:33:21.172423+0000","last_active":"2024-08-19T01:34:01.884029+0000","last_peered":"2024-08-19T01:34:01.884029+0000","last_clean":"2024-08-19T01:34:01.884029+0000","last_became_active":"2024-08-19T01:33:21.171950+0000","last_became_peered":"2024-08-19T01:33:21.171950+0000","last_unstale":"2024-08-19T01:34:01.884029+0000","last_undegraded":"2024-08-19T01:34:01.884029+0000","last_fullsized":"2024-08-19T01:34:01.884029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:44:11.446875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790970+0000","last_change":"2024-08-19T01:33:23.192933+0000","last_active":"2024-08-19T01:35:00.790970+0000","last_peered":"2024-08-19T01:35:00.790970+0000","last_clean":"2024-08-19T01:35:00.790970+0000","last_became_active":"2024-08-19T01:33:23.192601+0000","last_became_peered":"2024-08-19T01:33:23.192601+0000","last_unstale":"2024-08-19T01:35:00.790970+0000","last_undegraded":"2024-08-19T01:35:00.790970+0000","last_fullsized":"2024-08-19T01:35:00.790970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:09.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929233+0000","last_change":"2024-08-19T01:33:21.168225+0000","last_active":"2024-08-19T01:34:01.929233+0000","last_peered":"2024-08-19T01:34:01.929233+0000","last_clean":"2024-08-19T01:34:01.929233+0000","last_became_active":"2024-08-19T01:33:21.167881+0000","last_became_peered":"2024-08-19T01:33:21.167881+0000","last_unstale":"2024-08-19T01:34:01.929233+0000","last_undegraded":"2024-08-19T01:34:01.929233+0000","last_fullsized":"2024-08-19T01:34:01.929233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:53:38.037036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793586+0000","last_change":"2024-08-19T01:33:23.194079+0000","last_active":"2024-08-19T01:35:00.793586+0000","last_peered":"2024-08-19T01:35:00.793586+0000","last_clean":"2024-08-19T01:35:00.793586+0000","last_became_active":"2024-08-19T01:33:23.193749+0000","last_became_peered":"2024-08-19T01:33:23.193749+0000","last_unstale":"2024-08-19T01:35:00.793586+0000","last_undegraded":"2024-08-19T01:35:00.793586+0000","last_fullsized":"2024-08-19T01:35:00.793586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:53:41.102163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879491+0000","last_change":"2024-08-19T01:33:21.168916+0000","last_active":"2024-08-19T01:34:01.879491+0000","last_peered":"2024-08-19T01:34:01.879491+0000","last_clean":"2024-08-19T01:34:01.879491+0000","last_became_active":"2024-08-19T01:33:21.168618+0000","last_became_peered":"2024-08-19T01:33:21.168618+0000","last_unstale":"2024-08-19T01:34:01.879491+0000","last_undegraded":"2024-08-19T01:34:01.879491+0000","last_fullsized":"2024-08-19T01:34:01.879491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:45:11.567779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.798985+0000","last_change":"2024-08-19T01:33:23.195546+0000","last_active":"2024-08-19T01:35:00.798985+0000","last_peered":"2024-08-19T01:35:00.798985+0000","last_clean":"2024-08-19T01:35:00.798985+0000","last_became_active":"2024-08-19T01:33:23.195221+0000","last_became_peered":"2024-08-19T01:33:23.195221+0000","last_unstale":"2024-08-19T01:35:00.798985+0000","last_undegraded":"2024-08-19T01:35:00.798985+0000","last_fullsized":"2024-08-19T01:35:00.798985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:46:10.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.123644+0000","last_change":"2024-08-19T01:33:21.287946+0000","last_active":"2024-08-19T01:35:18.123644+0000","last_peered":"2024-08-19T01:35:18.123644+0000","last_clean":"2024-08-19T01:35:18.123644+0000","last_became_active":"2024-08-19T01:33:21.287594+0000","last_became_peered":"2024-08-19T01:33:21.287594+0000","last_unstale":"2024-08-19T01:35:18.123644+0000","last_undegraded":"2024-08-19T01:35:18.123644+0000","last_fullsized":"2024-08-19T01:35:18.123644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:10:06.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1662,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800058+0000","last_change":"2024-08-19T01:33:23.200914+0000","last_active":"2024-08-19T01:35:00.800058+0000","last_peered":"2024-08-19T01:35:00.800058+0000","last_clean":"2024-08-19T01:35:00.800058+0000","last_became_active":"2024-08-19T01:33:23.200611+0000","last_became_peered":"2024-08-19T01:33:23.200611+0000","last_unstale":"2024-08-19T01:35:00.800058+0000","last_undegraded":"2024-08-19T01:35:00.800058+0000","last_fullsized":"2024-08-19T01:35:00.800058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:22:00.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880899+0000","last_change":"2024-08-19T01:32:09.926979+0000","last_active":"2024-08-19T01:34:01.880899+0000","last_peered":"2024-08-19T01:34:01.880899+0000","last_clean":"2024-08-19T01:34:01.880899+0000","last_became_active":"2024-08-19T01:32:09.926560+0000","last_became_peered":"2024-08-19T01:32:09.926560+0000","last_unstale":"2024-08-19T01:34:01.880899+0000","last_undegraded":"2024-08-19T01:34:01.880899+0000","last_fullsized":"2024-08-19T01:34:01.880899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_clean_scrub_stamp":"2024-08-19T01:29:05.356703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:12:23.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878743+0000","last_change":"2024-08-19T01:33:21.165099+0000","last_active":"2024-08-19T01:34:01.878743+0000","last_peered":"2024-08-19T01:34:01.878743+0000","last_clean":"2024-08-19T01:34:01.878743+0000","last_became_active":"2024-08-19T01:33:21.164756+0000","last_became_peered":"2024-08-19T01:33:21.164756+0000","last_unstale":"2024-08-19T01:34:01.878743+0000","last_undegraded":"2024-08-19T01:34:01.878743+0000","last_fullsized":"2024-08-19T01:34:01.878743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:11:54.480263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793548+0000","last_change":"2024-08-19T01:33:23.200185+0000","last_active":"2024-08-19T01:35:00.793548+0000","last_peered":"2024-08-19T01:35:00.793548+0000","last_clean":"2024-08-19T01:35:00.793548+0000","last_became_active":"2024-08-19T01:33:23.199885+0000","last_became_peered":"2024-08-19T01:33:23.199885+0000","last_unstale":"2024-08-19T01:35:00.793548+0000","last_undegraded":"2024-08-19T01:35:00.793548+0000","last_fullsized":"2024-08-19T01:35:00.793548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:17.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880804+0000","last_change":"2024-08-19T01:33:21.172587+0000","last_active":"2024-08-19T01:34:01.880804+0000","last_peered":"2024-08-19T01:34:01.880804+0000","last_clean":"2024-08-19T01:34:01.880804+0000","last_became_active":"2024-08-19T01:33:21.172233+0000","last_became_peered":"2024-08-19T01:33:21.172233+0000","last_unstale":"2024-08-19T01:34:01.880804+0000","last_undegraded":"2024-08-19T01:34:01.880804+0000","last_fullsized":"2024-08-19T01:34:01.880804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:26.690403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.795949+0000","last_change":"2024-08-19T01:33:23.197272+0000","last_active":"2024-08-19T01:35:00.795949+0000","last_peered":"2024-08-19T01:35:00.795949+0000","last_clean":"2024-08-19T01:35:00.795949+0000","last_became_active":"2024-08-19T01:33:23.196869+0000","last_became_peered":"2024-08-19T01:33:23.196869+0000","last_unstale":"2024-08-19T01:35:00.795949+0000","last_undegraded":"2024-08-19T01:35:00.795949+0000","last_fullsized":"2024-08-19T01:35:00.795949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:30.020184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:19.129363+0000","last_change":"2024-08-19T01:33:21.287366+0000","last_active":"2024-08-19T01:35:19.129363+0000","last_peered":"2024-08-19T01:35:19.129363+0000","last_clean":"2024-08-19T01:35:19.129363+0000","last_became_active":"2024-08-19T01:33:21.287044+0000","last_became_peered":"2024-08-19T01:33:21.287044+0000","last_unstale":"2024-08-19T01:35:19.129363+0000","last_undegraded":"2024-08-19T01:35:19.129363+0000","last_fullsized":"2024-08-19T01:35:19.129363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:19:04.194284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792226+0000","last_change":"2024-08-19T01:34:01.885001+0000","last_active":"2024-08-19T01:35:00.792226+0000","last_peered":"2024-08-19T01:35:00.792226+0000","last_clean":"2024-08-19T01:35:00.792226+0000","last_became_active":"2024-08-19T01:34:01.884680+0000","last_became_peered":"2024-08-19T01:34:01.884680+0000","last_unstale":"2024-08-19T01:35:00.792226+0000","last_undegraded":"2024-08-19T01:35:00.792226+0000","last_fullsized":"2024-08-19T01:35:00.792226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:08.251298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"86'8","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.148454+0000","last_change":"2024-08-19T01:33:21.164639+0000","last_active":"2024-08-19T01:35:21.148454+0000","last_peered":"2024-08-19T01:35:21.148454+0000","last_clean":"2024-08-19T01:35:21.148454+0000","last_became_active":"2024-08-19T01:33:21.164330+0000","last_became_peered":"2024-08-19T01:33:21.164330+0000","last_unstale":"2024-08-19T01:35:21.148454+0000","last_undegraded":"2024-08-19T01:35:21.148454+0000","last_fullsized":"2024-08-19T01:35:21.148454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:26:04.035130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801157+0000","last_change":"2024-08-19T01:34:01.883666+0000","last_active":"2024-08-19T01:35:00.801157+0000","last_peered":"2024-08-19T01:35:00.801157+0000","last_clean":"2024-08-19T01:35:00.801157+0000","last_became_active":"2024-08-19T01:34:01.883264+0000","last_became_peered":"2024-08-19T01:34:01.883264+0000","last_unstale":"2024-08-19T01:35:00.801157+0000","last_undegraded":"2024-08-19T01:35:00.801157+0000","last_fullsized":"2024-08-19T01:35:00.801157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:41:52.266751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879931+0000","last_change":"2024-08-19T01:33:21.287906+0000","last_active":"2024-08-19T01:34:01.879931+0000","last_peered":"2024-08-19T01:34:01.879931+0000","last_clean":"2024-08-19T01:34:01.879931+0000","last_became_active":"2024-08-19T01:33:21.287597+0000","last_became_peered":"2024-08-19T01:33:21.287597+0000","last_unstale":"2024-08-19T01:34:01.879931+0000","last_undegraded":"2024-08-19T01:34:01.879931+0000","last_fullsized":"2024-08-19T01:34:01.879931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:30.605948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801131+0000","last_change":"2024-08-19T01:33:23.191026+0000","last_active":"2024-08-19T01:35:00.801131+0000","last_peered":"2024-08-19T01:35:00.801131+0000","last_clean":"2024-08-19T01:35:00.801131+0000","last_became_active":"2024-08-19T01:33:23.190382+0000","last_became_peered":"2024-08-19T01:33:23.190382+0000","last_unstale":"2024-08-19T01:35:00.801131+0000","last_undegraded":"2024-08-19T01:35:00.801131+0000","last_fullsized":"2024-08-19T01:35:00.801131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:33:45.065441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883898+0000","last_change":"2024-08-19T01:33:21.288008+0000","last_active":"2024-08-19T01:34:01.883898+0000","last_peered":"2024-08-19T01:34:01.883898+0000","last_clean":"2024-08-19T01:34:01.883898+0000","last_became_active":"2024-08-19T01:33:21.287613+0000","last_became_peered":"2024-08-19T01:33:21.287613+0000","last_unstale":"2024-08-19T01:34:01.883898+0000","last_undegraded":"2024-08-19T01:34:01.883898+0000","last_fullsized":"2024-08-19T01:34:01.883898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:35:18.169564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796936+0000","last_change":"2024-08-19T01:33:23.197201+0000","last_active":"2024-08-19T01:35:00.796936+0000","last_peered":"2024-08-19T01:35:00.796936+0000","last_clean":"2024-08-19T01:35:00.796936+0000","last_became_active":"2024-08-19T01:33:23.196946+0000","last_became_peered":"2024-08-19T01:33:23.196946+0000","last_unstale":"2024-08-19T01:35:00.796936+0000","last_undegraded":"2024-08-19T01:35:00.796936+0000","last_fullsized":"2024-08-19T01:35:00.796936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:14:31.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'18","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.195232+0000","last_change":"2024-08-19T01:33:21.167109+0000","last_active":"2024-08-19T01:35:18.195232+0000","last_peered":"2024-08-19T01:35:18.195232+0000","last_clean":"2024-08-19T01:35:18.195232+0000","last_became_active":"2024-08-19T01:33:21.166788+0000","last_became_peered":"2024-08-19T01:33:21.166788+0000","last_unstale":"2024-08-19T01:35:18.195232+0000","last_undegraded":"2024-08-19T01:35:18.195232+0000","last_fullsized":"2024-08-19T01:35:18.195232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:44.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803153+0000","last_change":"2024-08-19T01:33:23.199699+0000","last_active":"2024-08-19T01:35:00.803153+0000","last_peered":"2024-08-19T01:35:00.803153+0000","last_clean":"2024-08-19T01:35:00.803153+0000","last_became_active":"2024-08-19T01:33:23.199417+0000","last_became_peered":"2024-08-19T01:33:23.199417+0000","last_unstale":"2024-08-19T01:35:00.803153+0000","last_undegraded":"2024-08-19T01:35:00.803153+0000","last_fullsized":"2024-08-19T01:35:00.803153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:36.519135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878888+0000","last_change":"2024-08-19T01:33:21.164177+0000","last_active":"2024-08-19T01:34:01.878888+0000","last_peered":"2024-08-19T01:34:01.878888+0000","last_clean":"2024-08-19T01:34:01.878888+0000","last_became_active":"2024-08-19T01:33:21.163832+0000","last_became_peered":"2024-08-19T01:33:21.163832+0000","last_unstale":"2024-08-19T01:34:01.878888+0000","last_undegraded":"2024-08-19T01:34:01.878888+0000","last_fullsized":"2024-08-19T01:34:01.878888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:31.907677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804059+0000","last_change":"2024-08-19T01:33:23.195102+0000","last_active":"2024-08-19T01:35:00.804059+0000","last_peered":"2024-08-19T01:35:00.804059+0000","last_clean":"2024-08-19T01:35:00.804059+0000","last_became_active":"2024-08-19T01:33:23.194802+0000","last_became_peered":"2024-08-19T01:33:23.194802+0000","last_unstale":"2024-08-19T01:35:00.804059+0000","last_undegraded":"2024-08-19T01:35:00.804059+0000","last_fullsized":"2024-08-19T01:35:00.804059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:33.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048628+0000","last_change":"2024-08-19T01:33:21.287933+0000","last_active":"2024-08-19T01:34:02.048628+0000","last_peered":"2024-08-19T01:34:02.048628+0000","last_clean":"2024-08-19T01:34:02.048628+0000","last_became_active":"2024-08-19T01:33:21.287568+0000","last_became_peered":"2024-08-19T01:33:21.287568+0000","last_unstale":"2024-08-19T01:34:02.048628+0000","last_undegraded":"2024-08-19T01:34:02.048628+0000","last_fullsized":"2024-08-19T01:34:02.048628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:23.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.780229+0000","last_change":"2024-08-19T01:33:23.194320+0000","last_active":"2024-08-19T01:35:00.780229+0000","last_peered":"2024-08-19T01:35:00.780229+0000","last_clean":"2024-08-19T01:35:00.780229+0000","last_became_active":"2024-08-19T01:33:23.194002+0000","last_became_peered":"2024-08-19T01:33:23.194002+0000","last_unstale":"2024-08-19T01:35:00.780229+0000","last_undegraded":"2024-08-19T01:35:00.780229+0000","last_fullsized":"2024-08-19T01:35:00.780229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:59.198997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877938+0000","last_change":"2024-08-19T01:33:21.165421+0000","last_active":"2024-08-19T01:34:01.877938+0000","last_peered":"2024-08-19T01:34:01.877938+0000","last_clean":"2024-08-19T01:34:01.877938+0000","last_became_active":"2024-08-19T01:33:21.165013+0000","last_became_peered":"2024-08-19T01:33:21.165013+0000","last_unstale":"2024-08-19T01:34:01.877938+0000","last_undegraded":"2024-08-19T01:34:01.877938+0000","last_fullsized":"2024-08-19T01:34:01.877938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:07:18.546099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803714+0000","last_change":"2024-08-19T01:33:23.193544+0000","last_active":"2024-08-19T01:35:00.803714+0000","last_peered":"2024-08-19T01:35:00.803714+0000","last_clean":"2024-08-19T01:35:00.803714+0000","last_became_active":"2024-08-19T01:33:23.193213+0000","last_became_peered":"2024-08-19T01:33:23.193213+0000","last_unstale":"2024-08-19T01:35:00.803714+0000","last_undegraded":"2024-08-19T01:35:00.803714+0000","last_fullsized":"2024-08-19T01:35:00.803714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:20:54.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:17.948248+0000","last_change":"2024-08-19T01:34:01.889627+0000","last_active":"2024-08-19T01:35:17.948248+0000","last_peered":"2024-08-19T01:35:17.948248+0000","last_clean":"2024-08-19T01:35:17.948248+0000","last_became_active":"2024-08-19T01:34:01.885181+0000","last_became_peered":"2024-08-19T01:34:01.885181+0000","last_unstale":"2024-08-19T01:35:17.948248+0000","last_undegraded":"2024-08-19T01:35:17.948248+0000","last_fullsized":"2024-08-19T01:35:17.948248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:51.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801718+0000","last_change":"2024-08-19T01:33:23.196266+0000","last_active":"2024-08-19T01:35:00.801718+0000","last_peered":"2024-08-19T01:35:00.801718+0000","last_clean":"2024-08-19T01:35:00.801718+0000","last_became_active":"2024-08-19T01:33:23.194268+0000","last_became_peered":"2024-08-19T01:33:23.194268+0000","last_unstale":"2024-08-19T01:35:00.801718+0000","last_undegraded":"2024-08-19T01:35:00.801718+0000","last_fullsized":"2024-08-19T01:35:00.801718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:15:38.628746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.890210+0000","last_change":"2024-08-19T01:34:01.890210+0000","last_active":"2024-08-19T01:34:01.890210+0000","last_peered":"2024-08-19T01:34:01.890210+0000","last_clean":"2024-08-19T01:34:01.890210+0000","last_became_active":"2024-08-19T01:34:01.885955+0000","last_became_peered":"2024-08-19T01:34:01.885955+0000","last_unstale":"2024-08-19T01:34:01.890210+0000","last_undegraded":"2024-08-19T01:34:01.890210+0000","last_fullsized":"2024-08-19T01:34:01.890210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.176530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804239+0000","last_change":"2024-08-19T01:33:23.202534+0000","last_active":"2024-08-19T01:35:00.804239+0000","last_peered":"2024-08-19T01:35:00.804239+0000","last_clean":"2024-08-19T01:35:00.804239+0000","last_became_active":"2024-08-19T01:33:23.202191+0000","last_became_peered":"2024-08-19T01:33:23.202191+0000","last_unstale":"2024-08-19T01:35:00.804239+0000","last_undegraded":"2024-08-19T01:35:00.804239+0000","last_fullsized":"2024-08-19T01:35:00.804239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:03.478905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797711+0000","last_change":"2024-08-19T01:33:23.201136+0000","last_active":"2024-08-19T01:35:00.797711+0000","last_peered":"2024-08-19T01:35:00.797711+0000","last_clean":"2024-08-19T01:35:00.797711+0000","last_became_active":"2024-08-19T01:33:23.200864+0000","last_became_peered":"2024-08-19T01:33:23.200864+0000","last_unstale":"2024-08-19T01:35:00.797711+0000","last_undegraded":"2024-08-19T01:35:00.797711+0000","last_fullsized":"2024-08-19T01:35:00.797711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:32:42.091880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883696+0000","last_change":"2024-08-19T01:33:21.172149+0000","last_active":"2024-08-19T01:34:01.883696+0000","last_peered":"2024-08-19T01:34:01.883696+0000","last_clean":"2024-08-19T01:34:01.883696+0000","last_became_active":"2024-08-19T01:33:21.171833+0000","last_became_peered":"2024-08-19T01:33:21.171833+0000","last_unstale":"2024-08-19T01:34:01.883696+0000","last_undegraded":"2024-08-19T01:34:01.883696+0000","last_fullsized":"2024-08-19T01:34:01.883696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:25:11.097212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793188+0000","last_change":"2024-08-19T01:33:23.193675+0000","last_active":"2024-08-19T01:35:00.793188+0000","last_peered":"2024-08-19T01:35:00.793188+0000","last_clean":"2024-08-19T01:35:00.793188+0000","last_became_active":"2024-08-19T01:33:23.188719+0000","last_became_peered":"2024-08-19T01:33:23.188719+0000","last_unstale":"2024-08-19T01:35:00.793188+0000","last_undegraded":"2024-08-19T01:35:00.793188+0000","last_fullsized":"2024-08-19T01:35:00.793188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:54:12.698211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883781+0000","last_change":"2024-08-19T01:33:21.172675+0000","last_active":"2024-08-19T01:34:01.883781+0000","last_peered":"2024-08-19T01:34:01.883781+0000","last_clean":"2024-08-19T01:34:01.883781+0000","last_became_active":"2024-08-19T01:33:21.172320+0000","last_became_peered":"2024-08-19T01:33:21.172320+0000","last_unstale":"2024-08-19T01:34:01.883781+0000","last_undegraded":"2024-08-19T01:34:01.883781+0000","last_fullsized":"2024-08-19T01:34:01.883781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:29:41.511220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804015+0000","last_change":"2024-08-19T01:33:23.203113+0000","last_active":"2024-08-19T01:35:00.804015+0000","last_peered":"2024-08-19T01:35:00.804015+0000","last_clean":"2024-08-19T01:35:00.804015+0000","last_became_active":"2024-08-19T01:33:23.202807+0000","last_became_peered":"2024-08-19T01:33:23.202807+0000","last_unstale":"2024-08-19T01:35:00.804015+0000","last_undegraded":"2024-08-19T01:35:00.804015+0000","last_fullsized":"2024-08-19T01:35:00.804015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:24:21.570929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878245+0000","last_change":"2024-08-19T01:33:21.287382+0000","last_active":"2024-08-19T01:34:01.878245+0000","last_peered":"2024-08-19T01:34:01.878245+0000","last_clean":"2024-08-19T01:34:01.878245+0000","last_became_active":"2024-08-19T01:33:21.287040+0000","last_became_peered":"2024-08-19T01:33:21.287040+0000","last_unstale":"2024-08-19T01:34:01.878245+0000","last_undegraded":"2024-08-19T01:34:01.878245+0000","last_fullsized":"2024-08-19T01:34:01.878245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:01:49.973969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792352+0000","last_change":"2024-08-19T01:33:23.197869+0000","last_active":"2024-08-19T01:35:00.792352+0000","last_peered":"2024-08-19T01:35:00.792352+0000","last_clean":"2024-08-19T01:35:00.792352+0000","last_became_active":"2024-08-19T01:33:23.197529+0000","last_became_peered":"2024-08-19T01:33:23.197529+0000","last_unstale":"2024-08-19T01:35:00.792352+0000","last_undegraded":"2024-08-19T01:35:00.792352+0000","last_fullsized":"2024-08-19T01:35:00.792352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:14.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'15","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.143334+0000","last_change":"2024-08-19T01:33:21.172180+0000","last_active":"2024-08-19T01:35:18.143334+0000","last_peered":"2024-08-19T01:35:18.143334+0000","last_clean":"2024-08-19T01:35:18.143334+0000","last_became_active":"2024-08-19T01:33:21.171895+0000","last_became_peered":"2024-08-19T01:33:21.171895+0000","last_unstale":"2024-08-19T01:35:18.143334+0000","last_undegraded":"2024-08-19T01:35:18.143334+0000","last_fullsized":"2024-08-19T01:35:18.143334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:51:38.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75654,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1113,"num_read_kb":635,"num_write":276,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":218334,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":454575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1111694,"data_compressed":37649,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":453901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":454167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1049475,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":454641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1047107,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":454502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":453443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":517511,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":517023,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":454957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64244,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64732,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":68430,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:35:23.934 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-19T01:35:23.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-19T01:35:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 -- 172.21.15.103:0/2216343584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 msgr2=0x7f09e0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 --2- 172.21.15.103:0/2216343584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0102ed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009900 tx=0x7f09c802a410 comp rx=0 tx=0).stop 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 -- 172.21.15.103:0/2216343584 shutdown_connections 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 --2- 172.21.15.103:0/2216343584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09e0103410 0x7f09e0109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 --2- 172.21.15.103:0/2216343584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0102ed0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 --2- 172.21.15.103:0/2216343584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09e0101860 0x7f09e0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 -- 172.21.15.103:0/2216343584 >> 172.21.15.103:0/2216343584 conn(0x7f09e00fcfe0 msgr2=0x7f09e00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 -- 172.21.15.103:0/2216343584 shutdown_connections 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.035+0000 7f09e51d8640 1 -- 172.21.15.103:0/2216343584 wait complete. 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 Processor -- start 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 -- start start 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09e0101860 0x7f09e0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09e0103410 0x7f09e019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010d390 con 0x7f09e0102a50 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010d210 con 0x7f09e0101860 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09e51d8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f09e010d510 con 0x7f09e0103410 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51330/0 (socket says 172.21.15.103:51330) 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 -- 172.21.15.103:0/1275556841 learned_addr learned my addr 172.21.15.103:0/1275556841 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09ded9d640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09e0101860 0x7f09e0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09df59e640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09e0103410 0x7f09e019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 -- 172.21.15.103:0/1275556841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09e0103410 msgr2=0x7f09e019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09e0103410 0x7f09e019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 -- 172.21.15.103:0/1275556841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09e0101860 msgr2=0x7f09e0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09e0101860 0x7f09e0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09de59c640 1 -- 172.21.15.103:0/1275556841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e019f720 con 0x7f09e0102a50 2024-08-19T01:35:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.036+0000 7f09df59e640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09e0103410 0x7f09e019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.037+0000 7f09de59c640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0198040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005940 tx=0x7f09c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.037+0000 7f09d7fff640 1 -- 172.21.15.103:0/1275556841 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c803f030 con 0x7f09e0102a50 2024-08-19T01:35:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.037+0000 7f09d7fff640 1 -- 172.21.15.103:0/1275556841 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09c8003cb0 con 0x7f09e0102a50 2024-08-19T01:35:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.037+0000 7f09d7fff640 1 -- 172.21.15.103:0/1275556841 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8035070 con 0x7f09e0102a50 2024-08-19T01:35:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.037+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e019f9b0 con 0x7f09e0102a50 2024-08-19T01:35:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.037+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e019fea0 con 0x7f09e0102a50 2024-08-19T01:35:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.038+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0108d40 con 0x7f09e0102a50 2024-08-19T01:35:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.039+0000 7f09d7fff640 1 -- 172.21.15.103:0/1275556841 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f09c8042030 con 0x7f09e0102a50 2024-08-19T01:35:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.040+0000 7f09d7fff640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f09cc0a2b90 0x7f09cc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.040+0000 7f09ded9d640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f09cc0a2b90 0x7f09cc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.040+0000 7f09d7fff640 1 -- 172.21.15.103:0/1275556841 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f09c80ef620 con 0x7f09e0102a50 2024-08-19T01:35:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.040+0000 7f09ded9d640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f09cc0a2b90 0x7f09cc0a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09e01028b0 tx=0x7f09d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.044+0000 7f09d7fff640 1 -- 172.21.15.103:0/1275556841 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f09c80bc5b0 con 0x7f09e0102a50 2024-08-19T01:35:24.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.207+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 --> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09e0062360 con 0x7f09cc0a2b90 2024-08-19T01:35:24.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.213+0000 7f09d7fff640 1 -- 172.21.15.103:0/1275556841 <== mgr.14148 v2:172.21.15.103:6800/2656970648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259116 (secure 0 0 0) 0x7f09e0062360 con 0x7f09cc0a2b90 2024-08-19T01:35:24.218 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:24.219 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-08-19T01:35:24.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f09cc0a2b90 msgr2=0x7f09cc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:24.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f09cc0a2b90 0x7f09cc0a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09e01028b0 tx=0x7f09d0005f90 comp rx=0 tx=0).stop 2024-08-19T01:35:24.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 msgr2=0x7f09e0198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:24.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0198040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005940 tx=0x7f09c80042f0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 shutdown_connections 2024-08-19T01:35:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f09cc0a2b90 0x7f09cc0a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09e0103410 0x7f09e019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0102a50 0x7f09e0198040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 --2- 172.21.15.103:0/1275556841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09e0101860 0x7f09e0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 >> 172.21.15.103:0/1275556841 conn(0x7f09e00fcfe0 msgr2=0x7f09e0105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.216+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 shutdown_connections 2024-08-19T01:35:24.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:24.217+0000 7f09e51d8640 1 -- 172.21.15.103:0/1275556841 wait complete. 2024-08-19T01:35:24.228 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:23 smithi103 ceph-mon[18436]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:24.228 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:23 smithi103 ceph-mon[18436]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:24.229 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-08-19T01:35:22.800223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666256,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1239,"num_read_kb":744,"num_write":459,"num_write_kb":3103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364872,"kb_used_data":11560,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576952,"statfs":{"total":1151940427776,"available":1151566798848,"internally_reserved":0,"allocated":11837440,"data_stored":7422996,"data_compressed":112947,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6774,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002958"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804705+0000","last_change":"2024-08-19T01:33:23.202035+0000","last_active":"2024-08-19T01:35:00.804705+0000","last_peered":"2024-08-19T01:35:00.804705+0000","last_clean":"2024-08-19T01:35:00.804705+0000","last_became_active":"2024-08-19T01:33:23.201418+0000","last_became_peered":"2024-08-19T01:33:23.201418+0000","last_unstale":"2024-08-19T01:35:00.804705+0000","last_undegraded":"2024-08-19T01:35:00.804705+0000","last_fullsized":"2024-08-19T01:35:00.804705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:54:57.927424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802387+0000","last_change":"2024-08-19T01:33:23.202047+0000","last_active":"2024-08-19T01:35:00.802387+0000","last_peered":"2024-08-19T01:35:00.802387+0000","last_clean":"2024-08-19T01:35:00.802387+0000","last_became_active":"2024-08-19T01:33:23.201762+0000","last_became_peered":"2024-08-19T01:33:23.201762+0000","last_unstale":"2024-08-19T01:35:00.802387+0000","last_undegraded":"2024-08-19T01:35:00.802387+0000","last_fullsized":"2024-08-19T01:35:00.802387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:23:57.252284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804290+0000","last_change":"2024-08-19T01:33:23.202876+0000","last_active":"2024-08-19T01:35:00.804290+0000","last_peered":"2024-08-19T01:35:00.804290+0000","last_clean":"2024-08-19T01:35:00.804290+0000","last_became_active":"2024-08-19T01:33:23.202528+0000","last_became_peered":"2024-08-19T01:33:23.202528+0000","last_unstale":"2024-08-19T01:35:00.804290+0000","last_undegraded":"2024-08-19T01:35:00.804290+0000","last_fullsized":"2024-08-19T01:35:00.804290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:37:20.514390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790002+0000","last_change":"2024-08-19T01:33:23.199443+0000","last_active":"2024-08-19T01:35:00.790002+0000","last_peered":"2024-08-19T01:35:00.790002+0000","last_clean":"2024-08-19T01:35:00.790002+0000","last_became_active":"2024-08-19T01:33:23.198344+0000","last_became_peered":"2024-08-19T01:33:23.198344+0000","last_unstale":"2024-08-19T01:35:00.790002+0000","last_undegraded":"2024-08-19T01:35:00.790002+0000","last_fullsized":"2024-08-19T01:35:00.790002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:43:43.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800162+0000","last_change":"2024-08-19T01:33:23.191185+0000","last_active":"2024-08-19T01:35:00.800162+0000","last_peered":"2024-08-19T01:35:00.800162+0000","last_clean":"2024-08-19T01:35:00.800162+0000","last_became_active":"2024-08-19T01:33:23.190892+0000","last_became_peered":"2024-08-19T01:33:23.190892+0000","last_unstale":"2024-08-19T01:35:00.800162+0000","last_undegraded":"2024-08-19T01:35:00.800162+0000","last_fullsized":"2024-08-19T01:35:00.800162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:46:28.384609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.795199+0000","last_change":"2024-08-19T01:33:23.193036+0000","last_active":"2024-08-19T01:35:00.795199+0000","last_peered":"2024-08-19T01:35:00.795199+0000","last_clean":"2024-08-19T01:35:00.795199+0000","last_became_active":"2024-08-19T01:33:23.192686+0000","last_became_peered":"2024-08-19T01:33:23.192686+0000","last_unstale":"2024-08-19T01:35:00.795199+0000","last_undegraded":"2024-08-19T01:35:00.795199+0000","last_fullsized":"2024-08-19T01:35:00.795199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:46.235112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801472+0000","last_change":"2024-08-19T01:33:23.197636+0000","last_active":"2024-08-19T01:35:00.801472+0000","last_peered":"2024-08-19T01:35:00.801472+0000","last_clean":"2024-08-19T01:35:00.801472+0000","last_became_active":"2024-08-19T01:33:23.197323+0000","last_became_peered":"2024-08-19T01:33:23.197323+0000","last_unstale":"2024-08-19T01:35:00.801472+0000","last_undegraded":"2024-08-19T01:35:00.801472+0000","last_fullsized":"2024-08-19T01:35:00.801472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:46:36.168786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802002+0000","last_change":"2024-08-19T01:33:23.193717+0000","last_active":"2024-08-19T01:35:00.802002+0000","last_peered":"2024-08-19T01:35:00.802002+0000","last_clean":"2024-08-19T01:35:00.802002+0000","last_became_active":"2024-08-19T01:33:23.193376+0000","last_became_peered":"2024-08-19T01:33:23.193376+0000","last_unstale":"2024-08-19T01:35:00.802002+0000","last_undegraded":"2024-08-19T01:35:00.802002+0000","last_fullsized":"2024-08-19T01:35:00.802002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:16.290081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796833+0000","last_change":"2024-08-19T01:33:23.197956+0000","last_active":"2024-08-19T01:35:00.796833+0000","last_peered":"2024-08-19T01:35:00.796833+0000","last_clean":"2024-08-19T01:35:00.796833+0000","last_became_active":"2024-08-19T01:33:23.197667+0000","last_became_peered":"2024-08-19T01:33:23.197667+0000","last_unstale":"2024-08-19T01:35:00.796833+0000","last_undegraded":"2024-08-19T01:35:00.796833+0000","last_fullsized":"2024-08-19T01:35:00.796833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:16.841046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792010+0000","last_change":"2024-08-19T01:33:23.194569+0000","last_active":"2024-08-19T01:35:00.792010+0000","last_peered":"2024-08-19T01:35:00.792010+0000","last_clean":"2024-08-19T01:35:00.792010+0000","last_became_active":"2024-08-19T01:33:23.194223+0000","last_became_peered":"2024-08-19T01:33:23.194223+0000","last_unstale":"2024-08-19T01:35:00.792010+0000","last_undegraded":"2024-08-19T01:35:00.792010+0000","last_fullsized":"2024-08-19T01:35:00.792010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:14:30.184738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800081+0000","last_change":"2024-08-19T01:33:23.202396+0000","last_active":"2024-08-19T01:35:00.800081+0000","last_peered":"2024-08-19T01:35:00.800081+0000","last_clean":"2024-08-19T01:35:00.800081+0000","last_became_active":"2024-08-19T01:33:23.202063+0000","last_became_peered":"2024-08-19T01:33:23.202063+0000","last_unstale":"2024-08-19T01:35:00.800081+0000","last_undegraded":"2024-08-19T01:35:00.800081+0000","last_fullsized":"2024-08-19T01:35:00.800081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:57.513639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.789724+0000","last_change":"2024-08-19T01:33:23.200610+0000","last_active":"2024-08-19T01:35:00.789724+0000","last_peered":"2024-08-19T01:35:00.789724+0000","last_clean":"2024-08-19T01:35:00.789724+0000","last_became_active":"2024-08-19T01:33:23.200136+0000","last_became_peered":"2024-08-19T01:33:23.200136+0000","last_unstale":"2024-08-19T01:35:00.789724+0000","last_undegraded":"2024-08-19T01:35:00.789724+0000","last_fullsized":"2024-08-19T01:35:00.789724+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:47:39.252702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792991+0000","last_change":"2024-08-19T01:33:23.196062+0000","last_active":"2024-08-19T01:35:00.792991+0000","last_peered":"2024-08-19T01:35:00.792991+0000","last_clean":"2024-08-19T01:35:00.792991+0000","last_became_active":"2024-08-19T01:33:23.195577+0000","last_became_peered":"2024-08-19T01:33:23.195577+0000","last_unstale":"2024-08-19T01:35:00.792991+0000","last_undegraded":"2024-08-19T01:35:00.792991+0000","last_fullsized":"2024-08-19T01:35:00.792991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:46:09.111323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804514+0000","last_change":"2024-08-19T01:33:23.201664+0000","last_active":"2024-08-19T01:35:00.804514+0000","last_peered":"2024-08-19T01:35:00.804514+0000","last_clean":"2024-08-19T01:35:00.804514+0000","last_became_active":"2024-08-19T01:33:23.200241+0000","last_became_peered":"2024-08-19T01:33:23.200241+0000","last_unstale":"2024-08-19T01:35:00.804514+0000","last_undegraded":"2024-08-19T01:35:00.804514+0000","last_fullsized":"2024-08-19T01:35:00.804514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:17.199776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803104+0000","last_change":"2024-08-19T01:33:23.198102+0000","last_active":"2024-08-19T01:35:00.803104+0000","last_peered":"2024-08-19T01:35:00.803104+0000","last_clean":"2024-08-19T01:35:00.803104+0000","last_became_active":"2024-08-19T01:33:23.197778+0000","last_became_peered":"2024-08-19T01:33:23.197778+0000","last_unstale":"2024-08-19T01:35:00.803104+0000","last_undegraded":"2024-08-19T01:35:00.803104+0000","last_fullsized":"2024-08-19T01:35:00.803104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:24:03.554134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804605+0000","last_change":"2024-08-19T01:33:23.194581+0000","last_active":"2024-08-19T01:35:00.804605+0000","last_peered":"2024-08-19T01:35:00.804605+0000","last_clean":"2024-08-19T01:35:00.804605+0000","last_became_active":"2024-08-19T01:33:23.194256+0000","last_became_peered":"2024-08-19T01:33:23.194256+0000","last_unstale":"2024-08-19T01:35:00.804605+0000","last_undegraded":"2024-08-19T01:35:00.804605+0000","last_fullsized":"2024-08-19T01:35:00.804605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:17:20.954190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804738+0000","last_change":"2024-08-19T01:33:23.201242+0000","last_active":"2024-08-19T01:35:00.804738+0000","last_peered":"2024-08-19T01:35:00.804738+0000","last_clean":"2024-08-19T01:35:00.804738+0000","last_became_active":"2024-08-19T01:33:23.200835+0000","last_became_peered":"2024-08-19T01:33:23.200835+0000","last_unstale":"2024-08-19T01:35:00.804738+0000","last_undegraded":"2024-08-19T01:35:00.804738+0000","last_fullsized":"2024-08-19T01:35:00.804738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:35:54.312266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.799770+0000","last_change":"2024-08-19T01:33:23.187728+0000","last_active":"2024-08-19T01:35:00.799770+0000","last_peered":"2024-08-19T01:35:00.799770+0000","last_clean":"2024-08-19T01:35:00.799770+0000","last_became_active":"2024-08-19T01:33:23.187352+0000","last_became_peered":"2024-08-19T01:33:23.187352+0000","last_unstale":"2024-08-19T01:35:00.799770+0000","last_undegraded":"2024-08-19T01:35:00.799770+0000","last_fullsized":"2024-08-19T01:35:00.799770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:06:03.041328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797919+0000","last_change":"2024-08-19T01:33:23.193146+0000","last_active":"2024-08-19T01:35:00.797919+0000","last_peered":"2024-08-19T01:35:00.797919+0000","last_clean":"2024-08-19T01:35:00.797919+0000","last_became_active":"2024-08-19T01:33:23.192810+0000","last_became_peered":"2024-08-19T01:33:23.192810+0000","last_unstale":"2024-08-19T01:35:00.797919+0000","last_undegraded":"2024-08-19T01:35:00.797919+0000","last_fullsized":"2024-08-19T01:35:00.797919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:12.044175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.791343+0000","last_change":"2024-08-19T01:33:23.194635+0000","last_active":"2024-08-19T01:35:00.791343+0000","last_peered":"2024-08-19T01:35:00.791343+0000","last_clean":"2024-08-19T01:35:00.791343+0000","last_became_active":"2024-08-19T01:33:23.194308+0000","last_became_peered":"2024-08-19T01:33:23.194308+0000","last_unstale":"2024-08-19T01:35:00.791343+0000","last_undegraded":"2024-08-19T01:35:00.791343+0000","last_fullsized":"2024-08-19T01:35:00.791343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:12.119367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802081+0000","last_change":"2024-08-19T01:33:23.193108+0000","last_active":"2024-08-19T01:35:00.802081+0000","last_peered":"2024-08-19T01:35:00.802081+0000","last_clean":"2024-08-19T01:35:00.802081+0000","last_became_active":"2024-08-19T01:33:23.192830+0000","last_became_peered":"2024-08-19T01:33:23.192830+0000","last_unstale":"2024-08-19T01:35:00.802081+0000","last_undegraded":"2024-08-19T01:35:00.802081+0000","last_fullsized":"2024-08-19T01:35:00.802081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:33.355498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803102+0000","last_change":"2024-08-19T01:33:23.195789+0000","last_active":"2024-08-19T01:35:00.803102+0000","last_peered":"2024-08-19T01:35:00.803102+0000","last_clean":"2024-08-19T01:35:00.803102+0000","last_became_active":"2024-08-19T01:33:23.195064+0000","last_became_peered":"2024-08-19T01:33:23.195064+0000","last_unstale":"2024-08-19T01:35:00.803102+0000","last_undegraded":"2024-08-19T01:35:00.803102+0000","last_fullsized":"2024-08-19T01:35:00.803102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:05:33.848953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.802554+0000","last_change":"2024-08-19T01:33:23.200968+0000","last_active":"2024-08-19T01:35:00.802554+0000","last_peered":"2024-08-19T01:35:00.802554+0000","last_clean":"2024-08-19T01:35:00.802554+0000","last_became_active":"2024-08-19T01:33:23.200606+0000","last_became_peered":"2024-08-19T01:33:23.200606+0000","last_unstale":"2024-08-19T01:35:00.802554+0000","last_undegraded":"2024-08-19T01:35:00.802554+0000","last_fullsized":"2024-08-19T01:35:00.802554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:26:44.718160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.798216+0000","last_change":"2024-08-19T01:33:23.191204+0000","last_active":"2024-08-19T01:35:00.798216+0000","last_peered":"2024-08-19T01:35:00.798216+0000","last_clean":"2024-08-19T01:35:00.798216+0000","last_became_active":"2024-08-19T01:33:23.190709+0000","last_became_peered":"2024-08-19T01:33:23.190709+0000","last_unstale":"2024-08-19T01:35:00.798216+0000","last_undegraded":"2024-08-19T01:35:00.798216+0000","last_fullsized":"2024-08-19T01:35:00.798216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:56:10.988967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793707+0000","last_change":"2024-08-19T01:34:01.885389+0000","last_active":"2024-08-19T01:35:00.793707+0000","last_peered":"2024-08-19T01:35:00.793707+0000","last_clean":"2024-08-19T01:35:00.793707+0000","last_became_active":"2024-08-19T01:34:01.885100+0000","last_became_peered":"2024-08-19T01:34:01.885100+0000","last_unstale":"2024-08-19T01:35:00.793707+0000","last_undegraded":"2024-08-19T01:35:00.793707+0000","last_fullsized":"2024-08-19T01:35:00.793707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:21.616876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.782764+0000","last_change":"2024-08-19T01:33:23.187309+0000","last_active":"2024-08-19T01:35:00.782764+0000","last_peered":"2024-08-19T01:35:00.782764+0000","last_clean":"2024-08-19T01:35:00.782764+0000","last_became_active":"2024-08-19T01:33:23.186976+0000","last_became_peered":"2024-08-19T01:33:23.186976+0000","last_unstale":"2024-08-19T01:35:00.782764+0000","last_undegraded":"2024-08-19T01:35:00.782764+0000","last_fullsized":"2024-08-19T01:35:00.782764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:46:02.852000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790722+0000","last_change":"2024-08-19T01:33:23.190292+0000","last_active":"2024-08-19T01:35:00.790722+0000","last_peered":"2024-08-19T01:35:00.790722+0000","last_clean":"2024-08-19T01:35:00.790722+0000","last_became_active":"2024-08-19T01:33:23.189969+0000","last_became_peered":"2024-08-19T01:33:23.189969+0000","last_unstale":"2024-08-19T01:35:00.790722+0000","last_undegraded":"2024-08-19T01:35:00.790722+0000","last_fullsized":"2024-08-19T01:35:00.790722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:35:51.388150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796348+0000","last_change":"2024-08-19T01:33:23.192148+0000","last_active":"2024-08-19T01:35:00.796348+0000","last_peered":"2024-08-19T01:35:00.796348+0000","last_clean":"2024-08-19T01:35:00.796348+0000","last_became_active":"2024-08-19T01:33:23.191845+0000","last_became_peered":"2024-08-19T01:33:23.191845+0000","last_unstale":"2024-08-19T01:35:00.796348+0000","last_undegraded":"2024-08-19T01:35:00.796348+0000","last_fullsized":"2024-08-19T01:35:00.796348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:43:51.308393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803032+0000","last_change":"2024-08-19T01:33:23.195033+0000","last_active":"2024-08-19T01:35:00.803032+0000","last_peered":"2024-08-19T01:35:00.803032+0000","last_clean":"2024-08-19T01:35:00.803032+0000","last_became_active":"2024-08-19T01:33:23.194745+0000","last_became_peered":"2024-08-19T01:33:23.194745+0000","last_unstale":"2024-08-19T01:35:00.803032+0000","last_undegraded":"2024-08-19T01:35:00.803032+0000","last_fullsized":"2024-08-19T01:35:00.803032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:33:40.748311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797892+0000","last_change":"2024-08-19T01:33:23.195817+0000","last_active":"2024-08-19T01:35:00.797892+0000","last_peered":"2024-08-19T01:35:00.797892+0000","last_clean":"2024-08-19T01:35:00.797892+0000","last_became_active":"2024-08-19T01:33:23.194688+0000","last_became_peered":"2024-08-19T01:33:23.194688+0000","last_unstale":"2024-08-19T01:35:00.797892+0000","last_undegraded":"2024-08-19T01:35:00.797892+0000","last_fullsized":"2024-08-19T01:35:00.797892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:13:04.913921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793529+0000","last_change":"2024-08-19T01:33:23.187604+0000","last_active":"2024-08-19T01:35:00.793529+0000","last_peered":"2024-08-19T01:35:00.793529+0000","last_clean":"2024-08-19T01:35:00.793529+0000","last_became_active":"2024-08-19T01:33:23.187242+0000","last_became_peered":"2024-08-19T01:33:23.187242+0000","last_unstale":"2024-08-19T01:35:00.793529+0000","last_undegraded":"2024-08-19T01:35:00.793529+0000","last_fullsized":"2024-08-19T01:35:00.793529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:28.592525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792374+0000","last_change":"2024-08-19T01:33:23.191737+0000","last_active":"2024-08-19T01:35:00.792374+0000","last_peered":"2024-08-19T01:35:00.792374+0000","last_clean":"2024-08-19T01:35:00.792374+0000","last_became_active":"2024-08-19T01:33:23.191358+0000","last_became_peered":"2024-08-19T01:33:23.191358+0000","last_unstale":"2024-08-19T01:35:00.792374+0000","last_undegraded":"2024-08-19T01:35:00.792374+0000","last_fullsized":"2024-08-19T01:35:00.792374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:28:14.694157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.785064+0000","last_change":"2024-08-19T01:33:23.193110+0000","last_active":"2024-08-19T01:35:00.785064+0000","last_peered":"2024-08-19T01:35:00.785064+0000","last_clean":"2024-08-19T01:35:00.785064+0000","last_became_active":"2024-08-19T01:33:23.192789+0000","last_became_peered":"2024-08-19T01:33:23.192789+0000","last_unstale":"2024-08-19T01:35:00.785064+0000","last_undegraded":"2024-08-19T01:35:00.785064+0000","last_fullsized":"2024-08-19T01:35:00.785064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:45:48.141403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:20.124936+0000","last_change":"2024-08-19T01:33:21.173518+0000","last_active":"2024-08-19T01:35:20.124936+0000","last_peered":"2024-08-19T01:35:20.124936+0000","last_clean":"2024-08-19T01:35:20.124936+0000","last_became_active":"2024-08-19T01:33:21.172288+0000","last_became_peered":"2024-08-19T01:33:21.172288+0000","last_unstale":"2024-08-19T01:35:20.124936+0000","last_undegraded":"2024-08-19T01:35:20.124936+0000","last_fullsized":"2024-08-19T01:35:20.124936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:59:42.864724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1645,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801261+0000","last_change":"2024-08-19T01:33:23.190160+0000","last_active":"2024-08-19T01:35:00.801261+0000","last_peered":"2024-08-19T01:35:00.801261+0000","last_clean":"2024-08-19T01:35:00.801261+0000","last_became_active":"2024-08-19T01:33:23.189802+0000","last_became_peered":"2024-08-19T01:33:23.189802+0000","last_unstale":"2024-08-19T01:35:00.801261+0000","last_undegraded":"2024-08-19T01:35:00.801261+0000","last_fullsized":"2024-08-19T01:35:00.801261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:53:10.097897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880543+0000","last_change":"2024-08-19T01:33:21.167443+0000","last_active":"2024-08-19T01:34:01.880543+0000","last_peered":"2024-08-19T01:34:01.880543+0000","last_clean":"2024-08-19T01:34:01.880543+0000","last_became_active":"2024-08-19T01:33:21.167054+0000","last_became_peered":"2024-08-19T01:33:21.167054+0000","last_unstale":"2024-08-19T01:34:01.880543+0000","last_undegraded":"2024-08-19T01:34:01.880543+0000","last_fullsized":"2024-08-19T01:34:01.880543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:00:38.358807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.928860+0000","last_change":"2024-08-19T01:33:21.163097+0000","last_active":"2024-08-19T01:34:01.928860+0000","last_peered":"2024-08-19T01:34:01.928860+0000","last_clean":"2024-08-19T01:34:01.928860+0000","last_became_active":"2024-08-19T01:33:21.162802+0000","last_became_peered":"2024-08-19T01:33:21.162802+0000","last_unstale":"2024-08-19T01:34:01.928860+0000","last_undegraded":"2024-08-19T01:34:01.928860+0000","last_fullsized":"2024-08-19T01:34:01.928860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:15:41.244104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803050+0000","last_change":"2024-08-19T01:33:23.200975+0000","last_active":"2024-08-19T01:35:00.803050+0000","last_peered":"2024-08-19T01:35:00.803050+0000","last_clean":"2024-08-19T01:35:00.803050+0000","last_became_active":"2024-08-19T01:33:23.200667+0000","last_became_peered":"2024-08-19T01:33:23.200667+0000","last_unstale":"2024-08-19T01:35:00.803050+0000","last_undegraded":"2024-08-19T01:35:00.803050+0000","last_fullsized":"2024-08-19T01:35:00.803050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:31:35.105521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879336+0000","last_change":"2024-08-19T01:33:21.171252+0000","last_active":"2024-08-19T01:34:01.879336+0000","last_peered":"2024-08-19T01:34:01.879336+0000","last_clean":"2024-08-19T01:34:01.879336+0000","last_became_active":"2024-08-19T01:33:21.170944+0000","last_became_peered":"2024-08-19T01:33:21.170944+0000","last_unstale":"2024-08-19T01:34:01.879336+0000","last_undegraded":"2024-08-19T01:34:01.879336+0000","last_fullsized":"2024-08-19T01:34:01.879336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:10:15.869452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801852+0000","last_change":"2024-08-19T01:33:23.191931+0000","last_active":"2024-08-19T01:35:00.801852+0000","last_peered":"2024-08-19T01:35:00.801852+0000","last_clean":"2024-08-19T01:35:00.801852+0000","last_became_active":"2024-08-19T01:33:23.191601+0000","last_became_peered":"2024-08-19T01:33:23.191601+0000","last_unstale":"2024-08-19T01:35:00.801852+0000","last_undegraded":"2024-08-19T01:35:00.801852+0000","last_fullsized":"2024-08-19T01:35:00.801852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:29:52.874546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":243,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.125745+0000","last_change":"2024-08-19T01:33:21.170714+0000","last_active":"2024-08-19T01:35:21.125745+0000","last_peered":"2024-08-19T01:35:21.125745+0000","last_clean":"2024-08-19T01:35:21.125745+0000","last_became_active":"2024-08-19T01:33:21.170379+0000","last_became_peered":"2024-08-19T01:33:21.170379+0000","last_unstale":"2024-08-19T01:35:21.125745+0000","last_undegraded":"2024-08-19T01:35:21.125745+0000","last_fullsized":"2024-08-19T01:35:21.125745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:27:27.417634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.791818+0000","last_change":"2024-08-19T01:33:23.189680+0000","last_active":"2024-08-19T01:35:00.791818+0000","last_peered":"2024-08-19T01:35:00.791818+0000","last_clean":"2024-08-19T01:35:00.791818+0000","last_became_active":"2024-08-19T01:33:23.189029+0000","last_became_peered":"2024-08-19T01:33:23.189029+0000","last_unstale":"2024-08-19T01:35:00.791818+0000","last_undegraded":"2024-08-19T01:35:00.791818+0000","last_fullsized":"2024-08-19T01:35:00.791818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:39.667675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.881530+0000","last_change":"2024-08-19T01:33:21.169233+0000","last_active":"2024-08-19T01:34:01.881530+0000","last_peered":"2024-08-19T01:34:01.881530+0000","last_clean":"2024-08-19T01:34:01.881530+0000","last_became_active":"2024-08-19T01:33:21.168761+0000","last_became_peered":"2024-08-19T01:33:21.168761+0000","last_unstale":"2024-08-19T01:34:01.881530+0000","last_undegraded":"2024-08-19T01:34:01.881530+0000","last_fullsized":"2024-08-19T01:34:01.881530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:31:56.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804526+0000","last_change":"2024-08-19T01:33:23.194250+0000","last_active":"2024-08-19T01:35:00.804526+0000","last_peered":"2024-08-19T01:35:00.804526+0000","last_clean":"2024-08-19T01:35:00.804526+0000","last_became_active":"2024-08-19T01:33:23.193939+0000","last_became_peered":"2024-08-19T01:33:23.193939+0000","last_unstale":"2024-08-19T01:35:00.804526+0000","last_undegraded":"2024-08-19T01:35:00.804526+0000","last_fullsized":"2024-08-19T01:35:00.804526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:19:09.703701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879370+0000","last_change":"2024-08-19T01:33:21.287684+0000","last_active":"2024-08-19T01:34:01.879370+0000","last_peered":"2024-08-19T01:34:01.879370+0000","last_clean":"2024-08-19T01:34:01.879370+0000","last_became_active":"2024-08-19T01:33:21.287252+0000","last_became_peered":"2024-08-19T01:33:21.287252+0000","last_unstale":"2024-08-19T01:34:01.879370+0000","last_undegraded":"2024-08-19T01:34:01.879370+0000","last_fullsized":"2024-08-19T01:34:01.879370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:08.801345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.788353+0000","last_change":"2024-08-19T01:33:23.195077+0000","last_active":"2024-08-19T01:35:00.788353+0000","last_peered":"2024-08-19T01:35:00.788353+0000","last_clean":"2024-08-19T01:35:00.788353+0000","last_became_active":"2024-08-19T01:33:23.194753+0000","last_became_peered":"2024-08-19T01:33:23.194753+0000","last_unstale":"2024-08-19T01:35:00.788353+0000","last_undegraded":"2024-08-19T01:35:00.788353+0000","last_fullsized":"2024-08-19T01:35:00.788353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:12:09.186261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":282,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.125848+0000","last_change":"2024-08-19T01:33:21.168845+0000","last_active":"2024-08-19T01:35:21.125848+0000","last_peered":"2024-08-19T01:35:21.125848+0000","last_clean":"2024-08-19T01:35:21.125848+0000","last_became_active":"2024-08-19T01:33:21.168479+0000","last_became_peered":"2024-08-19T01:33:21.168479+0000","last_unstale":"2024-08-19T01:35:21.125848+0000","last_undegraded":"2024-08-19T01:35:21.125848+0000","last_fullsized":"2024-08-19T01:35:21.125848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:59:50.296329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65671,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":224,"num_read_kb":171,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.785907+0000","last_change":"2024-08-19T01:33:23.192252+0000","last_active":"2024-08-19T01:35:00.785907+0000","last_peered":"2024-08-19T01:35:00.785907+0000","last_clean":"2024-08-19T01:35:00.785907+0000","last_became_active":"2024-08-19T01:33:23.191934+0000","last_became_peered":"2024-08-19T01:33:23.191934+0000","last_unstale":"2024-08-19T01:35:00.785907+0000","last_undegraded":"2024-08-19T01:35:00.785907+0000","last_fullsized":"2024-08-19T01:35:00.785907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:31.134825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.882306+0000","last_change":"2024-08-19T01:33:21.168541+0000","last_active":"2024-08-19T01:34:01.882306+0000","last_peered":"2024-08-19T01:34:01.882306+0000","last_clean":"2024-08-19T01:34:01.882306+0000","last_became_active":"2024-08-19T01:33:21.168205+0000","last_became_peered":"2024-08-19T01:33:21.168205+0000","last_unstale":"2024-08-19T01:34:01.882306+0000","last_undegraded":"2024-08-19T01:34:01.882306+0000","last_fullsized":"2024-08-19T01:34:01.882306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:19:52.078947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.799749+0000","last_change":"2024-08-19T01:33:23.193013+0000","last_active":"2024-08-19T01:35:00.799749+0000","last_peered":"2024-08-19T01:35:00.799749+0000","last_clean":"2024-08-19T01:35:00.799749+0000","last_became_active":"2024-08-19T01:33:23.192728+0000","last_became_peered":"2024-08-19T01:33:23.192728+0000","last_unstale":"2024-08-19T01:35:00.799749+0000","last_undegraded":"2024-08-19T01:35:00.799749+0000","last_fullsized":"2024-08-19T01:35:00.799749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:23:17.587527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879101+0000","last_change":"2024-08-19T01:33:21.170118+0000","last_active":"2024-08-19T01:34:01.879101+0000","last_peered":"2024-08-19T01:34:01.879101+0000","last_clean":"2024-08-19T01:34:01.879101+0000","last_became_active":"2024-08-19T01:33:21.169757+0000","last_became_peered":"2024-08-19T01:33:21.169757+0000","last_unstale":"2024-08-19T01:34:01.879101+0000","last_undegraded":"2024-08-19T01:34:01.879101+0000","last_fullsized":"2024-08-19T01:34:01.879101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:09:26.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804979+0000","last_change":"2024-08-19T01:34:01.929944+0000","last_active":"2024-08-19T01:35:00.804979+0000","last_peered":"2024-08-19T01:35:00.804979+0000","last_clean":"2024-08-19T01:35:00.804979+0000","last_became_active":"2024-08-19T01:34:01.929679+0000","last_became_peered":"2024-08-19T01:34:01.929679+0000","last_unstale":"2024-08-19T01:35:00.804979+0000","last_undegraded":"2024-08-19T01:35:00.804979+0000","last_fullsized":"2024-08-19T01:35:00.804979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:25:24.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":140,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.128678+0000","last_change":"2024-08-19T01:33:21.165204+0000","last_active":"2024-08-19T01:35:18.128678+0000","last_peered":"2024-08-19T01:35:18.128678+0000","last_clean":"2024-08-19T01:35:18.128678+0000","last_became_active":"2024-08-19T01:33:21.164826+0000","last_became_peered":"2024-08-19T01:33:21.164826+0000","last_unstale":"2024-08-19T01:35:18.128678+0000","last_undegraded":"2024-08-19T01:35:18.128678+0000","last_fullsized":"2024-08-19T01:35:18.128678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:45:23.212064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800773+0000","last_change":"2024-08-19T01:33:23.198176+0000","last_active":"2024-08-19T01:35:00.800773+0000","last_peered":"2024-08-19T01:35:00.800773+0000","last_clean":"2024-08-19T01:35:00.800773+0000","last_became_active":"2024-08-19T01:33:23.196187+0000","last_became_peered":"2024-08-19T01:33:23.196187+0000","last_unstale":"2024-08-19T01:35:00.800773+0000","last_undegraded":"2024-08-19T01:35:00.800773+0000","last_fullsized":"2024-08-19T01:35:00.800773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:53:05.943467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.079412+0000","last_change":"2024-08-19T01:34:02.079028+0000","last_active":"2024-08-19T01:34:02.079412+0000","last_peered":"2024-08-19T01:34:02.079412+0000","last_clean":"2024-08-19T01:34:02.079412+0000","last_became_active":"2024-08-19T01:34:02.049890+0000","last_became_peered":"2024-08-19T01:34:02.049890+0000","last_unstale":"2024-08-19T01:34:02.079412+0000","last_undegraded":"2024-08-19T01:34:02.079412+0000","last_fullsized":"2024-08-19T01:34:02.079412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:58.595161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.789928+0000","last_change":"2024-08-19T01:33:23.201104+0000","last_active":"2024-08-19T01:35:00.789928+0000","last_peered":"2024-08-19T01:35:00.789928+0000","last_clean":"2024-08-19T01:35:00.789928+0000","last_became_active":"2024-08-19T01:33:23.200809+0000","last_became_peered":"2024-08-19T01:33:23.200809+0000","last_unstale":"2024-08-19T01:35:00.789928+0000","last_undegraded":"2024-08-19T01:35:00.789928+0000","last_fullsized":"2024-08-19T01:35:00.789928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:54.901412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.884029+0000","last_change":"2024-08-19T01:33:21.172423+0000","last_active":"2024-08-19T01:34:01.884029+0000","last_peered":"2024-08-19T01:34:01.884029+0000","last_clean":"2024-08-19T01:34:01.884029+0000","last_became_active":"2024-08-19T01:33:21.171950+0000","last_became_peered":"2024-08-19T01:33:21.171950+0000","last_unstale":"2024-08-19T01:34:01.884029+0000","last_undegraded":"2024-08-19T01:34:01.884029+0000","last_fullsized":"2024-08-19T01:34:01.884029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:44:11.446875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.790970+0000","last_change":"2024-08-19T01:33:23.192933+0000","last_active":"2024-08-19T01:35:00.790970+0000","last_peered":"2024-08-19T01:35:00.790970+0000","last_clean":"2024-08-19T01:35:00.790970+0000","last_became_active":"2024-08-19T01:33:23.192601+0000","last_became_peered":"2024-08-19T01:33:23.192601+0000","last_unstale":"2024-08-19T01:35:00.790970+0000","last_undegraded":"2024-08-19T01:35:00.790970+0000","last_fullsized":"2024-08-19T01:35:00.790970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:34:09.029652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.929233+0000","last_change":"2024-08-19T01:33:21.168225+0000","last_active":"2024-08-19T01:34:01.929233+0000","last_peered":"2024-08-19T01:34:01.929233+0000","last_clean":"2024-08-19T01:34:01.929233+0000","last_became_active":"2024-08-19T01:33:21.167881+0000","last_became_peered":"2024-08-19T01:33:21.167881+0000","last_unstale":"2024-08-19T01:34:01.929233+0000","last_undegraded":"2024-08-19T01:34:01.929233+0000","last_fullsized":"2024-08-19T01:34:01.929233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:53:38.037036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793586+0000","last_change":"2024-08-19T01:33:23.194079+0000","last_active":"2024-08-19T01:35:00.793586+0000","last_peered":"2024-08-19T01:35:00.793586+0000","last_clean":"2024-08-19T01:35:00.793586+0000","last_became_active":"2024-08-19T01:33:23.193749+0000","last_became_peered":"2024-08-19T01:33:23.193749+0000","last_unstale":"2024-08-19T01:35:00.793586+0000","last_undegraded":"2024-08-19T01:35:00.793586+0000","last_fullsized":"2024-08-19T01:35:00.793586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:53:41.102163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879491+0000","last_change":"2024-08-19T01:33:21.168916+0000","last_active":"2024-08-19T01:34:01.879491+0000","last_peered":"2024-08-19T01:34:01.879491+0000","last_clean":"2024-08-19T01:34:01.879491+0000","last_became_active":"2024-08-19T01:33:21.168618+0000","last_became_peered":"2024-08-19T01:33:21.168618+0000","last_unstale":"2024-08-19T01:34:01.879491+0000","last_undegraded":"2024-08-19T01:34:01.879491+0000","last_fullsized":"2024-08-19T01:34:01.879491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:45:11.567779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.798985+0000","last_change":"2024-08-19T01:33:23.195546+0000","last_active":"2024-08-19T01:35:00.798985+0000","last_peered":"2024-08-19T01:35:00.798985+0000","last_clean":"2024-08-19T01:35:00.798985+0000","last_became_active":"2024-08-19T01:33:23.195221+0000","last_became_peered":"2024-08-19T01:33:23.195221+0000","last_unstale":"2024-08-19T01:35:00.798985+0000","last_undegraded":"2024-08-19T01:35:00.798985+0000","last_fullsized":"2024-08-19T01:35:00.798985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:46:10.671060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.123644+0000","last_change":"2024-08-19T01:33:21.287946+0000","last_active":"2024-08-19T01:35:18.123644+0000","last_peered":"2024-08-19T01:35:18.123644+0000","last_clean":"2024-08-19T01:35:18.123644+0000","last_became_active":"2024-08-19T01:33:21.287594+0000","last_became_peered":"2024-08-19T01:33:21.287594+0000","last_unstale":"2024-08-19T01:35:18.123644+0000","last_undegraded":"2024-08-19T01:35:18.123644+0000","last_fullsized":"2024-08-19T01:35:18.123644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:10:06.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1662,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.800058+0000","last_change":"2024-08-19T01:33:23.200914+0000","last_active":"2024-08-19T01:35:00.800058+0000","last_peered":"2024-08-19T01:35:00.800058+0000","last_clean":"2024-08-19T01:35:00.800058+0000","last_became_active":"2024-08-19T01:33:23.200611+0000","last_became_peered":"2024-08-19T01:33:23.200611+0000","last_unstale":"2024-08-19T01:35:00.800058+0000","last_undegraded":"2024-08-19T01:35:00.800058+0000","last_fullsized":"2024-08-19T01:35:00.800058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:22:00.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880899+0000","last_change":"2024-08-19T01:32:09.926979+0000","last_active":"2024-08-19T01:34:01.880899+0000","last_peered":"2024-08-19T01:34:01.880899+0000","last_clean":"2024-08-19T01:34:01.880899+0000","last_became_active":"2024-08-19T01:32:09.926560+0000","last_became_peered":"2024-08-19T01:32:09.926560+0000","last_unstale":"2024-08-19T01:34:01.880899+0000","last_undegraded":"2024-08-19T01:34:01.880899+0000","last_fullsized":"2024-08-19T01:34:01.880899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:29:05.356703+0000","last_clean_scrub_stamp":"2024-08-19T01:29:05.356703+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:12:23.087216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878743+0000","last_change":"2024-08-19T01:33:21.165099+0000","last_active":"2024-08-19T01:34:01.878743+0000","last_peered":"2024-08-19T01:34:01.878743+0000","last_clean":"2024-08-19T01:34:01.878743+0000","last_became_active":"2024-08-19T01:33:21.164756+0000","last_became_peered":"2024-08-19T01:33:21.164756+0000","last_unstale":"2024-08-19T01:34:01.878743+0000","last_undegraded":"2024-08-19T01:34:01.878743+0000","last_fullsized":"2024-08-19T01:34:01.878743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:11:54.480263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793548+0000","last_change":"2024-08-19T01:33:23.200185+0000","last_active":"2024-08-19T01:35:00.793548+0000","last_peered":"2024-08-19T01:35:00.793548+0000","last_clean":"2024-08-19T01:35:00.793548+0000","last_became_active":"2024-08-19T01:33:23.199885+0000","last_became_peered":"2024-08-19T01:33:23.199885+0000","last_unstale":"2024-08-19T01:35:00.793548+0000","last_undegraded":"2024-08-19T01:35:00.793548+0000","last_fullsized":"2024-08-19T01:35:00.793548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:17.200848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.880804+0000","last_change":"2024-08-19T01:33:21.172587+0000","last_active":"2024-08-19T01:34:01.880804+0000","last_peered":"2024-08-19T01:34:01.880804+0000","last_clean":"2024-08-19T01:34:01.880804+0000","last_became_active":"2024-08-19T01:33:21.172233+0000","last_became_peered":"2024-08-19T01:33:21.172233+0000","last_unstale":"2024-08-19T01:34:01.880804+0000","last_undegraded":"2024-08-19T01:34:01.880804+0000","last_fullsized":"2024-08-19T01:34:01.880804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:18:26.690403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.795949+0000","last_change":"2024-08-19T01:33:23.197272+0000","last_active":"2024-08-19T01:35:00.795949+0000","last_peered":"2024-08-19T01:35:00.795949+0000","last_clean":"2024-08-19T01:35:00.795949+0000","last_became_active":"2024-08-19T01:33:23.196869+0000","last_became_peered":"2024-08-19T01:33:23.196869+0000","last_unstale":"2024-08-19T01:35:00.795949+0000","last_undegraded":"2024-08-19T01:35:00.795949+0000","last_fullsized":"2024-08-19T01:35:00.795949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:30.020184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":136,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:19.129363+0000","last_change":"2024-08-19T01:33:21.287366+0000","last_active":"2024-08-19T01:35:19.129363+0000","last_peered":"2024-08-19T01:35:19.129363+0000","last_clean":"2024-08-19T01:35:19.129363+0000","last_became_active":"2024-08-19T01:33:21.287044+0000","last_became_peered":"2024-08-19T01:33:21.287044+0000","last_unstale":"2024-08-19T01:35:19.129363+0000","last_undegraded":"2024-08-19T01:35:19.129363+0000","last_fullsized":"2024-08-19T01:35:19.129363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:19:04.194284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792226+0000","last_change":"2024-08-19T01:34:01.885001+0000","last_active":"2024-08-19T01:35:00.792226+0000","last_peered":"2024-08-19T01:35:00.792226+0000","last_clean":"2024-08-19T01:35:00.792226+0000","last_became_active":"2024-08-19T01:34:01.884680+0000","last_became_peered":"2024-08-19T01:34:01.884680+0000","last_unstale":"2024-08-19T01:35:00.792226+0000","last_undegraded":"2024-08-19T01:35:00.792226+0000","last_fullsized":"2024-08-19T01:35:00.792226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:08.251298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"86'8","reported_seq":142,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:21.148454+0000","last_change":"2024-08-19T01:33:21.164639+0000","last_active":"2024-08-19T01:35:21.148454+0000","last_peered":"2024-08-19T01:35:21.148454+0000","last_clean":"2024-08-19T01:35:21.148454+0000","last_became_active":"2024-08-19T01:33:21.164330+0000","last_became_peered":"2024-08-19T01:33:21.164330+0000","last_unstale":"2024-08-19T01:35:21.148454+0000","last_undegraded":"2024-08-19T01:35:21.148454+0000","last_fullsized":"2024-08-19T01:35:21.148454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:26:04.035130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801157+0000","last_change":"2024-08-19T01:34:01.883666+0000","last_active":"2024-08-19T01:35:00.801157+0000","last_peered":"2024-08-19T01:35:00.801157+0000","last_clean":"2024-08-19T01:35:00.801157+0000","last_became_active":"2024-08-19T01:34:01.883264+0000","last_became_peered":"2024-08-19T01:34:01.883264+0000","last_unstale":"2024-08-19T01:35:00.801157+0000","last_undegraded":"2024-08-19T01:35:00.801157+0000","last_fullsized":"2024-08-19T01:35:00.801157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:41:52.266751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.879931+0000","last_change":"2024-08-19T01:33:21.287906+0000","last_active":"2024-08-19T01:34:01.879931+0000","last_peered":"2024-08-19T01:34:01.879931+0000","last_clean":"2024-08-19T01:34:01.879931+0000","last_became_active":"2024-08-19T01:33:21.287597+0000","last_became_peered":"2024-08-19T01:33:21.287597+0000","last_unstale":"2024-08-19T01:34:01.879931+0000","last_undegraded":"2024-08-19T01:34:01.879931+0000","last_fullsized":"2024-08-19T01:34:01.879931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:05:30.605948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801131+0000","last_change":"2024-08-19T01:33:23.191026+0000","last_active":"2024-08-19T01:35:00.801131+0000","last_peered":"2024-08-19T01:35:00.801131+0000","last_clean":"2024-08-19T01:35:00.801131+0000","last_became_active":"2024-08-19T01:33:23.190382+0000","last_became_peered":"2024-08-19T01:33:23.190382+0000","last_unstale":"2024-08-19T01:35:00.801131+0000","last_undegraded":"2024-08-19T01:35:00.801131+0000","last_fullsized":"2024-08-19T01:35:00.801131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:33:45.065441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883898+0000","last_change":"2024-08-19T01:33:21.288008+0000","last_active":"2024-08-19T01:34:01.883898+0000","last_peered":"2024-08-19T01:34:01.883898+0000","last_clean":"2024-08-19T01:34:01.883898+0000","last_became_active":"2024-08-19T01:33:21.287613+0000","last_became_peered":"2024-08-19T01:33:21.287613+0000","last_unstale":"2024-08-19T01:34:01.883898+0000","last_undegraded":"2024-08-19T01:34:01.883898+0000","last_fullsized":"2024-08-19T01:34:01.883898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:35:18.169564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.796936+0000","last_change":"2024-08-19T01:33:23.197201+0000","last_active":"2024-08-19T01:35:00.796936+0000","last_peered":"2024-08-19T01:35:00.796936+0000","last_clean":"2024-08-19T01:35:00.796936+0000","last_became_active":"2024-08-19T01:33:23.196946+0000","last_became_peered":"2024-08-19T01:33:23.196946+0000","last_unstale":"2024-08-19T01:35:00.796936+0000","last_undegraded":"2024-08-19T01:35:00.796936+0000","last_fullsized":"2024-08-19T01:35:00.796936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T13:14:31.273739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'18","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.195232+0000","last_change":"2024-08-19T01:33:21.167109+0000","last_active":"2024-08-19T01:35:18.195232+0000","last_peered":"2024-08-19T01:35:18.195232+0000","last_clean":"2024-08-19T01:35:18.195232+0000","last_became_active":"2024-08-19T01:33:21.166788+0000","last_became_peered":"2024-08-19T01:33:21.166788+0000","last_unstale":"2024-08-19T01:35:18.195232+0000","last_undegraded":"2024-08-19T01:35:18.195232+0000","last_fullsized":"2024-08-19T01:35:18.195232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:30:44.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803153+0000","last_change":"2024-08-19T01:33:23.199699+0000","last_active":"2024-08-19T01:35:00.803153+0000","last_peered":"2024-08-19T01:35:00.803153+0000","last_clean":"2024-08-19T01:35:00.803153+0000","last_became_active":"2024-08-19T01:33:23.199417+0000","last_became_peered":"2024-08-19T01:33:23.199417+0000","last_unstale":"2024-08-19T01:35:00.803153+0000","last_undegraded":"2024-08-19T01:35:00.803153+0000","last_fullsized":"2024-08-19T01:35:00.803153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:36.519135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878888+0000","last_change":"2024-08-19T01:33:21.164177+0000","last_active":"2024-08-19T01:34:01.878888+0000","last_peered":"2024-08-19T01:34:01.878888+0000","last_clean":"2024-08-19T01:34:01.878888+0000","last_became_active":"2024-08-19T01:33:21.163832+0000","last_became_peered":"2024-08-19T01:33:21.163832+0000","last_unstale":"2024-08-19T01:34:01.878888+0000","last_undegraded":"2024-08-19T01:34:01.878888+0000","last_fullsized":"2024-08-19T01:34:01.878888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:10:31.907677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804059+0000","last_change":"2024-08-19T01:33:23.195102+0000","last_active":"2024-08-19T01:35:00.804059+0000","last_peered":"2024-08-19T01:35:00.804059+0000","last_clean":"2024-08-19T01:35:00.804059+0000","last_became_active":"2024-08-19T01:33:23.194802+0000","last_became_peered":"2024-08-19T01:33:23.194802+0000","last_unstale":"2024-08-19T01:35:00.804059+0000","last_undegraded":"2024-08-19T01:35:00.804059+0000","last_fullsized":"2024-08-19T01:35:00.804059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:28:33.478676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:02.048628+0000","last_change":"2024-08-19T01:33:21.287933+0000","last_active":"2024-08-19T01:34:02.048628+0000","last_peered":"2024-08-19T01:34:02.048628+0000","last_clean":"2024-08-19T01:34:02.048628+0000","last_became_active":"2024-08-19T01:33:21.287568+0000","last_became_peered":"2024-08-19T01:33:21.287568+0000","last_unstale":"2024-08-19T01:34:02.048628+0000","last_undegraded":"2024-08-19T01:34:02.048628+0000","last_fullsized":"2024-08-19T01:34:02.048628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:23:23.221508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.780229+0000","last_change":"2024-08-19T01:33:23.194320+0000","last_active":"2024-08-19T01:35:00.780229+0000","last_peered":"2024-08-19T01:35:00.780229+0000","last_clean":"2024-08-19T01:35:00.780229+0000","last_became_active":"2024-08-19T01:33:23.194002+0000","last_became_peered":"2024-08-19T01:33:23.194002+0000","last_unstale":"2024-08-19T01:35:00.780229+0000","last_undegraded":"2024-08-19T01:35:00.780229+0000","last_fullsized":"2024-08-19T01:35:00.780229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:36:59.198997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.877938+0000","last_change":"2024-08-19T01:33:21.165421+0000","last_active":"2024-08-19T01:34:01.877938+0000","last_peered":"2024-08-19T01:34:01.877938+0000","last_clean":"2024-08-19T01:34:01.877938+0000","last_became_active":"2024-08-19T01:33:21.165013+0000","last_became_peered":"2024-08-19T01:33:21.165013+0000","last_unstale":"2024-08-19T01:34:01.877938+0000","last_undegraded":"2024-08-19T01:34:01.877938+0000","last_fullsized":"2024-08-19T01:34:01.877938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:07:18.546099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.803714+0000","last_change":"2024-08-19T01:33:23.193544+0000","last_active":"2024-08-19T01:35:00.803714+0000","last_peered":"2024-08-19T01:35:00.803714+0000","last_clean":"2024-08-19T01:35:00.803714+0000","last_became_active":"2024-08-19T01:33:23.193213+0000","last_became_peered":"2024-08-19T01:33:23.193213+0000","last_unstale":"2024-08-19T01:35:00.803714+0000","last_undegraded":"2024-08-19T01:35:00.803714+0000","last_fullsized":"2024-08-19T01:35:00.803714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:20:54.581944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:17.948248+0000","last_change":"2024-08-19T01:34:01.889627+0000","last_active":"2024-08-19T01:35:17.948248+0000","last_peered":"2024-08-19T01:35:17.948248+0000","last_clean":"2024-08-19T01:35:17.948248+0000","last_became_active":"2024-08-19T01:34:01.885181+0000","last_became_peered":"2024-08-19T01:34:01.885181+0000","last_unstale":"2024-08-19T01:35:17.948248+0000","last_undegraded":"2024-08-19T01:35:17.948248+0000","last_fullsized":"2024-08-19T01:35:17.948248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:00:51.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.801718+0000","last_change":"2024-08-19T01:33:23.196266+0000","last_active":"2024-08-19T01:35:00.801718+0000","last_peered":"2024-08-19T01:35:00.801718+0000","last_clean":"2024-08-19T01:35:00.801718+0000","last_became_active":"2024-08-19T01:33:23.194268+0000","last_became_peered":"2024-08-19T01:33:23.194268+0000","last_unstale":"2024-08-19T01:35:00.801718+0000","last_undegraded":"2024-08-19T01:35:00.801718+0000","last_fullsized":"2024-08-19T01:35:00.801718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:15:38.628746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.890210+0000","last_change":"2024-08-19T01:34:01.890210+0000","last_active":"2024-08-19T01:34:01.890210+0000","last_peered":"2024-08-19T01:34:01.890210+0000","last_clean":"2024-08-19T01:34:01.890210+0000","last_became_active":"2024-08-19T01:34:01.885955+0000","last_became_peered":"2024-08-19T01:34:01.885955+0000","last_unstale":"2024-08-19T01:34:01.890210+0000","last_undegraded":"2024-08-19T01:34:01.890210+0000","last_fullsized":"2024-08-19T01:34:01.890210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.176530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804239+0000","last_change":"2024-08-19T01:33:23.202534+0000","last_active":"2024-08-19T01:35:00.804239+0000","last_peered":"2024-08-19T01:35:00.804239+0000","last_clean":"2024-08-19T01:35:00.804239+0000","last_became_active":"2024-08-19T01:33:23.202191+0000","last_became_peered":"2024-08-19T01:33:23.202191+0000","last_unstale":"2024-08-19T01:35:00.804239+0000","last_undegraded":"2024-08-19T01:35:00.804239+0000","last_fullsized":"2024-08-19T01:35:00.804239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:33:03.478905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.797711+0000","last_change":"2024-08-19T01:33:23.201136+0000","last_active":"2024-08-19T01:35:00.797711+0000","last_peered":"2024-08-19T01:35:00.797711+0000","last_clean":"2024-08-19T01:35:00.797711+0000","last_became_active":"2024-08-19T01:33:23.200864+0000","last_became_peered":"2024-08-19T01:33:23.200864+0000","last_unstale":"2024-08-19T01:35:00.797711+0000","last_undegraded":"2024-08-19T01:35:00.797711+0000","last_fullsized":"2024-08-19T01:35:00.797711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:32:42.091880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883696+0000","last_change":"2024-08-19T01:33:21.172149+0000","last_active":"2024-08-19T01:34:01.883696+0000","last_peered":"2024-08-19T01:34:01.883696+0000","last_clean":"2024-08-19T01:34:01.883696+0000","last_became_active":"2024-08-19T01:33:21.171833+0000","last_became_peered":"2024-08-19T01:33:21.171833+0000","last_unstale":"2024-08-19T01:34:01.883696+0000","last_undegraded":"2024-08-19T01:34:01.883696+0000","last_fullsized":"2024-08-19T01:34:01.883696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:25:11.097212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.793188+0000","last_change":"2024-08-19T01:33:23.193675+0000","last_active":"2024-08-19T01:35:00.793188+0000","last_peered":"2024-08-19T01:35:00.793188+0000","last_clean":"2024-08-19T01:35:00.793188+0000","last_became_active":"2024-08-19T01:33:23.188719+0000","last_became_peered":"2024-08-19T01:33:23.188719+0000","last_unstale":"2024-08-19T01:35:00.793188+0000","last_undegraded":"2024-08-19T01:35:00.793188+0000","last_fullsized":"2024-08-19T01:35:00.793188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T12:54:12.698211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.883781+0000","last_change":"2024-08-19T01:33:21.172675+0000","last_active":"2024-08-19T01:34:01.883781+0000","last_peered":"2024-08-19T01:34:01.883781+0000","last_clean":"2024-08-19T01:34:01.883781+0000","last_became_active":"2024-08-19T01:33:21.172320+0000","last_became_peered":"2024-08-19T01:33:21.172320+0000","last_unstale":"2024-08-19T01:34:01.883781+0000","last_undegraded":"2024-08-19T01:34:01.883781+0000","last_fullsized":"2024-08-19T01:34:01.883781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:29:41.511220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.804015+0000","last_change":"2024-08-19T01:33:23.203113+0000","last_active":"2024-08-19T01:35:00.804015+0000","last_peered":"2024-08-19T01:35:00.804015+0000","last_clean":"2024-08-19T01:35:00.804015+0000","last_became_active":"2024-08-19T01:33:23.202807+0000","last_became_peered":"2024-08-19T01:33:23.202807+0000","last_unstale":"2024-08-19T01:35:00.804015+0000","last_undegraded":"2024-08-19T01:35:00.804015+0000","last_fullsized":"2024-08-19T01:35:00.804015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:24:21.570929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:34:01.878245+0000","last_change":"2024-08-19T01:33:21.287382+0000","last_active":"2024-08-19T01:34:01.878245+0000","last_peered":"2024-08-19T01:34:01.878245+0000","last_clean":"2024-08-19T01:34:01.878245+0000","last_became_active":"2024-08-19T01:33:21.287040+0000","last_became_peered":"2024-08-19T01:33:21.287040+0000","last_unstale":"2024-08-19T01:34:01.878245+0000","last_undegraded":"2024-08-19T01:34:01.878245+0000","last_fullsized":"2024-08-19T01:34:01.878245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:01:49.973969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:00.792352+0000","last_change":"2024-08-19T01:33:23.197869+0000","last_active":"2024-08-19T01:35:00.792352+0000","last_peered":"2024-08-19T01:35:00.792352+0000","last_clean":"2024-08-19T01:35:00.792352+0000","last_became_active":"2024-08-19T01:33:23.197529+0000","last_became_peered":"2024-08-19T01:33:23.197529+0000","last_unstale":"2024-08-19T01:35:00.792352+0000","last_undegraded":"2024-08-19T01:35:00.792352+0000","last_fullsized":"2024-08-19T01:35:00.792352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:22.155234+0000","last_clean_scrub_stamp":"2024-08-19T01:33:22.155234+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:03:14.401151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'15","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-19T01:35:18.143334+0000","last_change":"2024-08-19T01:33:21.172180+0000","last_active":"2024-08-19T01:35:18.143334+0000","last_peered":"2024-08-19T01:35:18.143334+0000","last_clean":"2024-08-19T01:35:18.143334+0000","last_became_active":"2024-08-19T01:33:21.171895+0000","last_became_peered":"2024-08-19T01:33:21.171895+0000","last_unstale":"2024-08-19T01:35:18.143334+0000","last_undegraded":"2024-08-19T01:35:18.143334+0000","last_fullsized":"2024-08-19T01:35:18.143334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-19T01:33:20.144370+0000","last_clean_scrub_stamp":"2024-08-19T01:33:20.144370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:51:38.650649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75654,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1113,"num_read_kb":635,"num_write":276,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":218334,"data_compressed":5490,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281644,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":454575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710762,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1111694,"data_compressed":37649,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":453901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":454167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1049475,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528380,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":454641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":1047107,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":454502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":453443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":517511,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":517023,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":454957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34183,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64244,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64732,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":68430,"data_compressed":1830,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-19T01:35:24.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:35:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:23 smithi190 ceph-mon[22032]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:23 smithi190 ceph-mon[22032]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:24.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:23 smithi146 ceph-mon[22098]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:24.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:23 smithi146 ceph-mon[22098]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:24.521 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:25.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 -- 172.21.15.103:0/3939983221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c101150 msgr2=0x7efd5c10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 --2- 172.21.15.103:0/3939983221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c101150 0x7efd5c10d680 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efd54009cd0 tx=0x7efd5402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 -- 172.21.15.103:0/3939983221 shutdown_connections 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 --2- 172.21.15.103:0/3939983221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c101150 0x7efd5c10d680 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 --2- 172.21.15.103:0/3939983221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd5c1007b0 0x7efd5c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 --2- 172.21.15.103:0/3939983221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd5c1067b0 0x7efd5c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 -- 172.21.15.103:0/3939983221 >> 172.21.15.103:0/3939983221 conn(0x7efd5c0fc4d0 msgr2=0x7efd5c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 -- 172.21.15.103:0/3939983221 shutdown_connections 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.035+0000 7efd612a0640 1 -- 172.21.15.103:0/3939983221 wait complete. 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 Processor -- start 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 -- start start 2024-08-19T01:35:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c1007b0 0x7efd5c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd5c101150 0x7efd5c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd5c1067b0 0x7efd5c1a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c110a60 con 0x7efd5c1007b0 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1108e0 con 0x7efd5c1067b0 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd612a0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c110be0 con 0x7efd5c101150 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.036+0000 7efd5affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c1007b0 0x7efd5c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c1007b0 0x7efd5c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51348/0 (socket says 172.21.15.103:51348) 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 -- 172.21.15.103:0/266950507 learned_addr learned my addr 172.21.15.103:0/266950507 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5b7fe640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd5c1067b0 0x7efd5c1a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 -- 172.21.15.103:0/266950507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd5c101150 msgr2=0x7efd5c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd5c101150 0x7efd5c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 -- 172.21.15.103:0/266950507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd5c1067b0 msgr2=0x7efd5c1a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd5c1067b0 0x7efd5c1a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 -- 172.21.15.103:0/266950507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c1a4450 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd5affd640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c1007b0 0x7efd5c198e90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d950 tx=0x7efd4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd43fff640 1 -- 172.21.15.103:0/266950507 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c017070 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c1a4620 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd43fff640 1 -- 172.21.15.103:0/266950507 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd4c009c30 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd43fff640 1 -- 172.21.15.103:0/266950507 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00f620 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.037+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1a4b60 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.038+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5c100c10 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.039+0000 7efd43fff640 1 -- 172.21.15.103:0/266950507 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efd4c0109a0 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.040+0000 7efd43fff640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efd440a2c00 0x7efd440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.040+0000 7efd43fff640 1 -- 172.21.15.103:0/266950507 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efd4c0cec50 con 0x7efd5c1007b0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.040+0000 7efd5a7fc640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efd440a2c00 0x7efd440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.040+0000 7efd5a7fc640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efd440a2c00 0x7efd440a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd50005fd0 tx=0x7efd50005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:25.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.044+0000 7efd43fff640 1 -- 172.21.15.103:0/266950507 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efd4c0d4020 con 0x7efd5c1007b0 2024-08-19T01:35:25.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.244+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd5c06aab0 con 0x7efd5c1007b0 2024-08-19T01:35:25.247 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:24 smithi103 ceph-mon[18436]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:25.247 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:24 smithi103 ceph-mon[18436]: pgmap v328: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-08-19T01:35:25.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.245+0000 7efd43fff640 1 -- 172.21.15.103:0/266950507 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7efd4c009de0 con 0x7efd5c1007b0 2024-08-19T01:35:25.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:25.248 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-08-19T01:35:19:962857+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:35:19.321960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:35:25.248 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-08-19T01:35:25.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efd440a2c00 msgr2=0x7efd440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efd440a2c00 0x7efd440a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd50005fd0 tx=0x7efd50005850 comp rx=0 tx=0).stop 2024-08-19T01:35:25.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c1007b0 msgr2=0x7efd5c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:25.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c1007b0 0x7efd5c198e90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d950 tx=0x7efd4c00de10 comp rx=0 tx=0).stop 2024-08-19T01:35:25.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 shutdown_connections 2024-08-19T01:35:25.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efd440a2c00 0x7efd440a50c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd5c1067b0 0x7efd5c1a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd5c101150 0x7efd5c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 --2- 172.21.15.103:0/266950507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c1007b0 0x7efd5c198e90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:25.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 >> 172.21.15.103:0/266950507 conn(0x7efd5c0fc4d0 msgr2=0x7efd5c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:25.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 shutdown_connections 2024-08-19T01:35:25.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:25.248+0000 7efd612a0640 1 -- 172.21.15.103:0/266950507 wait complete. 2024-08-19T01:35:25.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:24 smithi190 ceph-mon[22032]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:25.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:24 smithi190 ceph-mon[22032]: pgmap v328: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-08-19T01:35:25.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:24 smithi146 ceph-mon[22098]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-19T01:35:25.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:24 smithi146 ceph-mon[22098]: pgmap v328: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-08-19T01:35:26.091 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:35:26.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:25 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/266950507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:26.346 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:25 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/266950507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:25 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/266950507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:35:26.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 -- 172.21.15.103:0/1935781683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070100b50 msgr2=0x7f8070100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 --2- 172.21.15.103:0/1935781683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070100b50 0x7f8070100f30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8060009900 tx=0x7f806002a480 comp rx=0 tx=0).stop 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 -- 172.21.15.103:0/1935781683 shutdown_connections 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 --2- 172.21.15.103:0/1935781683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070102c20 0x7f807010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 --2- 172.21.15.103:0/1935781683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80701022a0 0x7f80701026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 --2- 172.21.15.103:0/1935781683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070100b50 0x7f8070100f30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 -- 172.21.15.103:0/1935781683 >> 172.21.15.103:0/1935781683 conn(0x7f80700fc500 msgr2=0x7f80700fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 -- 172.21.15.103:0/1935781683 shutdown_connections 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 -- 172.21.15.103:0/1935781683 wait complete. 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 Processor -- start 2024-08-19T01:35:26.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.879+0000 7f8075be8640 1 -- start start 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f8075be8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070100b50 0x7f8070198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f8075be8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80701022a0 0x7f8070199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f8075be8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070102c20 0x7f80701a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f8075be8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8070110b80 con 0x7f8070102c20 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f8075be8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8070110a00 con 0x7f8070100b50 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f8075be8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8070110d00 con 0x7f80701022a0 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070102c20 0x7f80701a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070102c20 0x7f80701a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51360/0 (socket says 172.21.15.103:51360) 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 -- 172.21.15.103:0/3707427485 learned_addr learned my addr 172.21.15.103:0/3707427485 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f8067fff640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80701022a0 0x7f8070199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806f7fe640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070100b50 0x7f8070198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:26.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 -- 172.21.15.103:0/3707427485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80701022a0 msgr2=0x7f8070199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80701022a0 0x7f8070199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 -- 172.21.15.103:0/3707427485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070100b50 msgr2=0x7f8070198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070100b50 0x7f8070198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 -- 172.21.15.103:0/3707427485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80701a4590 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.880+0000 7f806ffff640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070102c20 0x7f80701a4050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f806800b970 tx=0x7f806800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.881+0000 7f806d7fa640 1 -- 172.21.15.103:0/3707427485 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068019070 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.881+0000 7f806d7fa640 1 -- 172.21.15.103:0/3707427485 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8068014440 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.881+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80701a4760 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.881+0000 7f806d7fa640 1 -- 172.21.15.103:0/3707427485 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80680075d0 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.881+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80701a4c70 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.882+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8070100f30 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.883+0000 7f806d7fa640 1 -- 172.21.15.103:0/3707427485 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f806801c070 con 0x7f8070102c20 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.883+0000 7f806d7fa640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f805c0a2c00 0x7f805c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:26.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.884+0000 7f806f7fe640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f805c0a2c00 0x7f805c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:26.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.884+0000 7f806d7fa640 1 -- 172.21.15.103:0/3707427485 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f80680ce550 con 0x7f8070102c20 2024-08-19T01:35:26.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.884+0000 7f806f7fe640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f805c0a2c00 0x7f805c0a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8060000c00 tx=0x7f80600095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:26.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:26.888+0000 7f806d7fa640 1 -- 172.21.15.103:0/3707427485 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f806809ae90 con 0x7f8070102c20 2024-08-19T01:35:27.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:26 smithi103 ceph-mon[18436]: pgmap v329: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-19T01:35:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.060+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8070108dc0 con 0x7f8070102c20 2024-08-19T01:35:27.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.061+0000 7f806d7fa640 1 -- 172.21.15.103:0/3707427485 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22704 (secure 0 0 0) 0x7f806809f190 con 0x7f8070102c20 2024-08-19T01:35:27.063 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:35:27.064 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:34:01.874407+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:35:27.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f805c0a2c00 msgr2=0x7f805c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f805c0a2c00 0x7f805c0a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8060000c00 tx=0x7f80600095f0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070102c20 msgr2=0x7f80701a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070102c20 0x7f80701a4050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f806800b970 tx=0x7f806800be30 comp rx=0 tx=0).stop 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 shutdown_connections 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f805c0a2c00 0x7f805c0a50c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8070102c20 0x7f80701a4050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80701022a0 0x7f8070199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 --2- 172.21.15.103:0/3707427485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8070100b50 0x7f8070198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.063+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 >> 172.21.15.103:0/3707427485 conn(0x7f80700fc500 msgr2=0x7f80700fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.064+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 shutdown_connections 2024-08-19T01:35:27.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:27.064+0000 7f8075be8640 1 -- 172.21.15.103:0/3707427485 wait complete. 2024-08-19T01:35:27.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:26 smithi190 ceph-mon[22032]: pgmap v329: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-19T01:35:27.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:26 smithi146 ceph-mon[22098]: pgmap v329: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-19T01:35:27.908 DEBUG:tasks.fwd_scrub:Yielding 2024-08-19T01:35:27.908 INFO:teuthology.run_tasks:Running task background_exec... 2024-08-19T01:35:27.923 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:35:27.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-08-19T01:35:27.924 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-19T01:35:27.924 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:35:27.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:35:27.927 INFO:teuthology.run_tasks:Running task workunit... 2024-08-19T01:35:27.941 INFO:tasks.workunit:Pulling workunits from ref 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-19T01:35:27.942 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-19T01:35:27.942 INFO:tasks.workunit:timeout=3h 2024-08-19T01:35:27.942 INFO:tasks.workunit:cleanup=True 2024-08-19T01:35:27.942 DEBUG:teuthology.orchestra.run.smithi103:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:27.960 INFO:teuthology.orchestra.run.smithi103.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 79h/121d Inode: 1 Links: 2 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:fusefs_t:s0 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-08-19 01:33:38.375738077 +0000 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-08-19 01:33:38.375738077 +0000 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-08-19 01:35:18.965116510 +0000 2024-08-19T01:35:27.961 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2024-08-19T01:35:27.961 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-19T01:35:27.961 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-19T01:35:28.032 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-19T01:35:28.089 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-19T01:35:28.193 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:28.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:27 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3707427485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:27 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3707427485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:28.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:27 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3707427485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:35:28.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.736+0000 7f1c97bab640 1 -- 172.21.15.103:0/1326768857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901078b0 msgr2=0x7f1c90107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.736+0000 7f1c97bab640 1 --2- 172.21.15.103:0/1326768857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901078b0 0x7f1c90107c90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009930 tx=0x7f1c8802a420 comp rx=0 tx=0).stop 2024-08-19T01:35:28.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.736+0000 7f1c97bab640 1 -- 172.21.15.103:0/1326768857 shutdown_connections 2024-08-19T01:35:28.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.736+0000 7f1c97bab640 1 --2- 172.21.15.103:0/1326768857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c90102250 0x7f1c9010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.736+0000 7f1c97bab640 1 --2- 172.21.15.103:0/1326768857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c901018b0 0x7f1c90101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.736+0000 7f1c97bab640 1 --2- 172.21.15.103:0/1326768857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901078b0 0x7f1c90107c90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.736+0000 7f1c97bab640 1 -- 172.21.15.103:0/1326768857 >> 172.21.15.103:0/1326768857 conn(0x7f1c900fd5f0 msgr2=0x7f1c900ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 -- 172.21.15.103:0/1326768857 shutdown_connections 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 -- 172.21.15.103:0/1326768857 wait complete. 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 Processor -- start 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 -- start start 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901018b0 0x7f1c90074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c90102250 0x7f1c90074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c901078b0 0x7f1c90074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90111b80 con 0x7f1c901018b0 2024-08-19T01:35:28.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90111a00 con 0x7f1c90102250 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c97bab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90111d00 con 0x7f1c901078b0 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c95947640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901018b0 0x7f1c90074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c95947640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901018b0 0x7f1c90074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51374/0 (socket says 172.21.15.103:51374) 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.737+0000 7f1c95947640 1 -- 172.21.15.103:0/344908053 learned_addr learned my addr 172.21.15.103:0/344908053 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95146640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c90102250 0x7f1c90074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95947640 1 -- 172.21.15.103:0/344908053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c901078b0 msgr2=0x7f1c90074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95947640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c901078b0 0x7f1c90074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95947640 1 -- 172.21.15.103:0/344908053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c90102250 msgr2=0x7f1c90074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95947640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c90102250 0x7f1c90074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95947640 1 -- 172.21.15.103:0/344908053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c901a6620 con 0x7f1c901018b0 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95146640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c90102250 0x7f1c90074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c95947640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901018b0 0x7f1c90074220 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c88005940 tx=0x7f1c880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:28.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c8effd640 1 -- 172.21.15.103:0/344908053 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8803f030 con 0x7f1c901018b0 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c8effd640 1 -- 172.21.15.103:0/344908053 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c88003cb0 con 0x7f1c901018b0 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c901a68b0 con 0x7f1c901018b0 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c8effd640 1 -- 172.21.15.103:0/344908053 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88035070 con 0x7f1c901018b0 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.738+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c901a6d90 con 0x7f1c901018b0 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.740+0000 7f1c8effd640 1 -- 172.21.15.103:0/344908053 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1c88042030 con 0x7f1c901018b0 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 Processor -- start 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 -- start start 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c8effd640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1c780a2b80 0x7f1c780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a4815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a481607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561a48116e40 con 0x561a48153b20 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561a480b87f0 con 0x561a48153200 2024-08-19T01:35:28.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561a4805ffb0 con 0x561a4815e3e0 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c96148640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1c780a2b80 0x7f1c780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c8effd640 1 -- 172.21.15.103:0/344908053 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1c880ef5f0 con 0x7f1c901018b0 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c95947640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c96148640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a481607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c95146640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a4815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c95947640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54272/0 (socket says 172.21.15.103:54272) 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.741+0000 7f1c95947640 1 -- 172.21.15.103:0/3875526757 learned_addr learned my addr 172.21.15.103:0/3875526757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c95146640 1 -- 172.21.15.103:0/3875526757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 msgr2=0x561a481607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c95146640 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a481607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c95146640 1 -- 172.21.15.103:0/3875526757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 msgr2=0x561a481535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c95146640 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c95146640 1 -- 172.21.15.103:0/3875526757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a47e2a6a0 con 0x561a48153b20 2024-08-19T01:35:28.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c95947640 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c96148640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1c780a2b80 0x7f1c780a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000abe0 tx=0x7f1c8000a6d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c95146640 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a4815dea0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1c840058c0 tx=0x7f1c8400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c8a7a96fcdf388 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c77fff640 1 -- 172.21.15.103:0/3875526757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84005020 con 0x561a48153b20 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c77fff640 1 -- 172.21.15.103:0/3875526757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c840047a0 con 0x561a48153b20 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.742+0000 7f1c77fff640 1 -- 172.21.15.103:0/3875526757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84010650 con 0x561a48153b20 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3875526757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 msgr2=0x561a4815dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a4815dea0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1c840058c0 tx=0x7f1c8400e350 comp rx=0 tx=0).stop 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3875526757 shutdown_connections 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a481607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a4815dea0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3875526757 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3875526757 >> 172.21.15.103:0/3875526757 conn(0x561a4814f790 msgr2=0x561a4814fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3875526757 shutdown_connections 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3875526757 wait complete. 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 -- start start 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.743+0000 7f1c99fa9740 1 Processor -- start 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153200 0x561a4815b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153b20 0x561a4815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a4815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561a48073cb0 con 0x561a48153200 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561a480b47b0 con 0x561a48153b20 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561a48160fb0 con 0x561a4815e3e0 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 10 client.0 _ll_register_callbacks cb 0x561a47dc7ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153200 0x561a4815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153200 0x561a4815b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51392/0 (socket says 172.21.15.103:51392) 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 -- 172.21.15.103:0/1308289979 learned_addr learned my addr 172.21.15.103:0/1308289979 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c96148640 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a4815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 -- 172.21.15.103:0/1308289979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 msgr2=0x561a4815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a4815bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 -- 172.21.15.103:0/1308289979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153b20 msgr2=0x561a4815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153b20 0x561a4815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 -- 172.21.15.103:0/1308289979 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a48072bc0 con 0x561a48153200 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c95947640 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153200 0x561a4815b260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1c88000c00 tx=0x7f1c880bb8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3cabe1cb806e0c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c757fa640 1 -- 172.21.15.103:0/1308289979 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88049040 con 0x561a48153200 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c757fa640 1 -- 172.21.15.103:0/1308289979 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1c880b84f0 con 0x561a48153200 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.744+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c840047a0 con 0x561a48153200 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c757fa640 1 -- 172.21.15.103:0/1308289979 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c88003e60 con 0x561a48153200 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:28.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a4816ce00 con 0x561a48153200 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c757fa640 1 -- 172.21.15.103:0/1308289979 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c840047a0 con 0x561a48153200 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c757fa640 1 -- 172.21.15.103:0/1308289979 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c8803b460 con 0x561a48153200 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 10 client.15270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 20 client.15270 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a4816e2b0 0x561a48170770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 4 client.15270 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a480c5d60 con 0x561a4816e2b0 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.745+0000 7f1c95146640 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a4816e2b0 0x561a48170770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.746+0000 7f1c757fa640 10 client.15270 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.746+0000 7f1c95146640 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a4816e2b0 0x561a48170770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82efc31c8a284d97 server_cookie=4b755503da6cf328 in_seq=0 out_seq=0 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.746+0000 7f1c757fa640 1 -- 172.21.15.103:0/1308289979 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a480c5d60 con 0x561a4816e2b0 2024-08-19T01:35:28.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.746+0000 7f1c757fa640 10 client.15270 handle_command_reply: tid=0 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.746+0000 7f1c99fa9740 1 client.15270 shutdown 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153200 msgr2=0x561a4815b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153200 0x561a4815b260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1c88000c00 tx=0x7f1c880bb8a0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 shutdown_connections 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a4816e2b0 0x561a48170770 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a4815e3e0 0x561a4815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153b20 0x561a4815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/1308289979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153200 0x561a4815b260 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c757fa640 0 client.15270 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 >> 172.21.15.103:0/1308289979 conn(0x561a4814f790 msgr2=0x561a481554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 shutdown_connections 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 1 -- 172.21.15.103:0/1308289979 wait complete. 2024-08-19T01:35:28.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.747+0000 7f1c99fa9740 20 client.15270 trim_cache size 0 max 16384 2024-08-19T01:35:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.764+0000 7f1c99fa9740 1 Processor -- start 2024-08-19T01:35:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.764+0000 7f1c99fa9740 1 -- start start 2024-08-19T01:35:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153200 0x561a4815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a48176040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4815e3e0 0x561a48176580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901af2b0 con 0x561a48153b20 2024-08-19T01:35:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b0df0 con 0x561a4815e3e0 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901af980 con 0x561a48153200 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a48176040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a48176040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51404/0 (socket says 172.21.15.103:51404) 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 -- 172.21.15.103:0/3806948261 learned_addr learned my addr 172.21.15.103:0/3806948261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95947640 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153200 0x561a4815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 -- 172.21.15.103:0/3806948261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153200 msgr2=0x561a4815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153200 0x561a4815d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c96148640 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4815e3e0 0x561a48176580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 -- 172.21.15.103:0/3806948261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4815e3e0 msgr2=0x561a48176580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4815e3e0 0x561a48176580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 -- 172.21.15.103:0/3806948261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a48150420 con 0x561a48153b20 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95947640 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153200 0x561a4815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c96148640 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4815e3e0 0x561a48176580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c95146640 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a48176040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c7d0 tx=0x7f1c84004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e526b8010e0462 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c777fe640 1 -- 172.21.15.103:0/3806948261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840049a0 con 0x561a48153b20 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c777fe640 1 -- 172.21.15.103:0/3806948261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c84005480 con 0x561a48153b20 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.765+0000 7f1c777fe640 1 -- 172.21.15.103:0/3806948261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8401a7c0 con 0x561a48153b20 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3806948261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 msgr2=0x561a48176040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a48176040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c7d0 tx=0x7f1c84004660 comp rx=0 tx=0).stop 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3806948261 shutdown_connections 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4815e3e0 0x561a48176580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a48153b20 0x561a48176040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/3806948261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153200 0x561a4815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3806948261 >> 172.21.15.103:0/3806948261 conn(0x561a4814f790 msgr2=0x561a4815cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3806948261 shutdown_connections 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 -- 172.21.15.103:0/3806948261 wait complete. 2024-08-19T01:35:28.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.766+0000 7f1c99fa9740 1 -- start start 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 1 Processor -- start 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153b20 0x561a481586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a4815e3e0 0x561a48158c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b0720 con 0x561a4815e3e0 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b0050 con 0x561a48153200 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901ad2b0 con 0x561a48153b20 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c99fa9740 10 client.0 _ll_register_callbacks cb 0x7f1c88009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a4815e3e0 0x561a48158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a4815e3e0 0x561a48158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51414/0 (socket says 172.21.15.103:51414) 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 -- 172.21.15.103:0/4292060932 learned_addr learned my addr 172.21.15.103:0/4292060932 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c95947640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c95146640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153b20 0x561a481586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 -- 172.21.15.103:0/4292060932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153b20 msgr2=0x561a481586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153b20 0x561a481586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 -- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 msgr2=0x561a481581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c96148640 1 -- 172.21.15.103:0/4292060932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a481d8010 con 0x561a4815e3e0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.767+0000 7f1c95146640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153b20 0x561a481586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c96148640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a4815e3e0 0x561a48158c30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1c80004040 tx=0x7f1c80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c67e3e3dfd9347 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c757fa640 1 -- 172.21.15.103:0/4292060932 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80029060 con 0x561a4815e3e0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c757fa640 1 -- 172.21.15.103:0/4292060932 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1c8001b7c0 con 0x561a4815e3e0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c84005480 con 0x561a4815e3e0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c757fa640 1 -- 172.21.15.103:0/4292060932 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1c8001b9c0 con 0x561a4815e3e0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a481d9200 con 0x561a4815e3e0 2024-08-19T01:35:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c757fa640 1 -- 172.21.15.103:0/4292060932 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c84005480 con 0x561a4815e3e0 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c757fa640 1 -- 172.21.15.103:0/4292060932 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1c8001a070 con 0x561a4815e3e0 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 10 client.15276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 20 client.15276 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a481d99b0 0x561a481dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 4 client.15276 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.768+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a480c0270 con 0x561a481d99b0 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.769+0000 7f1c95947640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a481d99b0 0x561a481dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.769+0000 7f1c95947640 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a481d99b0 0x561a481dbdd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a207543697c8b9 server_cookie=8be3617bca9ebb3b in_seq=0 out_seq=0 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.769+0000 7f1c757fa640 10 client.15276 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c757fa640 1 -- 172.21.15.103:0/4292060932 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a480c0270 con 0x561a481d99b0 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c757fa640 10 client.15276 handle_command_reply: tid=0 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 client.15276 shutdown 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a4815e3e0 msgr2=0x561a48158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a4815e3e0 0x561a48158c30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1c80004040 tx=0x7f1c80016000 comp rx=0 tx=0).stop 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 shutdown_connections 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561a481d99b0 0x561a481dbdd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a4815e3e0 0x561a48158c30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561a48153b20 0x561a481586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c757fa640 0 client.15276 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 --2- 172.21.15.103:0/4292060932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a48153200 0x561a481581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.770+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 >> 172.21.15.103:0/4292060932 conn(0x561a4814f790 msgr2=0x561a4815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 shutdown_connections 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c99fa9740 1 -- 172.21.15.103:0/4292060932 wait complete. 2024-08-19T01:35:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c99fa9740 20 client.15276 trim_cache size 0 max 16384 2024-08-19T01:35:28.784 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:28.784 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:35:28.784 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "a0896a22-e447-4477-adb4-2350366b9c93", 2024-08-19T01:35:28.784 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:35:28.784 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1c780a2b80 msgr2=0x7f1c780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1c780a2b80 0x7f1c780a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000abe0 tx=0x7f1c8000a6d0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901018b0 msgr2=0x7f1c90074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901018b0 0x7f1c90074220 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c88005940 tx=0x7f1c880042f0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 shutdown_connections 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1c780a2b80 0x7f1c780a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c901078b0 0x7f1c90074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c90102250 0x7f1c90074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 --2- 172.21.15.103:0/344908053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c901018b0 0x7f1c90074220 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 >> 172.21.15.103:0/344908053 conn(0x7f1c900fd5f0 msgr2=0x7f1c900fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:28.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 shutdown_connections 2024-08-19T01:35:28.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:28.771+0000 7f1c97bab640 1 -- 172.21.15.103:0/344908053 wait complete. 2024-08-19T01:35:29.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:29 smithi103 ceph-mon[18436]: pgmap v330: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-08-19T01:35:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:29 smithi190 ceph-mon[22032]: pgmap v330: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-08-19T01:35:29.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:29 smithi146 ceph-mon[22098]: pgmap v330: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-08-19T01:35:29.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:35:29.834 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:35:30.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:30 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:35:30.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:30 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:30.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:30 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:35:30.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:30 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 -- 172.21.15.103:0/720102293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 msgr2=0x7f1408102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 --2- 172.21.15.103:0/720102293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408102e40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1404009930 tx=0x7f140402a420 comp rx=0 tx=0).stop 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 -- 172.21.15.103:0/720102293 shutdown_connections 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 --2- 172.21.15.103:0/720102293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1408107fd0 0x7f140810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 --2- 172.21.15.103:0/720102293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1408103410 0x7f14081078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 --2- 172.21.15.103:0/720102293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408102e40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 -- 172.21.15.103:0/720102293 >> 172.21.15.103:0/720102293 conn(0x7f14080fc500 msgr2=0x7f14080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 -- 172.21.15.103:0/720102293 shutdown_connections 2024-08-19T01:35:30.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 -- 172.21.15.103:0/720102293 wait complete. 2024-08-19T01:35:30.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 Processor -- start 2024-08-19T01:35:30.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.330+0000 7f140f84b640 1 -- start start 2024-08-19T01:35:30.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140f84b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140f84b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1408103410 0x7f1408196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140f84b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1408107fd0 0x7f140819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140f84b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f140810fa90 con 0x7f1408102a60 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140f84b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f140810f910 con 0x7f1408107fd0 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140f84b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f140810fc10 con 0x7f1408103410 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51444/0 (socket says 172.21.15.103:51444) 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 -- 172.21.15.103:0/2872518395 learned_addr learned my addr 172.21.15.103:0/2872518395 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140cde6640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1408103410 0x7f1408196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140dde8640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1408107fd0 0x7f140819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 -- 172.21.15.103:0/2872518395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1408103410 msgr2=0x7f1408196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1408103410 0x7f1408196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 -- 172.21.15.103:0/2872518395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1408107fd0 msgr2=0x7f140819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1408107fd0 0x7f140819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 -- 172.21.15.103:0/2872518395 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140819afc0 con 0x7f1408102a60 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.331+0000 7f140d5e7640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408196010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1404005940 tx=0x7f14040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.332+0000 7f13fe7fc640 1 -- 172.21.15.103:0/2872518395 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140403f030 con 0x7f1408102a60 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.332+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140819b250 con 0x7f1408102a60 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.332+0000 7f13fe7fc640 1 -- 172.21.15.103:0/2872518395 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14040044c0 con 0x7f1408102a60 2024-08-19T01:35:30.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.332+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14081a2b90 con 0x7f1408102a60 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.332+0000 7f13fe7fc640 1 -- 172.21.15.103:0/2872518395 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404035070 con 0x7f1408102a60 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.334+0000 7f13fe7fc640 1 -- 172.21.15.103:0/2872518395 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1404042030 con 0x7f1408102a60 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 Processor -- start 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 -- start start 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c062eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c0657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56235bfbd980 con 0x56235c058b30 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56235bf55a70 con 0x56235c0633f0 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f1411c49740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56235bfb9940 con 0x56235c058210 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f140d5e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c062eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f140d5e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c062eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51460/0 (socket says 172.21.15.103:51460) 2024-08-19T01:35:30.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f140d5e7640 1 -- 172.21.15.103:0/1568875928 learned_addr learned my addr 172.21.15.103:0/1568875928 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f140dde8640 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f140cde6640 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c0657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.335+0000 7f13fe7fc640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13f40a2c00 0x7f13f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f140cde6640 1 -- 172.21.15.103:0/1568875928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 msgr2=0x56235c0585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f140cde6640 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f140cde6640 1 -- 172.21.15.103:0/1568875928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 msgr2=0x56235c062eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f140cde6640 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c062eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f140cde6640 1 -- 172.21.15.103:0/1568875928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235bfb86e0 con 0x56235c0633f0 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f140dde8640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13f40a2c00 0x7f13f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f13fe7fc640 1 -- 172.21.15.103:0/2872518395 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f14040ef140 con 0x7f1408102a60 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f140cde6640 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c0657e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f140806a0a0 tx=0x7f13f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee71e97bb76b76eb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f13f37fe640 1 -- 172.21.15.103:0/1568875928 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8004050 con 0x56235c0633f0 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f13f37fe640 1 -- 172.21.15.103:0/1568875928 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f80054d0 con 0x56235c0633f0 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.336+0000 7f13f37fe640 1 -- 172.21.15.103:0/1568875928 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8004cd0 con 0x56235c0633f0 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 -- 172.21.15.103:0/1568875928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 msgr2=0x56235c0657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c0657e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f140806a0a0 tx=0x7f13f8011810 comp rx=0 tx=0).stop 2024-08-19T01:35:30.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f140dde8640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13f40a2c00 0x7f13f40a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1400005c40 tx=0x7f1400004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 -- 172.21.15.103:0/1568875928 shutdown_connections 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c0657e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c062eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 --2- 172.21.15.103:0/1568875928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 -- 172.21.15.103:0/1568875928 >> 172.21.15.103:0/1568875928 conn(0x56235c0547a0 msgr2=0x56235c054b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 -- 172.21.15.103:0/1568875928 shutdown_connections 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 -- 172.21.15.103:0/1568875928 wait complete. 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 -- start start 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.337+0000 7f1411c49740 1 Processor -- start 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f1411c49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c060230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f1411c49740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058b30 0x56235c060770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f1411c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c0633f0 0x56235c060cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f1411c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56235c065fc0 con 0x56235c0633f0 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f1411c49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56235bf78aa0 con 0x56235c058210 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f1411c49740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56235c066140 con 0x56235c058b30 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f1411c49740 10 client.0 _ll_register_callbacks cb 0x56235bf67e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140dde8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c0633f0 0x56235c060cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140d5e7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c060230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140cde6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058b30 0x56235c060770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140d5e7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c060230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54350/0 (socket says 172.21.15.103:54350) 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140d5e7640 1 -- 172.21.15.103:0/1035081202 learned_addr learned my addr 172.21.15.103:0/1035081202 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140cde6640 1 -- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 msgr2=0x56235c060230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140cde6640 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c060230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140cde6640 1 -- 172.21.15.103:0/1035081202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c0633f0 msgr2=0x56235c060cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.338+0000 7f140cde6640 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c0633f0 0x56235c060cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f140cde6640 1 -- 172.21.15.103:0/1035081202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235bf5c650 con 0x56235c058b30 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f140d5e7640 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c060230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f140dde8640 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c0633f0 0x56235c060cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f140cde6640 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058b30 0x56235c060770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f140806a0a0 tx=0x7f13f8005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7f6a49a43908bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f13f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:35:30.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1035081202 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8028070 con 0x56235c058b30 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1035081202 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f13f80037f0 con 0x56235c058b30 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f80054d0 con 0x56235c058b30 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1035081202 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f13f8034050 con 0x56235c058b30 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f1411c49740 10 client.24889 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.339+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56235bd2f3c0 con 0x56235c058b30 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1035081202 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f80054d0 con 0x56235c058b30 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1035081202 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f13f8004e50 con 0x56235c058b30 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f1411c49740 10 client.24889 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f1411c49740 10 client.24889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f1411c49740 20 client.24889 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f1411c49740 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c072da0 0x56235c075190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f1411c49740 4 client.24889 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:30.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56235bfcaef0 con 0x56235c072da0 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.340+0000 7f140d5e7640 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c072da0 0x56235c075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.341+0000 7f13f0ff9640 10 client.24889 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.341+0000 7f140d5e7640 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c072da0 0x56235c075190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729cd4bbe2337027 server_cookie=43ff6ce05c8d6518 in_seq=0 out_seq=0 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.341+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1035081202 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56235bfcaef0 con 0x56235c072da0 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.341+0000 7f13f0ff9640 10 client.24889 handle_command_reply: tid=0 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.341+0000 7f1411c49740 1 client.24889 shutdown 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058b30 msgr2=0x56235c060770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058b30 0x56235c060770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f140806a0a0 tx=0x7f13f8005db0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 shutdown_connections 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c072da0 0x56235c075190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c0633f0 0x56235c060cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058b30 0x56235c060770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 --2- 172.21.15.103:0/1035081202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c060230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f13f0ff9640 0 client.24889 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:30.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 >> 172.21.15.103:0/1035081202 conn(0x56235c0547a0 msgr2=0x56235c05a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 shutdown_connections 2024-08-19T01:35:30.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 1 -- 172.21.15.103:0/1035081202 wait complete. 2024-08-19T01:35:30.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.342+0000 7f1411c49740 20 client.24889 trim_cache size 0 max 16384 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 Processor -- start 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 -- start start 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c07a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c0633f0 0x56235c0530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00e380 con 0x56235c058b30 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00fec0 con 0x56235c058210 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.368+0000 7f1411c49740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00ea50 con 0x56235c0633f0 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c07aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51492/0 (socket says 172.21.15.103:51492) 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 -- 172.21.15.103:0/3391261188 learned_addr learned my addr 172.21.15.103:0/3391261188 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:30.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140d5e7640 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c07a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140dde8640 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c0633f0 0x56235c0530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 -- 172.21.15.103:0/3391261188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c0633f0 msgr2=0x56235c0530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c0633f0 0x56235c0530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 -- 172.21.15.103:0/3391261188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 msgr2=0x56235c07a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c07a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 -- 172.21.15.103:0/3391261188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235c058960 con 0x56235c058b30 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f140cde6640 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c07aec0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f13f80235d0 tx=0x7f13f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fbd7e0f2479fdc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f13f2ffd640 1 -- 172.21.15.103:0/3391261188 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f803d070 con 0x56235c058b30 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f13f2ffd640 1 -- 172.21.15.103:0/3391261188 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f8023650 con 0x56235c058b30 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.369+0000 7f13f2ffd640 1 -- 172.21.15.103:0/3391261188 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8028050 con 0x56235c058b30 2024-08-19T01:35:30.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 -- 172.21.15.103:0/3391261188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 msgr2=0x56235c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c07aec0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f13f80235d0 tx=0x7f13f8011810 comp rx=0 tx=0).stop 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 -- 172.21.15.103:0/3391261188 shutdown_connections 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c0633f0 0x56235c0530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c07aec0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 --2- 172.21.15.103:0/3391261188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c058210 0x56235c07a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 -- 172.21.15.103:0/3391261188 >> 172.21.15.103:0/3391261188 conn(0x56235c0547a0 msgr2=0x56235c05eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 -- 172.21.15.103:0/3391261188 shutdown_connections 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 -- 172.21.15.103:0/3391261188 wait complete. 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.370+0000 7f1411c49740 1 -- start start 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 1 Processor -- start 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c066920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c066e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00f7f0 con 0x56235c058b30 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00f120 con 0x56235c0633f0 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00c360 con 0x56235c058210 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f1411c49740 10 client.0 _ll_register_callbacks cb 0x7f14040f2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c066920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c066920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51498/0 (socket says 172.21.15.103:51498) 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 -- 172.21.15.103:0/1253765089 learned_addr learned my addr 172.21.15.103:0/1253765089 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140d5e7640 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140dde8640 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c066e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 -- 172.21.15.103:0/1253765089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 msgr2=0x56235c0663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0663e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 -- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 msgr2=0x56235c066e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c066e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.371+0000 7f140cde6640 1 -- 172.21.15.103:0/1253765089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235c0db940 con 0x56235c058b30 2024-08-19T01:35:30.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f140cde6640 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c066920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af639ee0ad34f2b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f13f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1253765089 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8046060 con 0x56235c058b30 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1253765089 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f13f80056d0 con 0x56235c058b30 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56235bfc5400 con 0x56235c058b30 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1253765089 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f13f8028070 con 0x56235c058b30 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f1411c49740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56235c0dcb90 con 0x56235c058b30 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1253765089 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56235bfc5400 con 0x56235c058b30 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1253765089 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f13f8040030 con 0x56235c058b30 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f1411c49740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f1411c49740 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:30.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f1411c49740 20 client.15294 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:30.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.372+0000 7f1411c49740 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c0dd240 0x56235c0df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:30.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f1411c49740 4 client.15294 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:35:30.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56235bf42210 con 0x56235c0dd240 2024-08-19T01:35:30.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f140d5e7640 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c0dd240 0x56235c0df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:30.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f140d5e7640 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c0dd240 0x56235c0df660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d642b21d9070014b server_cookie=499a4839b98f9f37 in_seq=0 out_seq=0 2024-08-19T01:35:30.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f13f0ff9640 10 client.15294 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f13f0ff9640 1 -- 172.21.15.103:0/1253765089 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56235bf42210 con 0x56235c0dd240 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f13f0ff9640 10 client.15294 handle_command_reply: tid=0 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f1411c49740 1 client.15294 shutdown 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 msgr2=0x56235c066920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f1411c49740 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c066920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f80096c0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.373+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 shutdown_connections 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56235c0dd240 0x56235c0df660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56235c0633f0 0x56235c066e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56235c058b30 0x56235c066920 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f13f0ff9640 0 client.15294 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:30.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 1 --2- 172.21.15.103:0/1253765089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56235c058210 0x56235c0663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 >> 172.21.15.103:0/1253765089 conn(0x56235c0547a0 msgr2=0x56235c0530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 shutdown_connections 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 1 -- 172.21.15.103:0/1253765089 wait complete. 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f1411c49740 20 client.15294 trim_cache size 0 max 16384 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stdout: "a0896a22-e447-4477-adb4-2350366b9c93": { 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "a0896a22-e447-4477-adb4-2350366b9c93", 2024-08-19T01:35:30.384 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:35:30.385 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:35:30.385 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:35:30.385 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:35:30.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13f40a2c00 msgr2=0x7f13f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13f40a2c00 0x7f13f40a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1400005c40 tx=0x7f1400004a40 comp rx=0 tx=0).stop 2024-08-19T01:35:30.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 msgr2=0x7f1408196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:30.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408196010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1404005940 tx=0x7f14040040c0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 shutdown_connections 2024-08-19T01:35:30.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f13f40a2c00 0x7f13f40a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1408107fd0 0x7f140819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1408103410 0x7f1408196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 --2- 172.21.15.103:0/2872518395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408102a60 0x7f1408196010 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:30.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.374+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 >> 172.21.15.103:0/2872518395 conn(0x7f14080fc500 msgr2=0x7f1408105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:30.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.375+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 shutdown_connections 2024-08-19T01:35:30.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:35:30.375+0000 7f140f84b640 1 -- 172.21.15.103:0/2872518395 wait complete. 2024-08-19T01:35:30.387 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:30 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:35:30.387 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:30 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:30.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:30 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:35:30.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:30 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:30.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:30 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:35:30.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:30 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:30.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:30 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:35:30.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:30 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:35:31.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:31 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:35:31.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:31 smithi146 ceph-mon[22098]: pgmap v331: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-08-19T01:35:31.225 INFO:tasks.cephfs.filesystem:scrub status for tag:a0896a22-e447-4477-adb4-2350366b9c93 - {'path': '/', 'tag': 'a0896a22-e447-4477-adb4-2350366b9c93', 'options': 'recursive,force'} 2024-08-19T01:35:31.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:31 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:35:31.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:31 smithi103 ceph-mon[18436]: pgmap v331: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-08-19T01:35:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:31 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:35:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:31 smithi190 ceph-mon[22032]: pgmap v331: 97 pgs: 97 active+clean; 651 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-08-19T01:35:33.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:32 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:32 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:33.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:32 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:34.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:33 smithi103 ceph-mon[18436]: pgmap v332: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-08-19T01:35:34.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:35:34.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:35:34.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:34.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:33 smithi190 ceph-mon[22032]: pgmap v332: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-08-19T01:35:34.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:35:34.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:35:34.186 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:33 smithi146 ceph-mon[22098]: pgmap v332: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-08-19T01:35:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:35:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:35:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:35:36.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:35 smithi190 ceph-mon[22032]: pgmap v333: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-19T01:35:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:35 smithi146 ceph-mon[22098]: pgmap v333: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-19T01:35:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:35 smithi103 ceph-mon[18436]: pgmap v333: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-19T01:35:38.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:37 smithi103 ceph-mon[18436]: pgmap v334: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-08-19T01:35:38.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:37 smithi190 ceph-mon[22032]: pgmap v334: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-08-19T01:35:38.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:37 smithi146 ceph-mon[22098]: pgmap v334: 97 pgs: 97 active+clean; 657 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-08-19T01:35:40.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:39 smithi190 ceph-mon[22032]: pgmap v335: 97 pgs: 97 active+clean; 662 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-19T01:35:40.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:39 smithi146 ceph-mon[22098]: pgmap v335: 97 pgs: 97 active+clean; 662 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-19T01:35:40.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:39 smithi103 ceph-mon[18436]: pgmap v335: 97 pgs: 97 active+clean; 662 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 -- 172.21.15.103:0/344377715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 msgr2=0x7f32fc101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 --2- 172.21.15.103:0/344377715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc101c70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009900 tx=0x7f32f402a410 comp rx=0 tx=0).stop 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 -- 172.21.15.103:0/344377715 shutdown_connections 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 --2- 172.21.15.103:0/344377715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32fc103420 0x7f32fc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 --2- 172.21.15.103:0/344377715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32fc102a60 0x7f32fc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 --2- 172.21.15.103:0/344377715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc101c70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 -- 172.21.15.103:0/344377715 >> 172.21.15.103:0/344377715 conn(0x7f32fc0fd010 msgr2=0x7f32fc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 -- 172.21.15.103:0/344377715 shutdown_connections 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 -- 172.21.15.103:0/344377715 wait complete. 2024-08-19T01:35:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 Processor -- start 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.071+0000 7f33037cf640 1 -- start start 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32fc102a60 0x7f32fc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32fc103420 0x7f32fc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc10d3a0 con 0x7f32fc101870 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc10d220 con 0x7f32fc103420 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc10d520 con 0x7f32fc102a60 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59186/0 (socket says 172.21.15.103:59186) 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 -- 172.21.15.103:0/1119564235 learned_addr learned my addr 172.21.15.103:0/1119564235 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f3301d6c640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32fc103420 0x7f32fc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f3300d6a640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32fc102a60 0x7f32fc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 -- 172.21.15.103:0/1119564235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32fc102a60 msgr2=0x7f32fc197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32fc102a60 0x7f32fc197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 -- 172.21.15.103:0/1119564235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32fc103420 msgr2=0x7f32fc19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32fc103420 0x7f32fc19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 -- 172.21.15.103:0/1119564235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc19f670 con 0x7f32fc101870 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f330156b640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc197a50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32f40314e0 tx=0x7f32f4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f32fa7fc640 1 -- 172.21.15.103:0/1119564235 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f403f030 con 0x7f32fc101870 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc19f900 con 0x7f32fc101870 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f32fa7fc640 1 -- 172.21.15.103:0/1119564235 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f40044b0 con 0x7f32fc101870 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f32fa7fc640 1 -- 172.21.15.103:0/1119564235 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4035070 con 0x7f32fc101870 2024-08-19T01:35:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.072+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc19fdf0 con 0x7f32fc101870 2024-08-19T01:35:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f32fa7fc640 1 -- 172.21.15.103:0/1119564235 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f32f4042030 con 0x7f32fc101870 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 Processor -- start 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 -- start start 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6fea90 0x556baf6fee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6ff3d0 0x556baf703e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf704460 0x556baf706920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556baf65d9d0 con 0x556baf6ff3d0 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556baf605820 con 0x556baf6fea90 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.075+0000 7f3306664740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556baf659990 con 0x556baf704460 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6ff3d0 0x556baf703e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6ff3d0 0x556baf703e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59190/0 (socket says 172.21.15.103:59190) 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 -- 172.21.15.103:0/1975127912 learned_addr learned my addr 172.21.15.103:0/1975127912 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f3301d6c640 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6fea90 0x556baf6fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f32fa7fc640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32e40a2c30 0x7f32e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 -- 172.21.15.103:0/1975127912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf704460 msgr2=0x556baf706920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf704460 0x556baf706920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 -- 172.21.15.103:0/1975127912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6fea90 msgr2=0x556baf6fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6fea90 0x556baf6fee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 -- 172.21.15.103:0/1975127912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556baf371210 con 0x556baf6ff3d0 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f3301d6c640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32e40a2c30 0x7f32e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f330156b640 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6ff3d0 0x556baf703e90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f32f40dd7e0 tx=0x7f32f40b8160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e70b59b9985d1a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f32fa7fc640 1 -- 172.21.15.103:0/1119564235 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f32f40ef630 con 0x7f32fc101870 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f32e37fe640 1 -- 172.21.15.103:0/1975127912 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4049030 con 0x556baf6ff3d0 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f32e37fe640 1 -- 172.21.15.103:0/1975127912 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f40dd810 con 0x556baf6ff3d0 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.076+0000 7f32e37fe640 1 -- 172.21.15.103:0/1975127912 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40bb820 con 0x556baf6ff3d0 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 -- 172.21.15.103:0/1975127912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6ff3d0 msgr2=0x556baf703e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3301d6c640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32e40a2c30 0x7f32e40a50f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32ec005c40 tx=0x7f32ec004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6ff3d0 0x556baf703e90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f32f40dd7e0 tx=0x7f32f40b8160 comp rx=0 tx=0).stop 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 -- 172.21.15.103:0/1975127912 shutdown_connections 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf704460 0x556baf706920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6ff3d0 0x556baf703e90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 --2- 172.21.15.103:0/1975127912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6fea90 0x556baf6fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 -- 172.21.15.103:0/1975127912 >> 172.21.15.103:0/1975127912 conn(0x556baf6f5cf0 msgr2=0x556baf6f60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 -- 172.21.15.103:0/1975127912 shutdown_connections 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.077+0000 7f3306664740 1 -- 172.21.15.103:0/1975127912 wait complete. 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 -- start start 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 Processor -- start 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf7089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf708f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf70ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556baf7071f0 con 0x556baf6fea90 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556baf619520 con 0x556baf704460 2024-08-19T01:35:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556baf707370 con 0x556baf6ff3d0 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3306664740 10 client.0 _ll_register_callbacks cb 0x556baf370a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf7089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf7089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59204/0 (socket says 172.21.15.103:59204) 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f330156b640 1 -- 172.21.15.103:0/3183720569 learned_addr learned my addr 172.21.15.103:0/3183720569 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3301d6c640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf70ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.078+0000 7f3300d6a640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf708f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f330156b640 1 -- 172.21.15.103:0/3183720569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 msgr2=0x556baf708f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f330156b640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf708f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f330156b640 1 -- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 msgr2=0x556baf70ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f330156b640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf70ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f330156b640 1 -- 172.21.15.103:0/3183720569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556baf43d620 con 0x556baf6fea90 2024-08-19T01:35:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f3301d6c640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf70ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f330156b640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf7089f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f32f40ed1b0 tx=0x7f32f402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d502cd7e03a379 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f32e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f32e0ff9640 1 -- 172.21.15.103:0/3183720569 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4049040 con 0x556baf6fea90 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f32e0ff9640 1 -- 172.21.15.103:0/3183720569 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f32f40b7c10 con 0x556baf6fea90 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556baf66b100 con 0x556baf6fea90 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f32e0ff9640 1 -- 172.21.15.103:0/3183720569 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32f40c3830 con 0x556baf6fea90 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f3306664740 10 client.15306 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556baf7071f0 con 0x556baf6fea90 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f32e0ff9640 1 -- 172.21.15.103:0/3183720569 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556baf66b100 con 0x556baf6fea90 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.079+0000 7f32e0ff9640 1 -- 172.21.15.103:0/3183720569 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f32f40c2a60 con 0x556baf6fea90 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3306664740 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3306664740 10 client.15306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3306664740 20 client.15306 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3306664740 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf714220 0x556baf716610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3306664740 4 client.15306 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556baf665450 con 0x556baf714220 2024-08-19T01:35:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3300d6a640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf714220 0x556baf716610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f32e0ff9640 10 client.15306 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.080+0000 7f3300d6a640 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf714220 0x556baf716610 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8408296457e5f6 server_cookie=b675088ea1382d09 in_seq=0 out_seq=0 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f32e0ff9640 1 -- 172.21.15.103:0/3183720569 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556baf665450 con 0x556baf714220 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f32e0ff9640 10 client.15306 handle_command_reply: tid=0 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 client.15306 shutdown 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 msgr2=0x556baf7089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf7089f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f32f40ed1b0 tx=0x7f32f402a410 comp rx=0 tx=0).stop 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 shutdown_connections 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf714220 0x556baf716610 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf70ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f32e0ff9640 0 client.15306 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf708f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 --2- 172.21.15.103:0/3183720569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf7089f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 >> 172.21.15.103:0/3183720569 conn(0x556baf6f5cf0 msgr2=0x556baf6f7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 shutdown_connections 2024-08-19T01:35:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.081+0000 7f3306664740 1 -- 172.21.15.103:0/3183720569 wait complete. 2024-08-19T01:35:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.082+0000 7f3306664740 20 client.15306 trim_cache size 0 max 16384 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 Processor -- start 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 -- start start 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf6fee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf70ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf6f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1a6970 con 0x556baf6fea90 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1a84b0 con 0x556baf704460 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f3306664740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1a7040 con 0x556baf6ff3d0 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf6fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f330156b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf6fee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59208/0 (socket says 172.21.15.103:59208) 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.096+0000 7f330156b640 1 -- 172.21.15.103:0/14046630 learned_addr learned my addr 172.21.15.103:0/14046630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f3301d6c640 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf6f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f3300d6a640 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf70ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f330156b640 1 -- 172.21.15.103:0/14046630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 msgr2=0x556baf70ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f330156b640 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf70ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f330156b640 1 -- 172.21.15.103:0/14046630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 msgr2=0x556baf6f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f330156b640 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf6f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f330156b640 1 -- 172.21.15.103:0/14046630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556baf716bf0 con 0x556baf6fea90 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f3300d6a640 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf70ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f330156b640 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf6fee70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f32f40ed4d0 tx=0x7f32f40ba3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e31a94584375a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f32e2ffd640 1 -- 172.21.15.103:0/14046630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4049070 con 0x556baf6fea90 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f32e2ffd640 1 -- 172.21.15.103:0/14046630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32f4003e20 con 0x556baf6fea90 2024-08-19T01:35:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.097+0000 7f32e2ffd640 1 -- 172.21.15.103:0/14046630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40c4490 con 0x556baf6fea90 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- 172.21.15.103:0/14046630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 msgr2=0x556baf6fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf6fee70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f32f40ed4d0 tx=0x7f32f40ba3d0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- 172.21.15.103:0/14046630 shutdown_connections 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf704460 0x556baf6f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6ff3d0 0x556baf70ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 --2- 172.21.15.103:0/14046630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf6fea90 0x556baf6fee70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- 172.21.15.103:0/14046630 >> 172.21.15.103:0/14046630 conn(0x556baf6f5cf0 msgr2=0x556baf7087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- 172.21.15.103:0/14046630 shutdown_connections 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- 172.21.15.103:0/14046630 wait complete. 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- start start 2024-08-19T01:35:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 Processor -- start 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6fea90 0x556baf70d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6ff3d0 0x556baf70d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf704460 0x556baf70e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1a7de0 con 0x556baf704460 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1a7710 con 0x556baf6ff3d0 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.098+0000 7f3306664740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1a4950 con 0x556baf6fea90 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3306664740 10 client.0 _ll_register_callbacks cb 0x7f32f40b8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf704460 0x556baf70e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf704460 0x556baf70e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59216/0 (socket says 172.21.15.103:59216) 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 -- 172.21.15.103:0/4028021716 learned_addr learned my addr 172.21.15.103:0/4028021716 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f330156b640 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6fea90 0x556baf70d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3300d6a640 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6ff3d0 0x556baf70d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 -- 172.21.15.103:0/4028021716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6fea90 msgr2=0x556baf70d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6fea90 0x556baf70d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 -- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6ff3d0 msgr2=0x556baf70d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6ff3d0 0x556baf70d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 -- 172.21.15.103:0/4028021716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556baf70efa0 con 0x556baf704460 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3301d6c640 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf704460 0x556baf70e990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009370 tx=0x7f32ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1e6ef9b1967440 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f32e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f32e0ff9640 1 -- 172.21.15.103:0/4028021716 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0263f0 con 0x556baf704460 2024-08-19T01:35:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f32e0ff9640 1 -- 172.21.15.103:0/4028021716 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f32ec026590 con 0x556baf704460 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.099+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556baf5e2900 con 0x556baf704460 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f32e0ff9640 1 -- 172.21.15.103:0/4028021716 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32ec026790 con 0x556baf704460 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 10 client.15312 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556baf7102f0 con 0x556baf704460 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f32e0ff9640 1 -- 172.21.15.103:0/4028021716 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556baf5e2900 con 0x556baf704460 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f32e0ff9640 1 -- 172.21.15.103:0/4028021716 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f32ec018040 con 0x556baf704460 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 10 client.15312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 20 client.15312 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf77f6b0 0x556baf781ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 4 client.15312 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556baf62ef00 con 0x556baf77f6b0 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.100+0000 7f330156b640 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf77f6b0 0x556baf781ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.101+0000 7f32e0ff9640 10 client.15312 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.101+0000 7f330156b640 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf77f6b0 0x556baf781ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88d6dc3180f3d73 server_cookie=e58dca4b69849847 in_seq=0 out_seq=0 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f32e0ff9640 1 -- 172.21.15.103:0/4028021716 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556baf62ef00 con 0x556baf77f6b0 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f32e0ff9640 10 client.15312 handle_command_reply: tid=0 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 client.15312 shutdown 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf704460 msgr2=0x556baf70e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf704460 0x556baf70e990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009370 tx=0x7f32ec004ab0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 shutdown_connections 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556baf77f6b0 0x556baf781ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556baf704460 0x556baf70e990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556baf6ff3d0 0x556baf70d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f32e0ff9640 0 client.15312 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 --2- 172.21.15.103:0/4028021716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556baf6fea90 0x556baf70d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 >> 172.21.15.103:0/4028021716 conn(0x556baf6f5cf0 msgr2=0x556baf6f4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 shutdown_connections 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 1 -- 172.21.15.103:0/4028021716 wait complete. 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.112+0000 7f3306664740 20 client.15312 trim_cache size 0 max 16384 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:35:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:35:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:35:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:35:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32e40a2c30 msgr2=0x7f32e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32e40a2c30 0x7f32e40a50f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32ec005c40 tx=0x7f32ec004a40 comp rx=0 tx=0).stop 2024-08-19T01:35:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 msgr2=0x7f32fc197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc197a50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32f40314e0 tx=0x7f32f4031a30 comp rx=0 tx=0).stop 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 shutdown_connections 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f32e40a2c30 0x7f32e40a50f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32fc103420 0x7f32fc19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32fc102a60 0x7f32fc197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 --2- 172.21.15.103:0/1119564235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32fc101870 0x7f32fc197a50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 >> 172.21.15.103:0/1119564235 conn(0x7f32fc0fd010 msgr2=0x7f32fc105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 shutdown_connections 2024-08-19T01:35:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:41.113+0000 7f33037cf640 1 -- 172.21.15.103:0/1119564235 wait complete. 2024-08-19T01:35:42.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:41 smithi190 ceph-mon[22032]: pgmap v336: 97 pgs: 97 active+clean; 662 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-19T01:35:42.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:41 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:35:42.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:41 smithi146 ceph-mon[22098]: pgmap v336: 97 pgs: 97 active+clean; 662 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-19T01:35:42.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:41 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:35:42.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:41 smithi103 ceph-mon[18436]: pgmap v336: 97 pgs: 97 active+clean; 662 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-19T01:35:42.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:41 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:35:44.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:43 smithi190 ceph-mon[22032]: pgmap v337: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-19T01:35:44.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:43 smithi146 ceph-mon[22098]: pgmap v337: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-19T01:35:44.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:43 smithi103 ceph-mon[18436]: pgmap v337: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-19T01:35:46.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:45 smithi190 ceph-mon[22032]: pgmap v338: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-08-19T01:35:46.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:45 smithi146 ceph-mon[22098]: pgmap v338: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-08-19T01:35:46.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:45 smithi103 ceph-mon[18436]: pgmap v338: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-08-19T01:35:48.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:47 smithi190 ceph-mon[22032]: pgmap v339: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-08-19T01:35:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:47 smithi146 ceph-mon[22098]: pgmap v339: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-08-19T01:35:48.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:47 smithi103 ceph-mon[18436]: pgmap v339: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-08-19T01:35:50.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:49 smithi146 ceph-mon[22098]: pgmap v340: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-08-19T01:35:50.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:49 smithi103 ceph-mon[18436]: pgmap v340: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-08-19T01:35:50.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:49 smithi190 ceph-mon[22032]: pgmap v340: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-08-19T01:35:52.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:51 smithi146 ceph-mon[22098]: pgmap v341: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-08-19T01:35:52.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:51 smithi103 ceph-mon[18436]: pgmap v341: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-08-19T01:35:52.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:51 smithi190 ceph-mon[22032]: pgmap v341: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-08-19T01:35:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:53 smithi146 ceph-mon[22098]: pgmap v342: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-08-19T01:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 -- 172.21.15.103:0/1907928156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0100780 msgr2=0x7f56a0100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/1907928156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0100780 0x7f56a0100b80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5694009900 tx=0x7f569402a410 comp rx=0 tx=0).stop 2024-08-19T01:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 -- 172.21.15.103:0/1907928156 shutdown_connections 2024-08-19T01:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/1907928156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56a0102330 0x7f56a0108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/1907928156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0101970 0x7f56a0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/1907928156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0100780 0x7f56a0100b80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 -- 172.21.15.103:0/1907928156 >> 172.21.15.103:0/1907928156 conn(0x7f56a00fbee0 msgr2=0x7f56a00fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 -- 172.21.15.103:0/1907928156 shutdown_connections 2024-08-19T01:35:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.203+0000 7f56a6c1e640 1 -- 172.21.15.103:0/1907928156 wait complete. 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 Processor -- start 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 -- start start 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0100780 0x7f56a0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56a0101970 0x7f56a0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0102330 0x7f56a019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d400 con 0x7f56a0102330 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d280 con 0x7f56a0101970 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a6c1e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010d580 con 0x7f56a0100780 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0102330 0x7f56a019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0102330 0x7f56a019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47598/0 (socket says 172.21.15.103:47598) 2024-08-19T01:35:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 -- 172.21.15.103:0/3758350575 learned_addr learned my addr 172.21.15.103:0/3758350575 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f569ffff640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56a0101970 0x7f56a0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a49ba640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0100780 0x7f56a0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 -- 172.21.15.103:0/3758350575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0100780 msgr2=0x7f56a0197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0100780 0x7f56a0197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 -- 172.21.15.103:0/3758350575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56a0101970 msgr2=0x7f56a0197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56a0101970 0x7f56a0197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.204+0000 7f56a51bb640 1 -- 172.21.15.103:0/3758350575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a019f650 con 0x7f56a0102330 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.205+0000 7f56a51bb640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0102330 0x7f56a019efe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f569800eb10 tx=0x7f569800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.205+0000 7f569dffb640 1 -- 172.21.15.103:0/3758350575 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698004b20 con 0x7f56a0102330 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.205+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a019f940 con 0x7f56a0102330 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.205+0000 7f569dffb640 1 -- 172.21.15.103:0/3758350575 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5698004cc0 con 0x7f56a0102330 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.205+0000 7f569dffb640 1 -- 172.21.15.103:0/3758350575 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698010690 con 0x7f56a0102330 2024-08-19T01:35:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.205+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a019fe40 con 0x7f56a0102330 2024-08-19T01:35:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.207+0000 7f569dffb640 1 -- 172.21.15.103:0/3758350575 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5698005020 con 0x7f56a0102330 2024-08-19T01:35:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.208+0000 7f569dffb640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f56900a2c30 0x7f56900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.208+0000 7f56a49ba640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f56900a2c30 0x7f56900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.208+0000 7f569dffb640 1 -- 172.21.15.103:0/3758350575 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f56980cf6c0 con 0x7f56a0102330 2024-08-19T01:35:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.208+0000 7f56a99e2740 1 Processor -- start 2024-08-19T01:35:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a99e2740 1 -- start start 2024-08-19T01:35:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564ca90 0x56502564ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x565025651e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x565025654920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a99e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5650255ab9d0 con 0x56502564d3d0 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a99e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x565025553820 con 0x56502564ca90 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a99e2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5650255a7990 con 0x565025652460 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a49ba640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f56900a2c30 0x7f56900a50f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5694000c00 tx=0x7f56940314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x565025651e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x565025651e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47610/0 (socket says 172.21.15.103:47610) 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 -- 172.21.15.103:0/2603500797 learned_addr learned my addr 172.21.15.103:0/2603500797 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a51bb640 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x565025654920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a49ba640 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564ca90 0x56502564ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 -- 172.21.15.103:0/2603500797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 msgr2=0x565025654920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x565025654920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 -- 172.21.15.103:0/2603500797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564ca90 msgr2=0x56502564ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564ca90 0x56502564ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 -- 172.21.15.103:0/2603500797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650252bf210 con 0x56502564d3d0 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f56a49ba640 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564ca90 0x56502564ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.209+0000 7f569ffff640 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x565025651e90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f568c00bce0 tx=0x7f568c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc982a2821925bf9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f568affd640 1 -- 172.21.15.103:0/2603500797 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c015070 con 0x56502564d3d0 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f568affd640 1 -- 172.21.15.103:0/2603500797 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f568c01a910 con 0x56502564d3d0 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f568affd640 1 -- 172.21.15.103:0/2603500797 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c011900 con 0x56502564d3d0 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 -- 172.21.15.103:0/2603500797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 msgr2=0x565025651e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x565025651e90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f568c00bce0 tx=0x7f568c00f730 comp rx=0 tx=0).stop 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 -- 172.21.15.103:0/2603500797 shutdown_connections 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x565025654920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x565025651e90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2603500797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564ca90 0x56502564ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 -- 172.21.15.103:0/2603500797 >> 172.21.15.103:0/2603500797 conn(0x565025648100 msgr2=0x565025648500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 -- 172.21.15.103:0/2603500797 shutdown_connections 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.210+0000 7f56a99e2740 1 -- 172.21.15.103:0/2603500797 wait complete. 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 -- start start 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 Processor -- start 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564ca90 0x565025656cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025657210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x56502565e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565025655430 con 0x56502564ca90 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x565025567520 con 0x56502564d3d0 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5650256555b0 con 0x565025652460 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a99e2740 10 client.0 _ll_register_callbacks cb 0x5650252bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a49ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564ca90 0x565025656cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a49ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564ca90 0x565025656cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47624/0 (socket says 172.21.15.103:47624) 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a49ba640 1 -- 172.21.15.103:0/2664065689 learned_addr learned my addr 172.21.15.103:0/2664065689 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f569ffff640 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025657210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.211+0000 7f56a51bb640 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x56502565e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a49ba640 1 -- 172.21.15.103:0/2664065689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 msgr2=0x56502565e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a49ba640 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x56502565e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a49ba640 1 -- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 msgr2=0x565025657210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a49ba640 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025657210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a49ba640 1 -- 172.21.15.103:0/2664065689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502538b620 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a49ba640 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564ca90 0x565025656cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f56a01017d0 tx=0x7f5694009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bce18d6804b5cdd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f567ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f567ffff640 1 -- 172.21.15.103:0/2664065689 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694035050 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f567ffff640 1 -- 172.21.15.103:0/2664065689 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5694005230 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568c01a910 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f567ffff640 1 -- 172.21.15.103:0/2664065689 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56940044d0 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a99e2740 10 client.15324 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565025660fb0 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.212+0000 7f567ffff640 1 -- 172.21.15.103:0/2664065689 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f568c01a910 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f567ffff640 1 -- 172.21.15.103:0/2664065689 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5694003680 con 0x56502564ca90 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f56a99e2740 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f56a99e2740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:54.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f56a99e2740 20 client.15324 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5650256627b0 0x565025664be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f56a99e2740 4 client.15324 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:35:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650255b9100 con 0x5650256627b0 2024-08-19T01:35:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.213+0000 7f569ffff640 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5650256627b0 0x565025664be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.214+0000 7f569ffff640 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5650256627b0 0x565025664be0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d229cbeda372e337 server_cookie=773a781ae25e4b59 in_seq=0 out_seq=0 2024-08-19T01:35:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.214+0000 7f567ffff640 10 client.15324 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.214+0000 7f567ffff640 1 -- 172.21.15.103:0/2664065689 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650255b9100 con 0x5650256627b0 2024-08-19T01:35:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.214+0000 7f567ffff640 10 client.15324 handle_command_reply: tid=0 2024-08-19T01:35:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.214+0000 7f56a99e2740 1 client.15324 shutdown 2024-08-19T01:35:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564ca90 msgr2=0x565025656cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564ca90 0x565025656cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f56a01017d0 tx=0x7f5694009260 comp rx=0 tx=0).stop 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 shutdown_connections 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5650256627b0 0x565025664be0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565025652460 0x56502565e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025657210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 --2- 172.21.15.103:0/2664065689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564ca90 0x565025656cd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f567ffff640 0 client.15324 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 >> 172.21.15.103:0/2664065689 conn(0x565025648100 msgr2=0x565025642ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 shutdown_connections 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 1 -- 172.21.15.103:0/2664065689 wait complete. 2024-08-19T01:35:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.215+0000 7f56a99e2740 20 client.15324 trim_cache size 0 max 16384 2024-08-19T01:35:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 Processor -- start 2024-08-19T01:35:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 -- start start 2024-08-19T01:35:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565025648100 0x5650256647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025656fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a6930 con 0x565025648100 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a8470 con 0x56502564d3d0 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.229+0000 7f56a99e2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a7000 con 0x56502564ca90 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565025648100 0x5650256647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565025648100 0x5650256647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47640/0 (socket says 172.21.15.103:47640) 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 -- 172.21.15.103:0/3665977231 learned_addr learned my addr 172.21.15.103:0/3665977231 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a51bb640 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025656fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f569ffff640 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 -- 172.21.15.103:0/3665977231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 msgr2=0x56502565e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 -- 172.21.15.103:0/3665977231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 msgr2=0x565025656fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025656fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 -- 172.21.15.103:0/3665977231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502564d8b0 con 0x565025648100 2024-08-19T01:35:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f569ffff640 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a51bb640 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025656fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f56a49ba640 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565025648100 0x5650256647c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5694003ce0 tx=0x7f5694009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9667bb128eeb33d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f568a7fc640 1 -- 172.21.15.103:0/3665977231 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940035f0 con 0x565025648100 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f568a7fc640 1 -- 172.21.15.103:0/3665977231 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56940054e0 con 0x565025648100 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.230+0000 7f568a7fc640 1 -- 172.21.15.103:0/3665977231 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694004cc0 con 0x565025648100 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 -- 172.21.15.103:0/3665977231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565025648100 msgr2=0x5650256647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565025648100 0x5650256647c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5694003ce0 tx=0x7f5694009800 comp rx=0 tx=0).stop 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 -- 172.21.15.103:0/3665977231 shutdown_connections 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56502564d3d0 0x565025656fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 --2- 172.21.15.103:0/3665977231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565025648100 0x5650256647c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 -- 172.21.15.103:0/3665977231 >> 172.21.15.103:0/3665977231 conn(0x565025651fb0 msgr2=0x5650256568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 -- 172.21.15.103:0/3665977231 shutdown_connections 2024-08-19T01:35:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 -- 172.21.15.103:0/3665977231 wait complete. 2024-08-19T01:35:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 -- start start 2024-08-19T01:35:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.231+0000 7f56a99e2740 1 Processor -- start 2024-08-19T01:35:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565025648100 0x56502565bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a99e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x56502565d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a99e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a7da0 con 0x56502564d3d0 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a99e2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a76d0 con 0x565025648100 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a99e2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a4910 con 0x56502564ca90 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a99e2740 10 client.0 _ll_register_callbacks cb 0x7f569403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x56502565d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x56502565d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47648/0 (socket says 172.21.15.103:47648) 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 -- 172.21.15.103:0/1122263758 learned_addr learned my addr 172.21.15.103:0/1122263758 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f569ffff640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a49ba640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565025648100 0x56502565bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 -- 172.21.15.103:0/1122263758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 msgr2=0x56502565c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 -- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565025648100 msgr2=0x56502565bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565025648100 0x56502565bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f56a51bb640 1 -- 172.21.15.103:0/1122263758 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502565d8e0 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.232+0000 7f569ffff640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a51bb640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x56502565d2d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f569809bee0 tx=0x7f5698009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88393f24abde4553 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f567ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f567ffff640 1 -- 172.21.15.103:0/1122263758 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569809bf50 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f567ffff640 1 -- 172.21.15.103:0/1122263758 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f56980a4730 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56940054e0 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f567ffff640 1 -- 172.21.15.103:0/1122263758 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56980a0300 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 10 client.15330 fetch_fsmap learned FSMap version 27 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5650256cd9f0 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f567ffff640 1 -- 172.21.15.103:0/1122263758 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56940054e0 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f567ffff640 1 -- 172.21.15.103:0/1122263758 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f56980b9020 con 0x56502564d3d0 2024-08-19T01:35:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 20 client.15330 populate_metadata read hostname 'smithi103' 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x565025652460 0x5650256d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 4 client.15330 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650255b3450 con 0x565025652460 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.233+0000 7f56a49ba640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x565025652460 0x5650256d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.234+0000 7f56a49ba640 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x565025652460 0x5650256d03d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99c9816491b4c2e server_cookie=607113adbdc323d2 in_seq=0 out_seq=0 2024-08-19T01:35:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.234+0000 7f567ffff640 10 client.15330 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f567ffff640 1 -- 172.21.15.103:0/1122263758 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650255b3450 con 0x565025652460 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f567ffff640 10 client.15330 handle_command_reply: tid=0 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 client.15330 shutdown 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 msgr2=0x56502565d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x56502565d2d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f569809bee0 tx=0x7f5698009a70 comp rx=0 tx=0).stop 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 shutdown_connections 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x565025652460 0x5650256d03d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56502564d3d0 0x56502565d2d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56502564ca90 0x56502565c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 --2- 172.21.15.103:0/1122263758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x565025648100 0x56502565bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f567ffff640 0 client.15330 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.237+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 >> 172.21.15.103:0/1122263758 conn(0x565025651fb0 msgr2=0x5650256570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 shutdown_connections 2024-08-19T01:35:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a99e2740 1 -- 172.21.15.103:0/1122263758 wait complete. 2024-08-19T01:35:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a99e2740 20 client.15330 trim_cache size 0 max 16384 2024-08-19T01:35:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:35:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:35:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:35:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f56900a2c30 msgr2=0x7f56900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f56900a2c30 0x7f56900a50f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5694000c00 tx=0x7f56940314a0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0102330 msgr2=0x7f56a019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0102330 0x7f56a019efe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f569800eb10 tx=0x7f569800c2d0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 shutdown_connections 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f56900a2c30 0x7f56900a50f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56a0102330 0x7f56a019efe0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56a0101970 0x7f56a0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 --2- 172.21.15.103:0/3758350575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a0100780 0x7f56a0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:35:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.238+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 >> 172.21.15.103:0/3758350575 conn(0x7f56a00fbee0 msgr2=0x7f56a0104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:35:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.239+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 shutdown_connections 2024-08-19T01:35:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:35:54.239+0000 7f56a6c1e640 1 -- 172.21.15.103:0/3758350575 wait complete. 2024-08-19T01:35:54.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:53 smithi103 ceph-mon[18436]: pgmap v342: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-08-19T01:35:54.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:53 smithi190 ceph-mon[22032]: pgmap v342: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2024-08-19T01:35:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:55 smithi146 ceph-mon[22098]: pgmap v343: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:56.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:55 smithi103 ceph-mon[18436]: pgmap v343: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:56.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:55 smithi190 ceph-mon[22032]: pgmap v343: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-19T01:35:58.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:57 smithi146 ceph-mon[22098]: pgmap v344: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-19T01:35:58.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:57 smithi103 ceph-mon[18436]: pgmap v344: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-19T01:35:58.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:57 smithi190 ceph-mon[22032]: pgmap v344: 97 pgs: 97 active+clean; 663 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-19T01:36:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:35:59 smithi146 ceph-mon[22098]: pgmap v345: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:00.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:35:59 smithi103 ceph-mon[18436]: pgmap v345: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:35:59 smithi190 ceph-mon[22032]: pgmap v345: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:01.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:36:01.509 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:01.900 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:01 smithi103 ceph-mon[18436]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:02.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.121+0000 7f44db20e640 1 -- 172.21.15.103:0/1644266049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4105290 msgr2=0x7f44d410ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.121+0000 7f44db20e640 1 --2- 172.21.15.103:0/1644266049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4105290 0x7f44d410ce10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009ca0 tx=0x7f44cc02c790 comp rx=0 tx=0).stop 2024-08-19T01:36:02.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.121+0000 7f44db20e640 1 -- 172.21.15.103:0/1644266049 shutdown_connections 2024-08-19T01:36:02.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.121+0000 7f44db20e640 1 --2- 172.21.15.103:0/1644266049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4105290 0x7f44d410ce10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.121+0000 7f44db20e640 1 --2- 172.21.15.103:0/1644266049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44d4100b10 0x7f44d4104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.121+0000 7f44db20e640 1 --2- 172.21.15.103:0/1644266049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44d4100160 0x7f44d4100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.121+0000 7f44db20e640 1 -- 172.21.15.103:0/1644266049 >> 172.21.15.103:0/1644266049 conn(0x7f44d4076f60 msgr2=0x7f44d40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:02.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.122+0000 7f44db20e640 1 -- 172.21.15.103:0/1644266049 shutdown_connections 2024-08-19T01:36:02.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.122+0000 7f44db20e640 1 -- 172.21.15.103:0/1644266049 wait complete. 2024-08-19T01:36:02.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.123+0000 7f44db20e640 1 Processor -- start 2024-08-19T01:36:02.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.124+0000 7f44db20e640 1 -- start start 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.124+0000 7f44db20e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4100160 0x7f44d419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.124+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4100160 0x7f44d419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.124+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4100160 0x7f44d419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57104/0 (socket says 172.21.15.103:57104) 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.124+0000 7f44d8faa640 1 -- 172.21.15.103:0/896232437 learned_addr learned my addr 172.21.15.103:0/896232437 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.125+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44d4100b10 0x7f44d419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.125+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44d4105290 0x7f44d4195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.125+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410f920 con 0x7f44d4100160 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.125+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410f7a0 con 0x7f44d4100b10 2024-08-19T01:36:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.125+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410faa0 con 0x7f44d4105290 2024-08-19T01:36:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.125+0000 7f44d3fff640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44d4100b10 0x7f44d419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.125+0000 7f44d97ab640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44d4105290 0x7f44d4195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d3fff640 1 -- 172.21.15.103:0/896232437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44d4105290 msgr2=0x7f44d4195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d3fff640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44d4105290 0x7f44d4195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d3fff640 1 -- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4100160 msgr2=0x7f44d419ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d3fff640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4100160 0x7f44d419ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d3fff640 1 -- 172.21.15.103:0/896232437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d4196380 con 0x7f44d4100b10 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d3fff640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44d4100b10 0x7f44d419bf80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44c000e9f0 tx=0x7f44c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d1ffb640 1 -- 172.21.15.103:0/896232437 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c000ccc0 con 0x7f44d4100b10 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d4196670 con 0x7f44d4100b10 2024-08-19T01:36:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d1ffb640 1 -- 172.21.15.103:0/896232437 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c0005020 con 0x7f44d4100b10 2024-08-19T01:36:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.126+0000 7f44d1ffb640 1 -- 172.21.15.103:0/896232437 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0004a00 con 0x7f44d4100b10 2024-08-19T01:36:02.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.127+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41a29c0 con 0x7f44d4100b10 2024-08-19T01:36:02.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.130+0000 7f44d1ffb640 1 -- 172.21.15.103:0/896232437 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f44c0004ba0 con 0x7f44d4100b10 2024-08-19T01:36:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.131+0000 7f44d1ffb640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f44c40ab450 0x7f44c40ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.131+0000 7f44dd60c740 1 Processor -- start 2024-08-19T01:36:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.131+0000 7f44d1ffb640 1 -- 172.21.15.103:0/896232437 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f44c00cef60 con 0x7f44d4100b10 2024-08-19T01:36:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.131+0000 7f44d8faa640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f44c40ab450 0x7f44c40ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.131+0000 7f44dd60c740 1 -- start start 2024-08-19T01:36:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2da3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2da3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44dd60c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555de2d04980 con 0x555de2da4be0 2024-08-19T01:36:02.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44dd60c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555de2c9ca70 con 0x555de2da3b60 2024-08-19T01:36:02.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44dd60c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555de2d00940 con 0x555de2daa280 2024-08-19T01:36:02.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2da3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2da3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54336/0 (socket says 172.21.15.103:54336) 2024-08-19T01:36:02.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 -- 172.21.15.103:0/3703111173 learned_addr learned my addr 172.21.15.103:0/3703111173 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:02.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d97ab640 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 -- 172.21.15.103:0/3703111173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 msgr2=0x555de2dac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 -- 172.21.15.103:0/3703111173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 msgr2=0x555de2da3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2da3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.132+0000 7f44d3fff640 1 -- 172.21.15.103:0/3703111173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de2cff6e0 con 0x555de2da3b60 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.133+0000 7f44d3fff640 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2da3fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f44c0098160 tx=0x7f44c00096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df52f0c3ff47680 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.133+0000 7f44d8faa640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f44c40ab450 0x7f44c40ad910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f44c80045d0 tx=0x7f44c8011040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.133+0000 7f44beffd640 1 -- 172.21.15.103:0/3703111173 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c00b4070 con 0x555de2da3b60 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.133+0000 7f44beffd640 1 -- 172.21.15.103:0/3703111173 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c009fb40 con 0x555de2da3b60 2024-08-19T01:36:02.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.133+0000 7f44beffd640 1 -- 172.21.15.103:0/3703111173 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c001c040 con 0x555de2da3b60 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 -- 172.21.15.103:0/3703111173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 msgr2=0x555de2da3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2da3fa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f44c0098160 tx=0x7f44c00096e0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 -- 172.21.15.103:0/3703111173 shutdown_connections 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2da3fa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3703111173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2da3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 -- 172.21.15.103:0/3703111173 >> 172.21.15.103:0/3703111173 conn(0x555de2d9fbb0 msgr2=0x555de2d9ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 -- 172.21.15.103:0/3703111173 shutdown_connections 2024-08-19T01:36:02.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 -- 172.21.15.103:0/3703111173 wait complete. 2024-08-19T01:36:02.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.134+0000 7f44dd60c740 1 -- start start 2024-08-19T01:36:02.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 1 Processor -- start 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da3b60 0x555de2d9c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da4be0 0x555de2d9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2d9cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555de2dbdf30 con 0x555de2da3b60 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555de2cbfaa0 con 0x555de2da4be0 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555de2dbe0b0 con 0x555de2daa280 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44dd60c740 10 client.0 _ll_register_callbacks cb 0x555de2caee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da3b60 0x555de2d9c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da3b60 0x555de2d9c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57126/0 (socket says 172.21.15.103:57126) 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.135+0000 7f44d8faa640 1 -- 172.21.15.103:0/1952078258 learned_addr learned my addr 172.21.15.103:0/1952078258 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d3fff640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da4be0 0x555de2d9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d97ab640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2d9cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d8faa640 1 -- 172.21.15.103:0/1952078258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 msgr2=0x555de2d9cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d8faa640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2d9cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d8faa640 1 -- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da4be0 msgr2=0x555de2d9c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d8faa640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da4be0 0x555de2d9c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d8faa640 1 -- 172.21.15.103:0/1952078258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de2ca3650 con 0x555de2da3b60 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d97ab640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2d9cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:02.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44d8faa640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da3b60 0x555de2d9c260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44c800f9e0 tx=0x7f44c8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686a2e04043f0c81 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.136+0000 7f44b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.137+0000 7f44b3fff640 1 -- 172.21.15.103:0/1952078258 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c802a060 con 0x555de2da3b60 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.137+0000 7f44b3fff640 1 -- 172.21.15.103:0/1952078258 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f44c800dae0 con 0x555de2da3b60 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.137+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c009fb40 con 0x555de2da3b60 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.137+0000 7f44b3fff640 1 -- 172.21.15.103:0/1952078258 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44c800dce0 con 0x555de2da3b60 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.137+0000 7f44dd60c740 10 client.15339 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.137+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555de2a763c0 con 0x555de2da3b60 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.138+0000 7f44b3fff640 1 -- 172.21.15.103:0/1952078258 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c009fb40 con 0x555de2da3b60 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.138+0000 7f44b3fff640 1 -- 172.21.15.103:0/1952078258 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f44c8019070 con 0x555de2da3b60 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.139+0000 7f44dd60c740 10 client.15339 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.139+0000 7f44dd60c740 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.139+0000 7f44dd60c740 20 client.15339 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:02.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.139+0000 7f44dd60c740 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2dc1f70 0x555de2dc43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.139+0000 7f44dd60c740 4 client.15339 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.139+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555de2d11ef0 con 0x555de2dc1f70 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.139+0000 7f44d3fff640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2dc1f70 0x555de2dc43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.140+0000 7f44d3fff640 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2dc1f70 0x555de2dc43b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa09a20734b67f48 server_cookie=d4c322d35b3db65e in_seq=0 out_seq=0 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.140+0000 7f44b3fff640 10 client.15339 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.140+0000 7f44b3fff640 1 -- 172.21.15.103:0/1952078258 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555de2d11ef0 con 0x555de2dc1f70 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.140+0000 7f44b3fff640 10 client.15339 handle_command_reply: tid=0 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.140+0000 7f44dd60c740 1 client.15339 shutdown 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da3b60 msgr2=0x555de2d9c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da3b60 0x555de2d9c260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44c800f9e0 tx=0x7f44c8005ee0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 shutdown_connections 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2dc1f70 0x555de2dc43b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2d9cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da4be0 0x555de2d9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 --2- 172.21.15.103:0/1952078258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da3b60 0x555de2d9c260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 >> 172.21.15.103:0/1952078258 conn(0x555de2d9fbb0 msgr2=0x555de2da5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:02.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44b3fff640 0 client.15339 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:02.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 shutdown_connections 2024-08-19T01:36:02.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 1 -- 172.21.15.103:0/1952078258 wait complete. 2024-08-19T01:36:02.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.141+0000 7f44dd60c740 20 client.15339 trim_cache size 0 max 16384 2024-08-19T01:36:02.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.176+0000 7f44dd60c740 1 Processor -- start 2024-08-19T01:36:02.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.176+0000 7f44dd60c740 1 -- start start 2024-08-19T01:36:02.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.176+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dd7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.176+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2da4be0 0x555de2da76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.176+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2daa280 0x555de2da7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44dd60c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4073de0 con 0x555de2daa280 2024-08-19T01:36:02.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44dd60c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a4370 con 0x555de2da3b60 2024-08-19T01:36:02.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dd7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dd7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54352/0 (socket says 172.21.15.103:54352) 2024-08-19T01:36:02.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 -- 172.21.15.103:0/3906687719 learned_addr learned my addr 172.21.15.103:0/3906687719 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44dd60c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a2f00 con 0x555de2da4be0 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d3fff640 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2da4be0 0x555de2da76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 -- 172.21.15.103:0/3906687719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2da4be0 msgr2=0x555de2da76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2da4be0 0x555de2da76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 -- 172.21.15.103:0/3906687719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2daa280 msgr2=0x555de2da7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2daa280 0x555de2da7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 -- 172.21.15.103:0/3906687719 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de2da0840 con 0x555de2da3b60 2024-08-19T01:36:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.177+0000 7f44d8faa640 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dd7b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c80059f0 tx=0x7f44c8003fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df294f90a5aba850 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:02.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.180+0000 7f44be7fc640 1 -- 172.21.15.103:0/3906687719 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c801e070 con 0x555de2da3b60 2024-08-19T01:36:02.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.180+0000 7f44be7fc640 1 -- 172.21.15.103:0/3906687719 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c801a6a0 con 0x555de2da3b60 2024-08-19T01:36:02.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.180+0000 7f44be7fc640 1 -- 172.21.15.103:0/3906687719 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8034080 con 0x555de2da3b60 2024-08-19T01:36:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.181+0000 7f44dd60c740 1 -- 172.21.15.103:0/3906687719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 msgr2=0x555de2dd7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.181+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dd7b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c80059f0 tx=0x7f44c8003fd0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.181+0000 7f44dd60c740 1 -- 172.21.15.103:0/3906687719 shutdown_connections 2024-08-19T01:36:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.181+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2daa280 0x555de2da7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.181+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2da4be0 0x555de2da76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.181+0000 7f44dd60c740 1 --2- 172.21.15.103:0/3906687719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dd7b70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.181+0000 7f44dd60c740 1 -- 172.21.15.103:0/3906687719 >> 172.21.15.103:0/3906687719 conn(0x555de2d9fbb0 msgr2=0x555de2da6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.182+0000 7f44dd60c740 1 -- 172.21.15.103:0/3906687719 shutdown_connections 2024-08-19T01:36:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.182+0000 7f44dd60c740 1 -- 172.21.15.103:0/3906687719 wait complete. 2024-08-19T01:36:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.182+0000 7f44dd60c740 1 -- start start 2024-08-19T01:36:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.183+0000 7f44dd60c740 1 Processor -- start 2024-08-19T01:36:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.183+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dc6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.183+0000 7f44dd60c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2dc71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.183+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dc6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.184+0000 7f44d8faa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dc6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54366/0 (socket says 172.21.15.103:54366) 2024-08-19T01:36:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.184+0000 7f44d8faa640 1 -- 172.21.15.103:0/4048816544 learned_addr learned my addr 172.21.15.103:0/4048816544 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.184+0000 7f44d3fff640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2dc71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.184+0000 7f44dd60c740 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dc76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.184+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a3ca0 con 0x555de2da4be0 2024-08-19T01:36:02.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.185+0000 7f44d97ab640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dc76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.185+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44d41a35d0 con 0x555de2da3b60 2024-08-19T01:36:02.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.185+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4196fd0 con 0x555de2daa280 2024-08-19T01:36:02.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.185+0000 7f44dd60c740 10 client.0 _ll_register_callbacks cb 0x7f44c800f690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.186+0000 7f44d3fff640 1 -- 172.21.15.103:0/4048816544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 msgr2=0x555de2dc76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.186+0000 7f44d3fff640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dc76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.186+0000 7f44d3fff640 1 -- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 msgr2=0x555de2dc6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.186+0000 7f44d3fff640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dc6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.186+0000 7f44d3fff640 1 -- 172.21.15.103:0/4048816544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de2e56760 con 0x555de2da4be0 2024-08-19T01:36:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.186+0000 7f44d8faa640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dc6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.186+0000 7f44d97ab640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dc76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.187+0000 7f44d3fff640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2dc71a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f44c0097bf0 tx=0x7f44c009b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f1c37862ebf434 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.187+0000 7f44b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.187+0000 7f44b3fff640 1 -- 172.21.15.103:0/4048816544 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c001c040 con 0x555de2da4be0 2024-08-19T01:36:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.187+0000 7f44b3fff640 1 -- 172.21.15.103:0/4048816544 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f44c00ccb40 con 0x555de2da4be0 2024-08-19T01:36:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.187+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44c801a6a0 con 0x555de2da4be0 2024-08-19T01:36:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44b3fff640 1 -- 172.21.15.103:0/4048816544 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f44c00b4040 con 0x555de2da4be0 2024-08-19T01:36:02.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 10 client.15342 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:02.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555de2e57a50 con 0x555de2da4be0 2024-08-19T01:36:02.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44b3fff640 1 -- 172.21.15.103:0/4048816544 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44c009fb40 con 0x555de2da4be0 2024-08-19T01:36:02.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44b3fff640 1 -- 172.21.15.103:0/4048816544 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f44c0081020 con 0x555de2da4be0 2024-08-19T01:36:02.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:02.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 10 client.15342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:02.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 20 client.15342 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:02.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2e57d30 0x555de2e5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:02.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 4 client.15342 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:36:02.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.188+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f44c801a6a0 con 0x555de2e57d30 2024-08-19T01:36:02.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.189+0000 7f44d8faa640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2e57d30 0x555de2e5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:02.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.189+0000 7f44b3fff640 10 client.15342 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:02.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.191+0000 7f44d8faa640 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2e57d30 0x555de2e5a120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f07b05927b2c72 server_cookie=2e5927f18536d326 in_seq=0 out_seq=0 2024-08-19T01:36:02.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44b3fff640 1 -- 172.21.15.103:0/4048816544 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f44c801a6a0 con 0x555de2e57d30 2024-08-19T01:36:02.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44b3fff640 10 client.15342 handle_command_reply: tid=0 2024-08-19T01:36:02.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 client.15342 shutdown 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 msgr2=0x555de2dc71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2dc71a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f44c0097bf0 tx=0x7f44c009b8d0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 shutdown_connections 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555de2e57d30 0x555de2e5a120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44b3fff640 0 client.15342 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555de2daa280 0x555de2dc76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555de2da4be0 0x555de2dc71a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 --2- 172.21.15.103:0/4048816544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555de2da3b60 0x555de2dc6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.192+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 >> 172.21.15.103:0/4048816544 conn(0x555de2d9fbb0 msgr2=0x555de2da6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.193+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 shutdown_connections 2024-08-19T01:36:02.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.193+0000 7f44dd60c740 1 -- 172.21.15.103:0/4048816544 wait complete. 2024-08-19T01:36:02.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.193+0000 7f44dd60c740 20 client.15342 trim_cache size 0 max 16384 2024-08-19T01:36:02.195 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:36:02.195 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:36:02.195 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:36:02.195 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:36:02.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f44c40ab450 msgr2=0x7f44c40ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f44c40ab450 0x7f44c40ad910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f44c80045d0 tx=0x7f44c8011040 comp rx=0 tx=0).stop 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44d4100b10 msgr2=0x7f44d419bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44d4100b10 0x7f44d419bf80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44c000e9f0 tx=0x7f44c000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 shutdown_connections 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f44c40ab450 0x7f44c40ad910 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44d4105290 0x7f44d4195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44d4100b10 0x7f44d419bf80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 --2- 172.21.15.103:0/896232437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d4100160 0x7f44d419ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 >> 172.21.15.103:0/896232437 conn(0x7f44d4076f60 msgr2=0x7f44d4104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 shutdown_connections 2024-08-19T01:36:02.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:02.194+0000 7f44db20e640 1 -- 172.21.15.103:0/896232437 wait complete. 2024-08-19T01:36:02.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:01 smithi146 ceph-mon[22098]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:01 smithi190 ceph-mon[22032]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:03.063 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:36:03.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:36:03.337 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.862+0000 7fa13ba21640 1 -- 172.21.15.103:0/2630243200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134102c20 msgr2=0x7fa13410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.862+0000 7fa13ba21640 1 --2- 172.21.15.103:0/2630243200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134102c20 0x7fa13410d770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009ca0 tx=0x7fa12c02a950 comp rx=0 tx=0).stop 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.862+0000 7fa13ba21640 1 -- 172.21.15.103:0/2630243200 shutdown_connections 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.862+0000 7fa13ba21640 1 --2- 172.21.15.103:0/2630243200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134102c20 0x7fa13410d770 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.862+0000 7fa13ba21640 1 --2- 172.21.15.103:0/2630243200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa1341022a0 0x7fa1341026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.862+0000 7fa13ba21640 1 --2- 172.21.15.103:0/2630243200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa134100b50 0x7fa134100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.862+0000 7fa13ba21640 1 -- 172.21.15.103:0/2630243200 >> 172.21.15.103:0/2630243200 conn(0x7fa1340fc500 msgr2=0x7fa1340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.863+0000 7fa13ba21640 1 -- 172.21.15.103:0/2630243200 shutdown_connections 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.863+0000 7fa13ba21640 1 -- 172.21.15.103:0/2630243200 wait complete. 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.864+0000 7fa13ba21640 1 Processor -- start 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.864+0000 7fa13ba21640 1 -- start start 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa13ba21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134100b50 0x7fa134198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa13ba21640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1341022a0 0x7fa134199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa13ba21640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa134102c20 0x7fa1341a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134100b50 0x7fa134198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134100b50 0x7fa134198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57158/0 (socket says 172.21.15.103:57158) 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 -- 172.21.15.103:0/688223962 learned_addr learned my addr 172.21.15.103:0/688223962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa13ba21640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa134110b60 con 0x7fa134100b50 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341109e0 con 0x7fa134102c20 2024-08-19T01:36:03.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa138fbc640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1341022a0 0x7fa134199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa134110ce0 con 0x7fa1341022a0 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa139fbe640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa134102c20 0x7fa1341a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 -- 172.21.15.103:0/688223962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1341022a0 msgr2=0x7fa134199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1341022a0 0x7fa134199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 -- 172.21.15.103:0/688223962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa134102c20 msgr2=0x7fa1341a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa134102c20 0x7fa1341a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.865+0000 7fa1397bd640 1 -- 172.21.15.103:0/688223962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1341a4590 con 0x7fa134100b50 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.866+0000 7fa1397bd640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134100b50 0x7fa134198fd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c890 tx=0x7fa13000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.866+0000 7fa139fbe640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa134102c20 0x7fa1341a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.866+0000 7fa12a7fc640 1 -- 172.21.15.103:0/688223962 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130004220 con 0x7fa134100b50 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.866+0000 7fa12a7fc640 1 -- 172.21.15.103:0/688223962 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1300043c0 con 0x7fa134100b50 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.866+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1341a4760 con 0x7fa134100b50 2024-08-19T01:36:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.866+0000 7fa12a7fc640 1 -- 172.21.15.103:0/688223962 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130004a00 con 0x7fa134100b50 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.866+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1341a4ca0 con 0x7fa134100b50 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.867+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa134100f30 con 0x7fa134100b50 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.868+0000 7fa12a7fc640 1 -- 172.21.15.103:0/688223962 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa130005140 con 0x7fa134100b50 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.869+0000 7fa12a7fc640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa1200a2c00 0x7fa1200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.869+0000 7fa12a7fc640 1 -- 172.21.15.103:0/688223962 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa1300cef70 con 0x7fa134100b50 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.869+0000 7fa138fbc640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa1200a2c00 0x7fa1200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.870+0000 7fa138fbc640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa1200a2c00 0x7fa1200a50c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa13406a7f0 tx=0x7fa124005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:03.873+0000 7fa12a7fc640 1 -- 172.21.15.103:0/688223962 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa130014030 con 0x7fa134100b50 2024-08-19T01:36:04.142 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:03 smithi103 ceph-mon[18436]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.140+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa134108dc0 con 0x7fa134100b50 2024-08-19T01:36:04.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.142+0000 7fa12a7fc640 1 -- 172.21.15.103:0/688223962 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7fa13009bfa0 con 0x7fa134100b50 2024-08-19T01:36:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:36:04.144 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-08-19T01:35:19:962857+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:35:19.321960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:36:04.145 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-08-19T01:36:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa1200a2c00 msgr2=0x7fa1200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa1200a2c00 0x7fa1200a50c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa13406a7f0 tx=0x7fa124005f90 comp rx=0 tx=0).stop 2024-08-19T01:36:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134100b50 msgr2=0x7fa134198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134100b50 0x7fa134198fd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c890 tx=0x7fa13000cd50 comp rx=0 tx=0).stop 2024-08-19T01:36:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 shutdown_connections 2024-08-19T01:36:04.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa1200a2c00 0x7fa1200a50c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:04.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa134102c20 0x7fa1341a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:04.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1341022a0 0x7fa134199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:04.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 --2- 172.21.15.103:0/688223962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa134100b50 0x7fa134198fd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:04.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 >> 172.21.15.103:0/688223962 conn(0x7fa1340fc500 msgr2=0x7fa1340fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:04.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 shutdown_connections 2024-08-19T01:36:04.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:04.144+0000 7fa13ba21640 1 -- 172.21.15.103:0/688223962 wait complete. 2024-08-19T01:36:04.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:03 smithi146 ceph-mon[22098]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:04.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:03 smithi190 ceph-mon[22032]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:04.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:36:05.155 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:05.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:04 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/688223962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:05.279 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:04 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/688223962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:05.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:04 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/688223962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 -- 172.21.15.103:0/2816634420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9350102c20 msgr2=0x7f935010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 --2- 172.21.15.103:0/2816634420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9350102c20 0x7f935010d770 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9348009ca0 tx=0x7f934802a950 comp rx=0 tx=0).stop 2024-08-19T01:36:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 -- 172.21.15.103:0/2816634420 shutdown_connections 2024-08-19T01:36:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 --2- 172.21.15.103:0/2816634420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9350102c20 0x7f935010d770 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 --2- 172.21.15.103:0/2816634420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f93501022a0 0x7f93501026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 --2- 172.21.15.103:0/2816634420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9350100b50 0x7f9350100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 -- 172.21.15.103:0/2816634420 >> 172.21.15.103:0/2816634420 conn(0x7f93500fc500 msgr2=0x7f93500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 -- 172.21.15.103:0/2816634420 shutdown_connections 2024-08-19T01:36:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.758+0000 7f935637c640 1 -- 172.21.15.103:0/2816634420 wait complete. 2024-08-19T01:36:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.759+0000 7f935637c640 1 Processor -- start 2024-08-19T01:36:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.759+0000 7f935637c640 1 -- start start 2024-08-19T01:36:05.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.759+0000 7f935637c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9350100b50 0x7f9350195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f935637c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93501022a0 0x7f9350196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f935637c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9350102c20 0x7f935019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f935637c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f935010fa60 con 0x7f93501022a0 2024-08-19T01:36:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f935637c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f935010f8e0 con 0x7f9350100b50 2024-08-19T01:36:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f935637c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f935010fbe0 con 0x7f9350102c20 2024-08-19T01:36:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f934f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93501022a0 0x7f9350196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f934ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9350100b50 0x7f9350195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f934ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9350100b50 0x7f9350195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54402/0 (socket says 172.21.15.103:54402) 2024-08-19T01:36:05.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f934ffff640 1 -- 172.21.15.103:0/2974002790 learned_addr learned my addr 172.21.15.103:0/2974002790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:05.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f9354919640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9350102c20 0x7f935019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f9354919640 1 -- 172.21.15.103:0/2974002790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9350100b50 msgr2=0x7f9350195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f9354919640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9350100b50 0x7f9350195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f9354919640 1 -- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93501022a0 msgr2=0x7f9350196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f9354919640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93501022a0 0x7f9350196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f9354919640 1 -- 172.21.15.103:0/2974002790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935019aef0 con 0x7f9350102c20 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.760+0000 7f934f7fe640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93501022a0 0x7f9350196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.761+0000 7f9354919640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9350102c20 0x7f935019a810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9348005cc0 tx=0x7f934802f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.761+0000 7f934d7fa640 1 -- 172.21.15.103:0/2974002790 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348034040 con 0x7f9350102c20 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.761+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935019b180 con 0x7f9350102c20 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.761+0000 7f934d7fa640 1 -- 172.21.15.103:0/2974002790 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93480308f0 con 0x7f9350102c20 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.761+0000 7f934d7fa640 1 -- 172.21.15.103:0/2974002790 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934803cd50 con 0x7f9350102c20 2024-08-19T01:36:05.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.761+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935019b620 con 0x7f9350102c20 2024-08-19T01:36:05.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f934d7fa640 1 -- 172.21.15.103:0/2974002790 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f934804e060 con 0x7f9350102c20 2024-08-19T01:36:05.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 Processor -- start 2024-08-19T01:36:05.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 -- start start 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d7ff0 0x55cd315d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cd3153da40 con 0x55cd315e31d0 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cd314d59d0 con 0x55cd315d7ff0 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f935877a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd31539a00 con 0x55cd315d8910 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f934f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f934ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f934f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57176/0 (socket says 172.21.15.103:57176) 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.765+0000 7f934f7fe640 1 -- 172.21.15.103:0/1002828201 learned_addr learned my addr 172.21.15.103:0/1002828201 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934ffff640 1 -- 172.21.15.103:0/1002828201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d7ff0 msgr2=0x55cd315d83d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934ffff640 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d7ff0 0x55cd315d83d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934ffff640 1 -- 172.21.15.103:0/1002828201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 msgr2=0x55cd315e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:05.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934ffff640 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934ffff640 1 -- 172.21.15.103:0/1002828201 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd314dc650 con 0x55cd315d8910 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934f7fe640 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934ffff640 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315e2c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9340009cf0 tx=0x7f9340011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9314b3689bccb81e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f93367fc640 1 -- 172.21.15.103:0/1002828201 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340005220 con 0x55cd315d8910 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f934d7fa640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f93380a2b80 0x7f93380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f93367fc640 1 -- 172.21.15.103:0/1002828201 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9340004c50 con 0x55cd315d8910 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.766+0000 7f93367fc640 1 -- 172.21.15.103:0/1002828201 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400197f0 con 0x55cd315d8910 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.767+0000 7f934d7fa640 1 -- 172.21.15.103:0/2974002790 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f93480f3990 con 0x7f9350102c20 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.767+0000 7f934f7fe640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f93380a2b80 0x7f93380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 -- 172.21.15.103:0/1002828201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 msgr2=0x55cd315e2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315e2c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9340009cf0 tx=0x7f9340011810 comp rx=0 tx=0).stop 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f934f7fe640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f93380a2b80 0x7f93380a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55cd315d5900 tx=0x7f934400a680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:05.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 -- 172.21.15.103:0/1002828201 shutdown_connections 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315e2c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 --2- 172.21.15.103:0/1002828201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d7ff0 0x55cd315d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 -- 172.21.15.103:0/1002828201 >> 172.21.15.103:0/1002828201 conn(0x55cd315d4580 msgr2=0x55cd315d4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 -- 172.21.15.103:0/1002828201 shutdown_connections 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 -- 172.21.15.103:0/1002828201 wait complete. 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 -- start start 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.768+0000 7f935877a740 1 Processor -- start 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f935877a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f935877a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f935877a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f935877a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cd315e5da0 con 0x55cd315e31d0 2024-08-19T01:36:05.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f935877a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cd314f8b20 con 0x55cd315d8910 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f935877a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cd315e5f20 con 0x55cd315d7ff0 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f935877a740 10 client.0 _ll_register_callbacks cb 0x55cd314f6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f934ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57192/0 (socket says 172.21.15.103:57192) 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 -- 172.21.15.103:0/387086260 learned_addr learned my addr 172.21.15.103:0/387086260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f934f7fe640 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 -- 172.21.15.103:0/387086260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 msgr2=0x55cd315e0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 -- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 msgr2=0x55cd315e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f9354919640 1 -- 172.21.15.103:0/387086260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd312af3c0 con 0x55cd315e31d0 2024-08-19T01:36:05.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.769+0000 7f934ffff640 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f9354919640 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e0ac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f934802ae70 tx=0x7f93480bfd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1819b6002352de server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f932f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f932f7fe640 1 -- 172.21.15.103:0/387086260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480dee60 con 0x55cd315e31d0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9340004c50 con 0x55cd315e31d0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f932f7fe640 1 -- 172.21.15.103:0/387086260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f93480d9160 con 0x55cd315e31d0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f932f7fe640 1 -- 172.21.15.103:0/387086260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93480c0920 con 0x55cd315e31d0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f935877a740 10 client.15360 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.770+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd315e5da0 con 0x55cd315e31d0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f932f7fe640 1 -- 172.21.15.103:0/387086260 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9340004c50 con 0x55cd315e31d0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f932f7fe640 1 -- 172.21.15.103:0/387086260 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93480c0480 con 0x55cd315e31d0 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f935877a740 10 client.15360 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f935877a740 10 client.15360 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f935877a740 20 client.15360 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:05.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f935877a740 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd315f2e80 0x55cd315f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f935877a740 4 client.15360 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd3154afb0 con 0x55cd315f2e80 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.771+0000 7f934ffff640 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd315f2e80 0x55cd315f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.772+0000 7f932f7fe640 10 client.15360 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.772+0000 7f934ffff640 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd315f2e80 0x55cd315f52d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2cd70ed102f543a server_cookie=e112f392c22218da in_seq=0 out_seq=0 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f932f7fe640 1 -- 172.21.15.103:0/387086260 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd3154afb0 con 0x55cd315f2e80 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f932f7fe640 10 client.15360 handle_command_reply: tid=0 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 client.15360 shutdown 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 msgr2=0x55cd315e0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e0ac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f934802ae70 tx=0x7f93480bfd60 comp rx=0 tx=0).stop 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 shutdown_connections 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd315f2e80 0x55cd315f52d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd315e0ac0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f932f7fe640 0 client.15360 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:05.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 --2- 172.21.15.103:0/387086260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.773+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 >> 172.21.15.103:0/387086260 conn(0x55cd315d4580 msgr2=0x55cd315da260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:05.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.774+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 shutdown_connections 2024-08-19T01:36:05.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.774+0000 7f935877a740 1 -- 172.21.15.103:0/387086260 wait complete. 2024-08-19T01:36:05.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.774+0000 7f935877a740 20 client.15360 trim_cache size 0 max 16384 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 Processor -- start 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 -- start start 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315d7ff0 0x55cd315daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315e31d0 0x55cd315fabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f934ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315d7ff0 0x55cd315daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93501aada0 con 0x55cd315d7ff0 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93501ac8e0 con 0x55cd315e31d0 2024-08-19T01:36:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f935877a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93501ab470 con 0x55cd315d8910 2024-08-19T01:36:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f934ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315d7ff0 0x55cd315daa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57208/0 (socket says 172.21.15.103:57208) 2024-08-19T01:36:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.787+0000 7f934ffff640 1 -- 172.21.15.103:0/3123374475 learned_addr learned my addr 172.21.15.103:0/3123374475 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:05.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f934f7fe640 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f934ffff640 1 -- 172.21.15.103:0/3123374475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 msgr2=0x55cd315d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:05.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f934ffff640 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f934ffff640 1 -- 172.21.15.103:0/3123374475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315e31d0 msgr2=0x55cd315fabe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:05.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f934ffff640 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315e31d0 0x55cd315fabe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f934ffff640 1 -- 172.21.15.103:0/3123374475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd315d43b0 con 0x55cd315d7ff0 2024-08-19T01:36:05.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f934ffff640 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315d7ff0 0x55cd315daa50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f934001a000 tx=0x7f9340011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5e6e9df9e93140 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:05.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f9335ffb640 1 -- 172.21.15.103:0/3123374475 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934002b5a0 con 0x55cd315d7ff0 2024-08-19T01:36:05.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f9335ffb640 1 -- 172.21.15.103:0/3123374475 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9340031070 con 0x55cd315d7ff0 2024-08-19T01:36:05.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.788+0000 7f9335ffb640 1 -- 172.21.15.103:0/3123374475 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340019b30 con 0x55cd315d7ff0 2024-08-19T01:36:05.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 -- 172.21.15.103:0/3123374475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315d7ff0 msgr2=0x55cd315daa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315d7ff0 0x55cd315daa50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f934001a000 tx=0x7f9340011810 comp rx=0 tx=0).stop 2024-08-19T01:36:05.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 -- 172.21.15.103:0/3123374475 shutdown_connections 2024-08-19T01:36:05.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315e31d0 0x55cd315fabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d8910 0x55cd315d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 --2- 172.21.15.103:0/3123374475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315d7ff0 0x55cd315daa50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 -- 172.21.15.103:0/3123374475 >> 172.21.15.103:0/3123374475 conn(0x55cd315d4580 msgr2=0x55cd315df130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:05.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 -- 172.21.15.103:0/3123374475 shutdown_connections 2024-08-19T01:36:05.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.789+0000 7f935877a740 1 -- 172.21.15.103:0/3123374475 wait complete. 2024-08-19T01:36:05.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.790+0000 7f935877a740 1 -- start start 2024-08-19T01:36:05.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.790+0000 7f935877a740 1 Processor -- start 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.790+0000 7f935877a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.790+0000 7f935877a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.790+0000 7f935877a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd3165bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.790+0000 7f9354919640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd3165bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.790+0000 7f935877a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93501ac210 con 0x55cd315e31d0 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f935877a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f93501abb40 con 0x55cd315d8910 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd3165bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57222/0 (socket says 172.21.15.103:57222) 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f935877a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93501a8da0 con 0x55cd315d7ff0 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 -- 172.21.15.103:0/3163834363 learned_addr learned my addr 172.21.15.103:0/3163834363 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f935877a740 10 client.0 _ll_register_callbacks cb 0x7f93480095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f934ffff640 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f934f7fe640 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 -- 172.21.15.103:0/3163834363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 msgr2=0x55cd315e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 -- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 msgr2=0x55cd315e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 -- 172.21.15.103:0/3163834363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd3165c0e0 con 0x55cd315e31d0 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f9354919640 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd3165bba0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f93480f1830 tx=0x7f934802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78649459c4feb3c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f932f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.791+0000 7f932f7fe640 1 -- 172.21.15.103:0/3163834363 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93480dee60 con 0x55cd315e31d0 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f932f7fe640 1 -- 172.21.15.103:0/3163834363 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9348003ce0 con 0x55cd315e31d0 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9340031070 con 0x55cd315e31d0 2024-08-19T01:36:05.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f932f7fe640 1 -- 172.21.15.103:0/3163834363 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93480bfb80 con 0x55cd315e31d0 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 10 client.15366 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd3165d2f0 con 0x55cd315e31d0 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f932f7fe640 1 -- 172.21.15.103:0/3163834363 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9340031070 con 0x55cd315e31d0 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f932f7fe640 1 -- 172.21.15.103:0/3163834363 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93480c4120 con 0x55cd315e31d0 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 10 client.15366 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 20 client.15366 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd3165d820 0x55cd3165fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 4 client.15366 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.792+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd315454c0 con 0x55cd3165d820 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.793+0000 7f934ffff640 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd3165d820 0x55cd3165fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.793+0000 7f932f7fe640 10 client.15366 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:05.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f934ffff640 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd3165d820 0x55cd3165fc10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4f13d1de0051d87 server_cookie=d073cb8c02a4e0c7 in_seq=0 out_seq=0 2024-08-19T01:36:05.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f932f7fe640 1 -- 172.21.15.103:0/3163834363 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd315454c0 con 0x55cd3165d820 2024-08-19T01:36:05.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f932f7fe640 10 client.15366 handle_command_reply: tid=0 2024-08-19T01:36:05.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 client.15366 shutdown 2024-08-19T01:36:05.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 msgr2=0x55cd3165bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd3165bba0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f93480f1830 tx=0x7f934802a950 comp rx=0 tx=0).stop 2024-08-19T01:36:05.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 shutdown_connections 2024-08-19T01:36:05.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55cd3165d820 0x55cd3165fc10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd315e31d0 0x55cd3165bba0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cd315d8910 0x55cd315e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 --2- 172.21.15.103:0/3163834363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cd315d7ff0 0x55cd315e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 >> 172.21.15.103:0/3163834363 conn(0x55cd315d4580 msgr2=0x55cd315e16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f932f7fe640 0 client.15366 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 shutdown_connections 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.794+0000 7f935877a740 1 -- 172.21.15.103:0/3163834363 wait complete. 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935877a740 20 client.15366 trim_cache size 0 max 16384 2024-08-19T01:36:05.801 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:05.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f93380a2b80 msgr2=0x7f93380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935637c640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f93380a2b80 0x7f93380a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55cd315d5900 tx=0x7f934400a680 comp rx=0 tx=0).stop 2024-08-19T01:36:05.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9350102c20 msgr2=0x7f935019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:05.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935637c640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9350102c20 0x7f935019a810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9348005cc0 tx=0x7f934802f3a0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 shutdown_connections 2024-08-19T01:36:05.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935637c640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f93380a2b80 0x7f93380a5040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.795+0000 7f935637c640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9350102c20 0x7f935019a810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.796+0000 7f935637c640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93501022a0 0x7f9350196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.796+0000 7f935637c640 1 --2- 172.21.15.103:0/2974002790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9350100b50 0x7f9350195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:05.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.796+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 >> 172.21.15.103:0/2974002790 conn(0x7f93500fc500 msgr2=0x7f9350106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:05.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.796+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 shutdown_connections 2024-08-19T01:36:05.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:05.796+0000 7f935637c640 1 -- 172.21.15.103:0/2974002790 wait complete. 2024-08-19T01:36:06.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:05 smithi146 ceph-mon[22098]: pgmap v348: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:06.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:05 smithi103 ceph-mon[18436]: pgmap v348: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:05 smithi190 ceph-mon[22032]: pgmap v348: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:06.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:36:06.927 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.378+0000 7f4197d2c640 1 -- 172.21.15.103:0/3141028718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4190109430 msgr2=0x7f4190109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.378+0000 7f4197d2c640 1 --2- 172.21.15.103:0/3141028718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4190109430 0x7f4190109830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4180009900 tx=0x7f418002a3f0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.379+0000 7f4197d2c640 1 -- 172.21.15.103:0/3141028718 shutdown_connections 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.379+0000 7f4197d2c640 1 --2- 172.21.15.103:0/3141028718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41900733f0 0x7f4190118e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.379+0000 7f4197d2c640 1 --2- 172.21.15.103:0/3141028718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4190072a50 0x7f4190072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.379+0000 7f4197d2c640 1 --2- 172.21.15.103:0/3141028718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4190109430 0x7f4190109830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.379+0000 7f4197d2c640 1 -- 172.21.15.103:0/3141028718 >> 172.21.15.103:0/3141028718 conn(0x7f419006ea10 msgr2=0x7f4190070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.379+0000 7f4197d2c640 1 -- 172.21.15.103:0/3141028718 shutdown_connections 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.379+0000 7f4197d2c640 1 -- 172.21.15.103:0/3141028718 wait complete. 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 Processor -- start 2024-08-19T01:36:07.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 -- start start 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4190072a50 0x7f41901a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41900733f0 0x7f41901a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4190109430 0x7f41901adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4195ac8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4190072a50 0x7f41901a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4195ac8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4190072a50 0x7f41901a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54456/0 (socket says 172.21.15.103:54456) 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4195ac8640 1 -- 172.21.15.103:0/2478544738 learned_addr learned my addr 172.21.15.103:0/2478544738 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f419011c1d0 con 0x7f41900733f0 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f419011c050 con 0x7f4190072a50 2024-08-19T01:36:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f419011c350 con 0x7f4190109430 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.380+0000 7f41952c7640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41900733f0 0x7f41901a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4195ac8640 1 -- 172.21.15.103:0/2478544738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4190109430 msgr2=0x7f41901adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4195ac8640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4190109430 0x7f41901adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4195ac8640 1 -- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41900733f0 msgr2=0x7f41901a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4195ac8640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41900733f0 0x7f41901a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4195ac8640 1 -- 172.21.15.103:0/2478544738 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41901ae410 con 0x7f4190072a50 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4195ac8640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4190072a50 0x7f41901a67f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41800314e0 tx=0x7f4180003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4186ffd640 1 -- 172.21.15.103:0/2478544738 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418003f070 con 0x7f4190072a50 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4186ffd640 1 -- 172.21.15.103:0/2478544738 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4180031c40 con 0x7f4190072a50 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.381+0000 7f4186ffd640 1 -- 172.21.15.103:0/2478544738 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180035030 con 0x7f4190072a50 2024-08-19T01:36:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.382+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41901ae6a0 con 0x7f4190072a50 2024-08-19T01:36:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.382+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901aea50 con 0x7f4190072a50 2024-08-19T01:36:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.386+0000 7f4186ffd640 1 -- 172.21.15.103:0/2478544738 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4180042070 con 0x7f4190072a50 2024-08-19T01:36:07.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.386+0000 7f419abca740 1 Processor -- start 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.386+0000 7f419abca740 1 -- start start 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f419abca740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e48680 0x5630e3e48a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f419abca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e4da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f419abca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e4e050 0x5630e3e50510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f419abca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630e3dab9d0 con 0x5630e3e4e050 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f419abca740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5630e3d53820 con 0x5630e3e48680 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f419abca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5630e3da7990 con 0x5630e3e48fc0 2024-08-19T01:36:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f4195ac8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e4da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f41962c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e48680 0x5630e3e48a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f41962c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e48680 0x5630e3e48a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54472/0 (socket says 172.21.15.103:54472) 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f41962c9640 1 -- 172.21.15.103:0/2914096454 learned_addr learned my addr 172.21.15.103:0/2914096454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.387+0000 7f41952c7640 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e4e050 0x5630e3e50510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f41962c9640 1 -- 172.21.15.103:0/2914096454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 msgr2=0x5630e3e4da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f41962c9640 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e4da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f41962c9640 1 -- 172.21.15.103:0/2914096454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e4e050 msgr2=0x5630e3e50510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f41962c9640 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e4e050 0x5630e3e50510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f41962c9640 1 -- 172.21.15.103:0/2914096454 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e3abf210 con 0x5630e3e48680 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f41952c7640 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e4e050 0x5630e3e50510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f4186ffd640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f417c0a2c30 0x7f417c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f41962c9640 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e48680 0x5630e3e48a80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4188007c50 tx=0x7f418800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22832297176c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f4195ac8640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f417c0a2c30 0x7f417c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f417bfff640 1 -- 172.21.15.103:0/2914096454 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188014070 con 0x5630e3e48680 2024-08-19T01:36:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.388+0000 7f417bfff640 1 -- 172.21.15.103:0/2914096454 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f418800fb00 con 0x5630e3e48680 2024-08-19T01:36:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.389+0000 7f417bfff640 1 -- 172.21.15.103:0/2914096454 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800e9b0 con 0x5630e3e48680 2024-08-19T01:36:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.389+0000 7f4186ffd640 1 -- 172.21.15.103:0/2478544738 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f41800efa50 con 0x7f4190072a50 2024-08-19T01:36:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.389+0000 7f419abca740 1 -- 172.21.15.103:0/2914096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e48680 msgr2=0x5630e3e48a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.389+0000 7f4195ac8640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f417c0a2c30 0x7f417c0a50f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41800ed8f0 tx=0x7f41800f4040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.389+0000 7f419abca740 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e48680 0x5630e3e48a80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4188007c50 tx=0x7f418800c6a0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 -- 172.21.15.103:0/2914096454 shutdown_connections 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e4e050 0x5630e3e50510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e4da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 --2- 172.21.15.103:0/2914096454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e48680 0x5630e3e48a80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 -- 172.21.15.103:0/2914096454 >> 172.21.15.103:0/2914096454 conn(0x5630e3e43cf0 msgr2=0x5630e3e440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 -- 172.21.15.103:0/2914096454 shutdown_connections 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 -- 172.21.15.103:0/2914096454 wait complete. 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 -- start start 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.390+0000 7f419abca740 1 Processor -- start 2024-08-19T01:36:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f419abca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48680 0x5630e3e52890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f419abca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48fc0 0x5630e3e52dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f419abca740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f419abca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630e3e51020 con 0x5630e3e48fc0 2024-08-19T01:36:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f419abca740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5630e3d67520 con 0x5630e3e4e050 2024-08-19T01:36:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f419abca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5630e3e511a0 con 0x5630e3e48680 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f41962c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48680 0x5630e3e52890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f41952c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48fc0 0x5630e3e52dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.391+0000 7f41962c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48680 0x5630e3e52890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:39024/0 (socket says 172.21.15.103:39024) 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 -- 172.21.15.103:0/3169730109 learned_addr learned my addr 172.21.15.103:0/3169730109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 -- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 msgr2=0x5630e3e59e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e59e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 -- 172.21.15.103:0/3169730109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48fc0 msgr2=0x5630e3e52dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f419abca740 10 client.0 _ll_register_callbacks cb 0x5630e3abea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48fc0 0x5630e3e52dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 -- 172.21.15.103:0/3169730109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e3b8b620 con 0x5630e3e48680 2024-08-19T01:36:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f418800fb00 con 0x5630e3e48680 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41952c7640 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48fc0 0x5630e3e52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41962c9640 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48680 0x5630e3e52890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4188004a80 tx=0x7f418800e450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2f71cd2af31d7c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.392+0000 7f41797fa640 10 client.24967 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f41797fa640 1 -- 172.21.15.103:0/3169730109 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188014050 con 0x5630e3e48680 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f41797fa640 1 -- 172.21.15.103:0/3169730109 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f418800e8c0 con 0x5630e3e48680 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f41797fa640 1 -- 172.21.15.103:0/3169730109 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f418800eac0 con 0x5630e3e48680 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 10 client.24967 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5630e3e5cb10 con 0x5630e3e48680 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f41797fa640 1 -- 172.21.15.103:0/3169730109 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4188027020 con 0x5630e3e48680 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 10 client.24967 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 10 client.24967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 20 client.24967 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3e60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 4 client.24967 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.393+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630e3db9100 con 0x5630e3e5df50 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.394+0000 7f41952c7640 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3e60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.394+0000 7f41797fa640 10 client.24967 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.394+0000 7f41952c7640 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3e60340 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1de639b7749a49 server_cookie=d1c8aa8b534a1680 in_seq=0 out_seq=0 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f41797fa640 1 -- 172.21.15.103:0/3169730109 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630e3db9100 con 0x5630e3e5df50 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f41797fa640 10 client.24967 handle_command_reply: tid=0 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 client.24967 shutdown 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48680 msgr2=0x5630e3e52890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48680 0x5630e3e52890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4188004a80 tx=0x7f418800e450 comp rx=0 tx=0).stop 2024-08-19T01:36:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 shutdown_connections 2024-08-19T01:36:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3e60340 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e59e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f41797fa640 0 client.24967 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48fc0 0x5630e3e52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 --2- 172.21.15.103:0/3169730109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48680 0x5630e3e52890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 >> 172.21.15.103:0/3169730109 conn(0x5630e3e43cf0 msgr2=0x5630e3e45500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 shutdown_connections 2024-08-19T01:36:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 1 -- 172.21.15.103:0/3169730109 wait complete. 2024-08-19T01:36:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.395+0000 7f419abca740 20 client.24967 trim_cache size 0 max 16384 2024-08-19T01:36:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 Processor -- start 2024-08-19T01:36:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 -- start start 2024-08-19T01:36:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e44ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e59dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41901b7950 con 0x5630e3e48680 2024-08-19T01:36:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41901b9490 con 0x5630e3e4e050 2024-08-19T01:36:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f419abca740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41901b8020 con 0x5630e3e48fc0 2024-08-19T01:36:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f41962c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e44ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f41962c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e44ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57274/0 (socket says 172.21.15.103:57274) 2024-08-19T01:36:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f41962c9640 1 -- 172.21.15.103:0/2970045214 learned_addr learned my addr 172.21.15.103:0/2970045214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.424+0000 7f41952c7640 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e59dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f41962c9640 1 -- 172.21.15.103:0/2970045214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 msgr2=0x5630e3e59dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f41962c9640 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e59dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f41962c9640 1 -- 172.21.15.103:0/2970045214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 msgr2=0x5630e3e450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f41962c9640 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f41962c9640 1 -- 172.21.15.103:0/2970045214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e3e5af00 con 0x5630e3e48680 2024-08-19T01:36:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f41962c9640 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e44ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f418800fa60 tx=0x7f418801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1e189da74c6694 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f417b7fe640 1 -- 172.21.15.103:0/2970045214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188014070 con 0x5630e3e48680 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f417b7fe640 1 -- 172.21.15.103:0/2970045214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f418800fb00 con 0x5630e3e48680 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.425+0000 7f417b7fe640 1 -- 172.21.15.103:0/2970045214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418800ec60 con 0x5630e3e48680 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 -- 172.21.15.103:0/2970045214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 msgr2=0x5630e3e44ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e44ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f418800fa60 tx=0x7f418801d000 comp rx=0 tx=0).stop 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 -- 172.21.15.103:0/2970045214 shutdown_connections 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e59dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 --2- 172.21.15.103:0/2970045214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e44ba0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 -- 172.21.15.103:0/2970045214 >> 172.21.15.103:0/2970045214 conn(0x5630e3e43cf0 msgr2=0x5630e3e52640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 -- 172.21.15.103:0/2970045214 shutdown_connections 2024-08-19T01:36:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.426+0000 7f419abca740 1 -- 172.21.15.103:0/2970045214 wait complete. 2024-08-19T01:36:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.427+0000 7f419abca740 1 -- start start 2024-08-19T01:36:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.427+0000 7f419abca740 1 Processor -- start 2024-08-19T01:36:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f419abca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e57110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f41962c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e57110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f41962c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e57110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57288/0 (socket says 172.21.15.103:57288) 2024-08-19T01:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f41962c9640 1 -- 172.21.15.103:0/2748375138 learned_addr learned my addr 172.21.15.103:0/2748375138 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e57650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e58660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41901b8dc0 con 0x5630e3e48680 2024-08-19T01:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41901b86f0 con 0x5630e3e4e050 2024-08-19T01:36:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41901b5930 con 0x5630e3e48fc0 2024-08-19T01:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.428+0000 7f41952c7640 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e57650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41952c7640 1 -- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 msgr2=0x5630e3e58660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41952c7640 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e58660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41952c7640 1 -- 172.21.15.103:0/2748375138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 msgr2=0x5630e3e57110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41952c7640 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e57110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41952c7640 1 -- 172.21.15.103:0/2748375138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e3e58c70 con 0x5630e3e48fc0 2024-08-19T01:36:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41962c9640 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f419abca740 10 client.0 _ll_register_callbacks cb 0x7f418c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41952c7640 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e57650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5630e3e60ba0 tx=0x7f418c012e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a066e6fd05f4eddb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41797fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f418800fb00 con 0x5630e3e48fc0 2024-08-19T01:36:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41797fa640 1 -- 172.21.15.103:0/2748375138 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c020af0 con 0x5630e3e48fc0 2024-08-19T01:36:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41797fa640 1 -- 172.21.15.103:0/2748375138 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f418c020c90 con 0x5630e3e48fc0 2024-08-19T01:36:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f41797fa640 1 -- 172.21.15.103:0/2748375138 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f418c005020 con 0x5630e3e48fc0 2024-08-19T01:36:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f419abca740 10 client.24973 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.429+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5630e3e59e40 con 0x5630e3e48fc0 2024-08-19T01:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.430+0000 7f41797fa640 1 -- 172.21.15.103:0/2748375138 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f418c0051c0 con 0x5630e3e48fc0 2024-08-19T01:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.430+0000 7f419abca740 10 client.24973 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.430+0000 7f419abca740 10 client.24973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.430+0000 7f419abca740 20 client.24973 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.430+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3ecb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.430+0000 7f419abca740 4 client.24973 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.430+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630e3db3450 con 0x5630e3e5df50 2024-08-19T01:36:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.431+0000 7f41962c9640 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3ecb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.431+0000 7f41797fa640 10 client.24973 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.431+0000 7f41962c9640 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3ecb090 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bda27058f086c7e server_cookie=e180b2900a970712 in_seq=0 out_seq=0 2024-08-19T01:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f41797fa640 1 -- 172.21.15.103:0/2748375138 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630e3db3450 con 0x5630e3e5df50 2024-08-19T01:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f41797fa640 10 client.24973 handle_command_reply: tid=0 2024-08-19T01:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 client.24973 shutdown 2024-08-19T01:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 msgr2=0x5630e3e57650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e57650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5630e3e60ba0 tx=0x7f418c012e50 comp rx=0 tx=0).stop 2024-08-19T01:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 shutdown_connections 2024-08-19T01:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630e3e5df50 0x5630e3ecb090 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630e3e4e050 0x5630e3e58660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630e3e48fc0 0x5630e3e57650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f41797fa640 0 client.24973 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 --2- 172.21.15.103:0/2748375138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630e3e48680 0x5630e3e57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.435+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 >> 172.21.15.103:0/2748375138 conn(0x5630e3e43cf0 msgr2=0x5630e3e52580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 shutdown_connections 2024-08-19T01:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f419abca740 1 -- 172.21.15.103:0/2748375138 wait complete. 2024-08-19T01:36:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f419abca740 20 client.24973 trim_cache size 0 max 16384 2024-08-19T01:36:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:36:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:36:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:36:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:36:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f417c0a2c30 msgr2=0x7f417c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f4197d2c640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f417c0a2c30 0x7f417c0a50f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41800ed8f0 tx=0x7f41800f4040 comp rx=0 tx=0).stop 2024-08-19T01:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4190072a50 msgr2=0x7f41901a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f4197d2c640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4190072a50 0x7f41901a67f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41800314e0 tx=0x7f4180003c10 comp rx=0 tx=0).stop 2024-08-19T01:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.436+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 shutdown_connections 2024-08-19T01:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.437+0000 7f4197d2c640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f417c0a2c30 0x7f417c0a50f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.437+0000 7f4197d2c640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4190109430 0x7f41901adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.437+0000 7f4197d2c640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41900733f0 0x7f41901a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.437+0000 7f4197d2c640 1 --2- 172.21.15.103:0/2478544738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4190072a50 0x7f41901a67f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.437+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 >> 172.21.15.103:0/2478544738 conn(0x7f419006ea10 msgr2=0x7f4190071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.437+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 shutdown_connections 2024-08-19T01:36:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:07.437+0000 7f4197d2c640 1 -- 172.21.15.103:0/2478544738 wait complete. 2024-08-19T01:36:07.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.500+0000 7f455eb15640 1 -- 172.21.15.103:0/1199356221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581078e0 msgr2=0x7f4558107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.500+0000 7f455eb15640 1 --2- 172.21.15.103:0/1199356221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581078e0 0x7f4558107cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f454c009890 tx=0x7f454c02a360 comp rx=0 tx=0).stop 2024-08-19T01:36:07.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.501+0000 7f455eb15640 1 -- 172.21.15.103:0/1199356221 shutdown_connections 2024-08-19T01:36:07.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.501+0000 7f455eb15640 1 --2- 172.21.15.103:0/1199356221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4558102280 0x7f455810e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.501+0000 7f455eb15640 1 --2- 172.21.15.103:0/1199356221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45581018e0 0x7f4558101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.501+0000 7f455eb15640 1 --2- 172.21.15.103:0/1199356221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581078e0 0x7f4558107cc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.501+0000 7f455eb15640 1 -- 172.21.15.103:0/1199356221 >> 172.21.15.103:0/1199356221 conn(0x7f45580fd600 msgr2=0x7f45580ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.501+0000 7f455eb15640 1 -- 172.21.15.103:0/1199356221 shutdown_connections 2024-08-19T01:36:07.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.502+0000 7f455eb15640 1 -- 172.21.15.103:0/1199356221 wait complete. 2024-08-19T01:36:07.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.502+0000 7f455eb15640 1 Processor -- start 2024-08-19T01:36:07.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.502+0000 7f455eb15640 1 -- start start 2024-08-19T01:36:07.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455eb15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581018e0 0x7f4558197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455eb15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4558102280 0x7f4558198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455eb15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45581078e0 0x7f455819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455eb15640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4558111b90 con 0x7f45581078e0 2024-08-19T01:36:07.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455eb15640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4558111a10 con 0x7f4558102280 2024-08-19T01:36:07.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455eb15640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4558111d10 con 0x7f45581018e0 2024-08-19T01:36:07.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581018e0 0x7f4558197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f4557fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4558102280 0x7f4558198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581018e0 0x7f4558197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:39058/0 (socket says 172.21.15.103:39058) 2024-08-19T01:36:07.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455c8b1640 1 -- 172.21.15.103:0/85291944 learned_addr learned my addr 172.21.15.103:0/85291944 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455c8b1640 1 -- 172.21.15.103:0/85291944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4558102280 msgr2=0x7f4558198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455c8b1640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4558102280 0x7f4558198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.503+0000 7f455c8b1640 1 -- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45581078e0 msgr2=0x7f455819c7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-19T01:36:07.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f455d0b2640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45581078e0 0x7f455819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f455c8b1640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45581078e0 0x7f455819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f455c8b1640 1 -- 172.21.15.103:0/85291944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455819cea0 con 0x7f45581018e0 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f455d0b2640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45581078e0 0x7f455819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f455c8b1640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581018e0 0x7f4558197ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f454c02a950 tx=0x7f454c030b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f4555ffb640 1 -- 172.21.15.103:0/85291944 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c037040 con 0x7f45581018e0 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f4555ffb640 1 -- 172.21.15.103:0/85291944 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454c041070 con 0x7f45581018e0 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.504+0000 7f4555ffb640 1 -- 172.21.15.103:0/85291944 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c030630 con 0x7f45581018e0 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.505+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455819d130 con 0x7f45581018e0 2024-08-19T01:36:07.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.505+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455819d560 con 0x7f45581018e0 2024-08-19T01:36:07.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.508+0000 7f4555ffb640 1 -- 172.21.15.103:0/85291944 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f454c004040 con 0x7f45581018e0 2024-08-19T01:36:07.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.508+0000 7f4560f13740 1 Processor -- start 2024-08-19T01:36:07.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.508+0000 7f4560f13740 1 -- start start 2024-08-19T01:36:07.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f4560f13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ed570 0x557a820ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f4560f13740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ec3d0 0x557a820ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f4560f13740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82101c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f4560f13740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a82051a40 con 0x557a820ed570 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f4560f13740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557a81fe99d0 con 0x557a820ec3d0 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f4560f13740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557a8204da00 con 0x557a820ff870 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f4555ffb640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f45480a2c00 0x7f45480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f455d0b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ed570 0x557a820ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f455d0b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ed570 0x557a820ebe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57326/0 (socket says 172.21.15.103:57326) 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.509+0000 7f455d0b2640 1 -- 172.21.15.103:0/2729150941 learned_addr learned my addr 172.21.15.103:0/2729150941 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f4555ffb640 1 -- 172.21.15.103:0/85291944 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f454c0eedc0 con 0x7f45581018e0 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f455d0b2640 1 -- 172.21.15.103:0/2729150941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 msgr2=0x557a82101c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f455d0b2640 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82101c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f455d0b2640 1 -- 172.21.15.103:0/2729150941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ec3d0 msgr2=0x557a820ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f455c8b1640 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ec3d0 0x557a820ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f455d0b2640 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ec3d0 0x557a820ec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f455d0b2640 1 -- 172.21.15.103:0/2729150941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a81ff0650 con 0x557a820ed570 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.510+0000 7f455d0b2640 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ed570 0x557a820ebe90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4550009e90 tx=0x7f455000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfb4305c9f0f448 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.511+0000 7f455d0b2640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f45480a2c00 0x7f45480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.511+0000 7f4542ffd640 1 -- 172.21.15.103:0/2729150941 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455001a680 con 0x557a820ed570 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.511+0000 7f4542ffd640 1 -- 172.21.15.103:0/2729150941 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f455001acd0 con 0x557a820ed570 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4542ffd640 1 -- 172.21.15.103:0/2729150941 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45500193e0 con 0x557a820ed570 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 -- 172.21.15.103:0/2729150941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ed570 msgr2=0x557a820ebe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f455d0b2640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f45480a2c00 0x7f45480a50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4550011fd0 tx=0x7f4550019680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ed570 0x557a820ebe90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4550009e90 tx=0x7f455000f730 comp rx=0 tx=0).stop 2024-08-19T01:36:07.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 -- 172.21.15.103:0/2729150941 shutdown_connections 2024-08-19T01:36:07.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ec3d0 0x557a820ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 --2- 172.21.15.103:0/2729150941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ed570 0x557a820ebe90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 -- 172.21.15.103:0/2729150941 >> 172.21.15.103:0/2729150941 conn(0x557a820e8580 msgr2=0x557a820e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 -- 172.21.15.103:0/2729150941 shutdown_connections 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.512+0000 7f4560f13740 1 -- 172.21.15.103:0/2729150941 wait complete. 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 -- start start 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 Processor -- start 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ec3d0 0x557a820efb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ff870 0x557a820f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a82102640 con 0x557a820ff870 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557a8200cb20 con 0x557a820ed570 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557a821027c0 con 0x557a820ec3d0 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.513+0000 7f4560f13740 10 client.0 _ll_register_callbacks cb 0x557a8200abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:07.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ec3d0 0x557a820efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455d0b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ff870 0x557a820f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ec3d0 0x557a820efb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:39084/0 (socket says 172.21.15.103:39084) 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 -- 172.21.15.103:0/3397260023 learned_addr learned my addr 172.21.15.103:0/3397260023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f4557fff640 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 -- 172.21.15.103:0/3397260023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 msgr2=0x557a820f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 -- 172.21.15.103:0/3397260023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ff870 msgr2=0x557a820f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ff870 0x557a820f05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 -- 172.21.15.103:0/3397260023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a81dc33c0 con 0x557a820ec3d0 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455d0b2640 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ff870 0x557a820f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f455c8b1640 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ec3d0 0x557a820efb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b8020 tx=0x7f454c0bb190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa07a57f10121a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f4537fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.514+0000 7f4537fff640 1 -- 172.21.15.103:0/3397260023 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0bbd50 con 0x557a820ec3d0 2024-08-19T01:36:07.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4537fff640 1 -- 172.21.15.103:0/3397260023 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f454c005150 con 0x557a820ec3d0 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455001acd0 con 0x557a820ec3d0 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4537fff640 1 -- 172.21.15.103:0/3397260023 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f454c0bb650 con 0x557a820ec3d0 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 10 client.24982 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a821026f0 con 0x557a820ec3d0 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4537fff640 1 -- 172.21.15.103:0/3397260023 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454c0bfa00 con 0x557a820ec3d0 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4537fff640 1 -- 172.21.15.103:0/3397260023 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f454c0bb8b0 con 0x557a820ec3d0 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 10 client.24982 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 10 client.24982 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 20 client.24982 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82111b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 4 client.24982 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f455001acd0 con 0x557a8210f720 2024-08-19T01:36:07.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.515+0000 7f4557fff640 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82111b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.516+0000 7f4537fff640 10 client.24982 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.516+0000 7f4557fff640 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82111b60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c82092b58480a6bc server_cookie=31be951e67666ca4 in_seq=0 out_seq=0 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4537fff640 1 -- 172.21.15.103:0/3397260023 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f455001acd0 con 0x557a8210f720 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4537fff640 10 client.24982 handle_command_reply: tid=0 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 client.24982 shutdown 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ec3d0 msgr2=0x557a820efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ec3d0 0x557a820efb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b8020 tx=0x7f454c0bb190 comp rx=0 tx=0).stop 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 shutdown_connections 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82111b60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ff870 0x557a820f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4537fff640 0 client.24982 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:07.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 --2- 172.21.15.103:0/3397260023 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ec3d0 0x557a820efb50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 >> 172.21.15.103:0/3397260023 conn(0x557a820e8580 msgr2=0x557a820ee2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 shutdown_connections 2024-08-19T01:36:07.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 1 -- 172.21.15.103:0/3397260023 wait complete. 2024-08-19T01:36:07.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.517+0000 7f4560f13740 20 client.24982 trim_cache size 0 max 16384 2024-08-19T01:36:07.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 Processor -- start 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 -- start start 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a820e8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a820e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a820e8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57336/0 (socket says 172.21.15.103:57336) 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f455c8b1640 1 -- 172.21.15.103:0/2083024227 learned_addr learned my addr 172.21.15.103:0/2083024227 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82100c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45581ab000 con 0x557a820ec3d0 2024-08-19T01:36:07.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f45581acb40 con 0x557a820ed570 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.543+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f45581ab6d0 con 0x557a820ff870 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455d0b2640 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f4557fff640 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455c8b1640 1 -- 172.21.15.103:0/2083024227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 msgr2=0x557a82100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455c8b1640 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455c8b1640 1 -- 172.21.15.103:0/2083024227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 msgr2=0x557a820fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455c8b1640 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455c8b1640 1 -- 172.21.15.103:0/2083024227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a820fece0 con 0x557a820ec3d0 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455d0b2640 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f455c8b1640 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a820e8960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f454c0bb350 tx=0x7f454c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49162dcb79d9b0e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f45427fc640 1 -- 172.21.15.103:0/2083024227 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0bafb0 con 0x557a820ec3d0 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f45427fc640 1 -- 172.21.15.103:0/2083024227 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454c0bbd50 con 0x557a820ec3d0 2024-08-19T01:36:07.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.544+0000 7f45427fc640 1 -- 172.21.15.103:0/2083024227 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0c42c0 con 0x557a820ec3d0 2024-08-19T01:36:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.545+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 msgr2=0x557a820e8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.545+0000 7f4560f13740 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a820e8960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f454c0bb350 tx=0x7f454c000bc0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 shutdown_connections 2024-08-19T01:36:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ff870 0x557a82100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ed570 0x557a820fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 --2- 172.21.15.103:0/2083024227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a820e8960 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 >> 172.21.15.103:0/2083024227 conn(0x557a820e8580 msgr2=0x557a820ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 shutdown_connections 2024-08-19T01:36:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 -- 172.21.15.103:0/2083024227 wait complete. 2024-08-19T01:36:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 -- start start 2024-08-19T01:36:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.546+0000 7f4560f13740 1 Processor -- start 2024-08-19T01:36:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f4560f13740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a82116460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a82116460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f455c8b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a82116460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57340/0 (socket says 172.21.15.103:57340) 2024-08-19T01:36:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f455c8b1640 1 -- 172.21.15.103:0/1821743029 learned_addr learned my addr 172.21.15.103:0/1821743029 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ed570 0x557a821169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ff870 0x557a82116ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45581ac470 con 0x557a820ec3d0 2024-08-19T01:36:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f45581abda0 con 0x557a820ff870 2024-08-19T01:36:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f45581a8fe0 con 0x557a820ed570 2024-08-19T01:36:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f455d0b2640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ff870 0x557a82116ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.547+0000 7f4557fff640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ed570 0x557a821169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4560f13740 10 client.0 _ll_register_callbacks cb 0x7f454c005420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f455d0b2640 1 -- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ed570 msgr2=0x557a821169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f455d0b2640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ed570 0x557a821169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f455d0b2640 1 -- 172.21.15.103:0/1821743029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 msgr2=0x557a82116460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f455d0b2640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a82116460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f455d0b2640 1 -- 172.21.15.103:0/1821743029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a82106ad0 con 0x557a820ff870 2024-08-19T01:36:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f455c8b1640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a82116460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c0bbd50 con 0x557a820ff870 2024-08-19T01:36:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f455d0b2640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ff870 0x557a82116ee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4550011b90 tx=0x7f455001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cfb80fc13bc0c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4537fff640 10 client.24989 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:36:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4537fff640 1 -- 172.21.15.103:0/1821743029 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455002c040 con 0x557a820ff870 2024-08-19T01:36:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4537fff640 1 -- 172.21.15.103:0/1821743029 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4550010a30 con 0x557a820ff870 2024-08-19T01:36:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4537fff640 1 -- 172.21.15.103:0/1821743029 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4550010c70 con 0x557a820ff870 2024-08-19T01:36:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4560f13740 10 client.24989 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.548+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({fsmap=27}) -- 0x557a82107c80 con 0x557a820ff870 2024-08-19T01:36:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4537fff640 1 -- 172.21.15.103:0/1821743029 <== mon.2 v2:172.21.15.146:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f45500049c0 con 0x557a820ff870 2024-08-19T01:36:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4560f13740 10 client.24989 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4560f13740 10 client.24989 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:36:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4560f13740 20 client.24989 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4560f13740 4 client.24989 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:07.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a8205efb0 con 0x557a8210f720 2024-08-19T01:36:07.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f455c8b1640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:07.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f455c8b1640 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82108040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff0ed986539e8c24 server_cookie=c5d8632a6b4ec537 in_seq=0 out_seq=0 2024-08-19T01:36:07.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.549+0000 7f4537fff640 10 client.24989 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:07.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4537fff640 1 -- 172.21.15.103:0/1821743029 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a8205efb0 con 0x557a8210f720 2024-08-19T01:36:07.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4537fff640 10 client.24989 handle_command_reply: tid=0 2024-08-19T01:36:07.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 client.24989 shutdown 2024-08-19T01:36:07.564 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:07.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ff870 msgr2=0x557a82116ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ff870 0x557a82116ee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4550011b90 tx=0x7f455001a000 comp rx=0 tx=0).stop 2024-08-19T01:36:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 shutdown_connections 2024-08-19T01:36:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x557a8210f720 0x557a82108040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557a820ff870 0x557a82116ee0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4537fff640 0 client.24989 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557a820ed570 0x557a821169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 --2- 172.21.15.103:0/1821743029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a820ec3d0 0x557a82116460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 >> 172.21.15.103:0/1821743029 conn(0x557a820e8580 msgr2=0x557a820e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 shutdown_connections 2024-08-19T01:36:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 1 -- 172.21.15.103:0/1821743029 wait complete. 2024-08-19T01:36:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.550+0000 7f4560f13740 20 client.24989 trim_cache size 0 max 16384 2024-08-19T01:36:07.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f45480a2c00 msgr2=0x7f45480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f45480a2c00 0x7f45480a50c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4550011fd0 tx=0x7f4550019680 comp rx=0 tx=0).stop 2024-08-19T01:36:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581018e0 msgr2=0x7f4558197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581018e0 0x7f4558197ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f454c02a950 tx=0x7f454c030b20 comp rx=0 tx=0).stop 2024-08-19T01:36:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 shutdown_connections 2024-08-19T01:36:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f45480a2c00 0x7f45480a50c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45581078e0 0x7f455819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4558102280 0x7f4558198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 --2- 172.21.15.103:0/85291944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45581018e0 0x7f4558197ef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 >> 172.21.15.103:0/85291944 conn(0x7f45580fd600 msgr2=0x7f45580fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 shutdown_connections 2024-08-19T01:36:07.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:07.551+0000 7f455eb15640 1 -- 172.21.15.103:0/85291944 wait complete. 2024-08-19T01:36:08.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:07 smithi146 ceph-mon[22098]: pgmap v349: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:08.265 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:07 smithi103 ceph-mon[18436]: pgmap v349: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:08.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:36:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:07 smithi190 ceph-mon[22032]: pgmap v349: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:08.574 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:09.130 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:08 smithi103 ceph-mon[18436]: pgmap v350: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 -- 172.21.15.103:0/1866226696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102c20 msgr2=0x7fdba010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 --2- 172.21.15.103:0/1866226696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102c20 0x7fdba010d770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009cd0 tx=0x7fdb9802a9a0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 -- 172.21.15.103:0/1866226696 shutdown_connections 2024-08-19T01:36:09.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 --2- 172.21.15.103:0/1866226696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba0102c20 0x7fdba010d770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 --2- 172.21.15.103:0/1866226696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba01022a0 0x7fdba01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 --2- 172.21.15.103:0/1866226696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba0100b50 0x7fdba0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 -- 172.21.15.103:0/1866226696 >> 172.21.15.103:0/1866226696 conn(0x7fdba00fc500 msgr2=0x7fdba00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 -- 172.21.15.103:0/1866226696 shutdown_connections 2024-08-19T01:36:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.146+0000 7fdba8722640 1 -- 172.21.15.103:0/1866226696 wait complete. 2024-08-19T01:36:09.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.147+0000 7fdba8722640 1 Processor -- start 2024-08-19T01:36:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.147+0000 7fdba8722640 1 -- start start 2024-08-19T01:36:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.147+0000 7fdba8722640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba0100b50 0x7fdba010d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.147+0000 7fdba8722640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba01022a0 0x7fdba0109470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba8722640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba0102c20 0x7fdba01099b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba8722640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0110bb0 con 0x7fdba0102c20 2024-08-19T01:36:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba0102c20 0x7fdba01099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba0102c20 0x7fdba01099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57356/0 (socket says 172.21.15.103:57356) 2024-08-19T01:36:09.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 -- 172.21.15.103:0/1258366350 learned_addr learned my addr 172.21.15.103:0/1258366350 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0110a30 con 0x7fdba0100b50 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba64be640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba0100b50 0x7fdba010d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0110d30 con 0x7fdba01022a0 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 -- 172.21.15.103:0/1258366350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba01022a0 msgr2=0x7fdba0109470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba01022a0 0x7fdba0109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 -- 172.21.15.103:0/1258366350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba0100b50 msgr2=0x7fdba010d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba0100b50 0x7fdba010d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 -- 172.21.15.103:0/1258366350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba010a0b0 con 0x7fdba0102c20 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.148+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba0102c20 0x7fdba01099b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb98004af0 tx=0x7fdb98030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.149+0000 7fdb977fe640 1 -- 172.21.15.103:0/1258366350 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98047070 con 0x7fdba0102c20 2024-08-19T01:36:09.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.149+0000 7fdb977fe640 1 -- 172.21.15.103:0/1258366350 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb98030c90 con 0x7fdba0102c20 2024-08-19T01:36:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.149+0000 7fdb977fe640 1 -- 172.21.15.103:0/1258366350 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98034040 con 0x7fdba0102c20 2024-08-19T01:36:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.149+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba01aac90 con 0x7fdba0102c20 2024-08-19T01:36:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.149+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba01ab030 con 0x7fdba0102c20 2024-08-19T01:36:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.152+0000 7fdbaab20740 1 Processor -- start 2024-08-19T01:36:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.153+0000 7fdb977fe640 1 -- 172.21.15.103:0/1258366350 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdb98037020 con 0x7fdba0102c20 2024-08-19T01:36:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.153+0000 7fdbaab20740 1 -- start start 2024-08-19T01:36:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd6cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdbaab20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ffd625a40 con 0x561ffd6c0910 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdbaab20740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561ffd5bd9d0 con 0x561ffd6bfff0 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdbaab20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561ffd621a00 con 0x561ffd6cb1d0 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba64be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba64be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57366/0 (socket says 172.21.15.103:57366) 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba64be640 1 -- 172.21.15.103:0/4292493886 learned_addr learned my addr 172.21.15.103:0/4292493886 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd6cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba5cbd640 1 -- 172.21.15.103:0/4292493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 msgr2=0x561ffd6c03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba5cbd640 1 -- 172.21.15.103:0/4292493886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 msgr2=0x561ffd6cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdba5cbd640 1 -- 172.21.15.103:0/4292493886 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ffd5c4650 con 0x561ffd6cb1d0 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.154+0000 7fdb977fe640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb8c0a2cd0 0x7fdb8c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.155+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd6cd5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdba010acc0 tx=0x7fdb90009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7bbaa24c48ef98 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.155+0000 7fdb977fe640 1 -- 172.21.15.103:0/1258366350 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdb980ef460 con 0x7fdba0102c20 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.155+0000 7fdb94ff9640 1 -- 172.21.15.103:0/4292493886 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90005220 con 0x561ffd6cb1d0 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.155+0000 7fdb94ff9640 1 -- 172.21.15.103:0/4292493886 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb90004c50 con 0x561ffd6cb1d0 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.155+0000 7fdba64be640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb8c0a2cd0 0x7fdb8c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.155+0000 7fdb94ff9640 1 -- 172.21.15.103:0/4292493886 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000f770 con 0x561ffd6cb1d0 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 -- 172.21.15.103:0/4292493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 msgr2=0x561ffd6cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd6cd5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdba010acc0 tx=0x7fdb90009470 comp rx=0 tx=0).stop 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdba64be640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb8c0a2cd0 0x7fdb8c0a5190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x561ffd6bd3e0 tx=0x7fdb9c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:09.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 -- 172.21.15.103:0/4292493886 shutdown_connections 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd6cd5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 --2- 172.21.15.103:0/4292493886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 -- 172.21.15.103:0/4292493886 >> 172.21.15.103:0/4292493886 conn(0x561ffd6bc580 msgr2=0x561ffd6bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 -- 172.21.15.103:0/4292493886 shutdown_connections 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.156+0000 7fdbaab20740 1 -- 172.21.15.103:0/4292493886 wait complete. 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 -- start start 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 Processor -- start 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6cb1d0 0x561ffd6c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ffd6cdda0 con 0x561ffd6c0910 2024-08-19T01:36:09.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561ffd5e0b20 con 0x561ffd6cb1d0 2024-08-19T01:36:09.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561ffd6cdf20 con 0x561ffd6bfff0 2024-08-19T01:36:09.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdbaab20740 10 client.0 _ll_register_callbacks cb 0x561ffd5debb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:09.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdba5cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdba5cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57378/0 (socket says 172.21.15.103:57378) 2024-08-19T01:36:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.157+0000 7fdba5cbd640 1 -- 172.21.15.103:0/3664461056 learned_addr learned my addr 172.21.15.103:0/3664461056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:09.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba64be640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba5cbd640 1 -- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 msgr2=0x561ffd6c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba5cbd640 1 -- 172.21.15.103:0/3664461056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6cb1d0 msgr2=0x561ffd6c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6cb1d0 0x561ffd6c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6cb1d0 0x561ffd6c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba5cbd640 1 -- 172.21.15.103:0/3664461056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ffd3973c0 con 0x561ffd6c0910 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6cb1d0 0x561ffd6c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6c8620 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb90004700 tx=0x7fdb900049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d5796e42d8fc77 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdb89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3664461056 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90012250 con 0x561ffd6c0910 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.158+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3664461056 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdb90005020 con 0x561ffd6c0910 2024-08-19T01:36:09.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb90004c50 con 0x561ffd6c0910 2024-08-19T01:36:09.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3664461056 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb9003e050 con 0x561ffd6c0910 2024-08-19T01:36:09.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 10 client.15411 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:09.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ffd70ca20 con 0x561ffd6c0910 2024-08-19T01:36:09.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3664461056 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb90004c50 con 0x561ffd6c0910 2024-08-19T01:36:09.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3664461056 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdb90035070 con 0x561ffd6c0910 2024-08-19T01:36:09.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 10 client.15411 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 10 client.15411 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 20 client.15411 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd70dd70 0x561ffd710160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 4 client.15411 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.159+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ffd632fb0 con 0x561ffd70dd70 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.160+0000 7fdba64be640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd70dd70 0x561ffd710160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.160+0000 7fdba64be640 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd70dd70 0x561ffd710160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5e19708e381b94c server_cookie=a6b0dbe1afa6abdb in_seq=0 out_seq=0 2024-08-19T01:36:09.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.160+0000 7fdb89ffb640 10 client.15411 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3664461056 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ffd632fb0 con 0x561ffd70dd70 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdb89ffb640 10 client.15411 handle_command_reply: tid=0 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 client.15411 shutdown 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 msgr2=0x561ffd6c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6c8620 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb90004700 tx=0x7fdb900049d0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 shutdown_connections 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd70dd70 0x561ffd710160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6cb1d0 0x561ffd6c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6c0910 0x561ffd6c8620 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3664461056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdb89ffb640 0 client.15411 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 >> 172.21.15.103:0/3664461056 conn(0x561ffd6bc580 msgr2=0x561ffd6c22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 shutdown_connections 2024-08-19T01:36:09.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 1 -- 172.21.15.103:0/3664461056 wait complete. 2024-08-19T01:36:09.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.161+0000 7fdbaab20740 20 client.15411 trim_cache size 0 max 16384 2024-08-19T01:36:09.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.186+0000 7fdbaab20740 1 Processor -- start 2024-08-19T01:36:09.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.186+0000 7fdbaab20740 1 -- start start 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd7158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd715e10 0x561ffd6c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba64be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba64be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54602/0 (socket says 172.21.15.103:54602) 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba6cbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd715e10 0x561ffd6c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba6cbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd715e10 0x561ffd6c1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57388/0 (socket says 172.21.15.103:57388) 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba64be640 1 -- 172.21.15.103:0/1815029828 learned_addr learned my addr 172.21.15.103:0/1815029828 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01a29d0 con 0x561ffd715e10 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdba006d100 con 0x561ffd6bfff0 2024-08-19T01:36:09.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdba006bc90 con 0x561ffd6cb1d0 2024-08-19T01:36:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd7158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba64be640 1 -- 172.21.15.103:0/1815029828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 msgr2=0x561ffd7158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba64be640 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd7158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.187+0000 7fdba64be640 1 -- 172.21.15.103:0/1815029828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd715e10 msgr2=0x561ffd6c1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdba64be640 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd715e10 0x561ffd6c1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdba64be640 1 -- 172.21.15.103:0/1815029828 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ffd6bd210 con 0x561ffd6bfff0 2024-08-19T01:36:09.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdba5cbd640 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd7158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd715e10 0x561ffd6c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdba64be640 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c89b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005fd0 tx=0x7fdb9c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7be1e5be69c97f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdb8bfff640 1 -- 172.21.15.103:0/1815029828 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c01e1c0 con 0x561ffd6bfff0 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdb8bfff640 1 -- 172.21.15.103:0/1815029828 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb9c0288f0 con 0x561ffd6bfff0 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.188+0000 7fdb8bfff640 1 -- 172.21.15.103:0/1815029828 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c01c870 con 0x561ffd6bfff0 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.189+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 msgr2=0x561ffd6c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.189+0000 7fdbaab20740 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c89b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005fd0 tx=0x7fdb9c016000 comp rx=0 tx=0).stop 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.189+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 shutdown_connections 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.189+0000 7fdbaab20740 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd715e10 0x561ffd6c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.189+0000 7fdbaab20740 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6cb1d0 0x561ffd7158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.189+0000 7fdbaab20740 1 --2- 172.21.15.103:0/1815029828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd6bfff0 0x561ffd6c89b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.189+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 >> 172.21.15.103:0/1815029828 conn(0x561ffd6bc580 msgr2=0x561ffd6c3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.190+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 shutdown_connections 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.190+0000 7fdbaab20740 1 -- 172.21.15.103:0/1815029828 wait complete. 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.190+0000 7fdbaab20740 1 -- start start 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.190+0000 7fdbaab20740 1 Processor -- start 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.190+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.190+0000 7fdbaab20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6cb1d0 0x561ffd6dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba5cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6cb1d0 0x561ffd6dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba5cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6cb1d0 0x561ffd6dfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57390/0 (socket says 172.21.15.103:57390) 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba5cbd640 1 -- 172.21.15.103:0/3008756090 learned_addr learned my addr 172.21.15.103:0/3008756090 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd715e10 0x561ffd776890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdba006ca30 con 0x561ffd6cb1d0 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdba006c360 con 0x561ffd715e10 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdba010a9a0 con 0x561ffd6bfff0 2024-08-19T01:36:09.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba64be640 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdbaab20740 10 client.0 _ll_register_callbacks cb 0x7fdb9c00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd715e10 0x561ffd776890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba6cbf640 1 -- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 msgr2=0x561ffd6df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba6cbf640 1 -- 172.21.15.103:0/3008756090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6cb1d0 msgr2=0x561ffd6dfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6cb1d0 0x561ffd6dfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba6cbf640 1 -- 172.21.15.103:0/3008756090 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ffd776ea0 con 0x561ffd715e10 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdba6cbf640 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd715e10 0x561ffd776890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb980ecfe0 tx=0x7fdb980bc530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c9b270f09e6f12 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.191+0000 7fdb89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.192+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3008756090 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980c4960 con 0x561ffd715e10 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.192+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3008756090 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdb980b9ff0 con 0x561ffd715e10 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.192+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9c0288f0 con 0x561ffd715e10 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.192+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3008756090 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb980ca020 con 0x561ffd715e10 2024-08-19T01:36:09.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.192+0000 7fdbaab20740 10 client.25004 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:09.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.192+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ffd778100 con 0x561ffd715e10 2024-08-19T01:36:09.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.192+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3008756090 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb980d9070 con 0x561ffd715e10 2024-08-19T01:36:09.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.193+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3008756090 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdb980ef0b0 con 0x561ffd715e10 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.193+0000 7fdbaab20740 10 client.25004 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.193+0000 7fdbaab20740 10 client.25004 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.193+0000 7fdbaab20740 20 client.25004 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.193+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd7788c0 0x561ffd77ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.193+0000 7fdbaab20740 4 client.25004 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.193+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdb9c0288f0 con 0x561ffd7788c0 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.194+0000 7fdba64be640 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd7788c0 0x561ffd77ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.194+0000 7fdba64be640 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd7788c0 0x561ffd77ace0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c8567c0a2a936b4 server_cookie=f8bcec8493a7fd14 in_seq=0 out_seq=0 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.194+0000 7fdb89ffb640 10 client.25004 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.194+0000 7fdb89ffb640 1 -- 172.21.15.103:0/3008756090 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdb9c0288f0 con 0x561ffd7788c0 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdb89ffb640 10 client.25004 handle_command_reply: tid=0 2024-08-19T01:36:09.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 client.25004 shutdown 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd715e10 msgr2=0x561ffd776890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd715e10 0x561ffd776890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb980ecfe0 tx=0x7fdb980bc530 comp rx=0 tx=0).stop 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 shutdown_connections 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x561ffd7788c0 0x561ffd77ace0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561ffd715e10 0x561ffd776890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ffd6cb1d0 0x561ffd6dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 --2- 172.21.15.103:0/3008756090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561ffd6bfff0 0x561ffd6df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 >> 172.21.15.103:0/3008756090 conn(0x561ffd6bc580 msgr2=0x561ffd6c1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.195+0000 7fdb89ffb640 0 client.25004 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.196+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 shutdown_connections 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.196+0000 7fdbaab20740 1 -- 172.21.15.103:0/3008756090 wait complete. 2024-08-19T01:36:09.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.196+0000 7fdbaab20740 20 client.25004 trim_cache size 0 max 16384 2024-08-19T01:36:09.201 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.196+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb8c0a2cd0 msgr2=0x7fdb8c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.196+0000 7fdba8722640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb8c0a2cd0 0x7fdb8c0a5190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x561ffd6bd3e0 tx=0x7fdb9c00b410 comp rx=0 tx=0).stop 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.196+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba0102c20 msgr2=0x7fdba01099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.196+0000 7fdba8722640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba0102c20 0x7fdba01099b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb98004af0 tx=0x7fdb98030890 comp rx=0 tx=0).stop 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 shutdown_connections 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb8c0a2cd0 0x7fdb8c0a5190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdba0102c20 0x7fdba01099b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdba01022a0 0x7fdba0109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 --2- 172.21.15.103:0/1258366350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdba0100b50 0x7fdba010d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:09.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 >> 172.21.15.103:0/1258366350 conn(0x7fdba00fc500 msgr2=0x7fdba0069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:09.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 shutdown_connections 2024-08-19T01:36:09.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:09.197+0000 7fdba8722640 1 -- 172.21.15.103:0/1258366350 wait complete. 2024-08-19T01:36:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:08 smithi190 ceph-mon[22032]: pgmap v350: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:09.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:08 smithi146 ceph-mon[22098]: pgmap v350: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:09.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:36:10.235 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.819+0000 7f025544b640 1 -- 172.21.15.103:0/4145644184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250103410 msgr2=0x7f02501078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.819+0000 7f025544b640 1 --2- 172.21.15.103:0/4145644184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250103410 0x7f02501078c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0244009930 tx=0x7f024402a420 comp rx=0 tx=0).stop 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 -- 172.21.15.103:0/4145644184 shutdown_connections 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 --2- 172.21.15.103:0/4145644184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250107fd0 0x7f025010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 --2- 172.21.15.103:0/4145644184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250103410 0x7f02501078c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 --2- 172.21.15.103:0/4145644184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0250102a60 0x7f0250102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 -- 172.21.15.103:0/4145644184 >> 172.21.15.103:0/4145644184 conn(0x7f02500fc500 msgr2=0x7f02500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 -- 172.21.15.103:0/4145644184 shutdown_connections 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 -- 172.21.15.103:0/4145644184 wait complete. 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.820+0000 7f025544b640 1 Processor -- start 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f025544b640 1 -- start start 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f025544b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250102a60 0x7f025019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f025544b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250107fd0 0x7f025019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f025544b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02501a03a0 0x7f02501aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f025544b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111c80 con 0x7f0250107fd0 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f025544b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111b00 con 0x7f02501a03a0 2024-08-19T01:36:10.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f025544b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111e00 con 0x7f0250102a60 2024-08-19T01:36:10.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250107fd0 0x7f025019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f024effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250102a60 0x7f025019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02501a03a0 0x7f02501aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f024effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250102a60 0x7f025019f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:39172/0 (socket says 172.21.15.103:39172) 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f024effd640 1 -- 172.21.15.103:0/3447151129 learned_addr learned my addr 172.21.15.103:0/3447151129 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.821+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250107fd0 0x7f025019fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57410/0 (socket says 172.21.15.103:57410) 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024f7fe640 1 -- 172.21.15.103:0/3447151129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250102a60 msgr2=0x7f025019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024f7fe640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250102a60 0x7f025019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024f7fe640 1 -- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250107fd0 msgr2=0x7f025019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024f7fe640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250107fd0 0x7f025019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024f7fe640 1 -- 172.21.15.103:0/3447151129 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02501ab1c0 con 0x7f02501a03a0 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024effd640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250102a60 0x7f025019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024e7fc640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250107fd0 0x7f025019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f024f7fe640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02501a03a0 0x7f02501aa9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f024800e940 tx=0x7f024800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f0237fff640 1 -- 172.21.15.103:0/3447151129 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248004d20 con 0x7f02501a03a0 2024-08-19T01:36:10.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02501ab450 con 0x7f02501a03a0 2024-08-19T01:36:10.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.822+0000 7f0237fff640 1 -- 172.21.15.103:0/3447151129 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0248005020 con 0x7f02501a03a0 2024-08-19T01:36:10.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.823+0000 7f0237fff640 1 -- 172.21.15.103:0/3447151129 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480116d0 con 0x7f02501a03a0 2024-08-19T01:36:10.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.823+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501ab9b0 con 0x7f02501a03a0 2024-08-19T01:36:10.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.825+0000 7f0237fff640 1 -- 172.21.15.103:0/3447151129 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f024800cb20 con 0x7f02501a03a0 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0237fff640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02380a2c80 0x7f02380a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0237fff640 1 -- 172.21.15.103:0/3447151129 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f02480cf390 con 0x7f02501a03a0 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 Processor -- start 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 -- start start 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e41ff0 0x55cf68e423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e4cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf68da7a40 con 0x55cf68e41ff0 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cf68d3f9d0 con 0x55cf68e42910 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.826+0000 7f0257849740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cf68da3a00 con 0x55cf68e4d1d0 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024effd640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02380a2c80 0x7f02380a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e4cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e4cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54644/0 (socket says 172.21.15.103:54644) 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024e7fc640 1 -- 172.21.15.103:0/696268264 learned_addr learned my addr 172.21.15.103:0/696268264 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024effd640 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e41ff0 0x55cf68e423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024f7fe640 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024effd640 1 -- 172.21.15.103:0/696268264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 msgr2=0x55cf68e4f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024effd640 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024effd640 1 -- 172.21.15.103:0/696268264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 msgr2=0x55cf68e4cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024effd640 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e4cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.827+0000 7f024effd640 1 -- 172.21.15.103:0/696268264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf68d46650 con 0x55cf68e41ff0 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.828+0000 7f024effd640 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e41ff0 0x55cf68e423d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f02400058c0 tx=0x7f0240013960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431ce3fd2f05a992 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.828+0000 7f02357fa640 1 -- 172.21.15.103:0/696268264 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240022070 con 0x55cf68e41ff0 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.828+0000 7f02357fa640 1 -- 172.21.15.103:0/696268264 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f024001ce50 con 0x55cf68e41ff0 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.828+0000 7f02357fa640 1 -- 172.21.15.103:0/696268264 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240004970 con 0x55cf68e41ff0 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 -- 172.21.15.103:0/696268264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e41ff0 msgr2=0x55cf68e423d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f024effd640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02380a2c80 0x7f02380a5140 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f024001f000 tx=0x7f0240017000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e41ff0 0x55cf68e423d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f02400058c0 tx=0x7f0240013960 comp rx=0 tx=0).stop 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 -- 172.21.15.103:0/696268264 shutdown_connections 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e4cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 --2- 172.21.15.103:0/696268264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e41ff0 0x55cf68e423d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 -- 172.21.15.103:0/696268264 >> 172.21.15.103:0/696268264 conn(0x55cf68e3e580 msgr2=0x55cf68e3e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 -- 172.21.15.103:0/696268264 shutdown_connections 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.829+0000 7f0257849740 1 -- 172.21.15.103:0/696268264 wait complete. 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 -- start start 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 Processor -- start 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f024effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f024effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e4a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57426/0 (socket says 172.21.15.103:57426) 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f024effd640 1 -- 172.21.15.103:0/425826966 learned_addr learned my addr 172.21.15.103:0/425826966 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 0x55cf68e59260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf68e4fda0 con 0x55cf68e42910 2024-08-19T01:36:10.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cf68d62b20 con 0x55cf68e4d1d0 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.830+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cf68e4ff20 con 0x55cf68e4acf0 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024e7fc640 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024effd640 1 -- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 msgr2=0x55cf68e59260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024effd640 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 0x55cf68e59260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024effd640 1 -- 172.21.15.103:0/425826966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4d1d0 msgr2=0x55cf68e4a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024effd640 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024effd640 1 -- 172.21.15.103:0/425826966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf68b193c0 con 0x55cf68e42910 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024e7fc640 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f0257849740 10 client.0 _ll_register_callbacks cb 0x55cf68d60bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f024effd640 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e4a270 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0240005fd0 tx=0x7f024001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6558ae1bacbc16a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024001ce50 con 0x55cf68e42910 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.831+0000 7f0232ff5640 10 client.15429 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0232ff5640 1 -- 172.21.15.103:0/425826966 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240035050 con 0x55cf68e42910 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0232ff5640 1 -- 172.21.15.103:0/425826966 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0240030630 con 0x55cf68e42910 2024-08-19T01:36:10.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0232ff5640 1 -- 172.21.15.103:0/425826966 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0240022040 con 0x55cf68e42910 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 10 client.15429 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55cf68e5bcc0 con 0x55cf68e42910 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0232ff5640 1 -- 172.21.15.103:0/425826966 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f024000f040 con 0x55cf68e42910 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 10 client.15429 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 20 client.15429 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68e5f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 4 client.15429 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.832+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf68db4fb0 con 0x55cf68e5d0c0 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.833+0000 7f024e7fc640 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68e5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.833+0000 7f0232ff5640 10 client.15429 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.833+0000 7f024e7fc640 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68e5f560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd6a16de8a628f8a server_cookie=ba5812076eb1d900 in_seq=0 out_seq=0 2024-08-19T01:36:10.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0232ff5640 1 -- 172.21.15.103:0/425826966 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf68db4fb0 con 0x55cf68e5d0c0 2024-08-19T01:36:10.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0232ff5640 10 client.15429 handle_command_reply: tid=0 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 client.15429 shutdown 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 msgr2=0x55cf68e4a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e4a270 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0240005fd0 tx=0x7f024001cbd0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 shutdown_connections 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68e5f560 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0232ff5640 0 client.15429 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 0x55cf68e59260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4d1d0 0x55cf68e4a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.834+0000 7f0257849740 1 --2- 172.21.15.103:0/425826966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e4a270 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.835+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 >> 172.21.15.103:0/425826966 conn(0x55cf68e3e580 msgr2=0x55cf68e442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.835+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 shutdown_connections 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.835+0000 7f0257849740 1 -- 172.21.15.103:0/425826966 wait complete. 2024-08-19T01:36:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.835+0000 7f0257849740 20 client.15429 trim_cache size 0 max 16384 2024-08-19T01:36:10.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 Processor -- start 2024-08-19T01:36:10.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 -- start start 2024-08-19T01:36:10.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e59050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4acf0 0x55cf68e3dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e64e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b3dc0 con 0x55cf68e42910 2024-08-19T01:36:10.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b5900 con 0x55cf68e4acf0 2024-08-19T01:36:10.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f0257849740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b4490 con 0x55cf68e4d1d0 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.848+0000 7f024effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e59050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4acf0 0x55cf68e3dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e59050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57432/0 (socket says 172.21.15.103:57432) 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e64e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024effd640 1 -- 172.21.15.103:0/2861635382 learned_addr learned my addr 172.21.15.103:0/2861635382 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024e7fc640 1 -- 172.21.15.103:0/2861635382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 msgr2=0x55cf68e64e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024e7fc640 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e64e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024e7fc640 1 -- 172.21.15.103:0/2861635382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 msgr2=0x55cf68e59050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024e7fc640 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e59050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024e7fc640 1 -- 172.21.15.103:0/2861635382 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf68e3ca90 con 0x55cf68e4acf0 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024f7fe640 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e64e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:10.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.849+0000 7f024effd640 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e59050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:10.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.850+0000 7f024e7fc640 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4acf0 0x55cf68e3dce0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0244009230 tx=0x7f0244005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91383e9f4250191c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.850+0000 7f0234ff9640 1 -- 172.21.15.103:0/2861635382 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244047020 con 0x55cf68e4acf0 2024-08-19T01:36:10.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.850+0000 7f0234ff9640 1 -- 172.21.15.103:0/2861635382 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0244031880 con 0x55cf68e4acf0 2024-08-19T01:36:10.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.850+0000 7f0234ff9640 1 -- 172.21.15.103:0/2861635382 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244035040 con 0x55cf68e4acf0 2024-08-19T01:36:10.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.851+0000 7f0257849740 1 -- 172.21.15.103:0/2861635382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4acf0 msgr2=0x55cf68e3dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.851+0000 7f0257849740 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4acf0 0x55cf68e3dce0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0244009230 tx=0x7f0244005d50 comp rx=0 tx=0).stop 2024-08-19T01:36:10.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.851+0000 7f0257849740 1 -- 172.21.15.103:0/2861635382 shutdown_connections 2024-08-19T01:36:10.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.851+0000 7f0257849740 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4d1d0 0x55cf68e64e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.851+0000 7f0257849740 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e4acf0 0x55cf68e3dce0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.851+0000 7f0257849740 1 --2- 172.21.15.103:0/2861635382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e42910 0x55cf68e59050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.852+0000 7f0257849740 1 -- 172.21.15.103:0/2861635382 >> 172.21.15.103:0/2861635382 conn(0x55cf68e3e580 msgr2=0x55cf68e4c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.852+0000 7f0257849740 1 -- 172.21.15.103:0/2861635382 shutdown_connections 2024-08-19T01:36:10.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.852+0000 7f0257849740 1 -- 172.21.15.103:0/2861635382 wait complete. 2024-08-19T01:36:10.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.852+0000 7f0257849740 1 -- start start 2024-08-19T01:36:10.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.852+0000 7f0257849740 1 Processor -- start 2024-08-19T01:36:10.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f0257849740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f0257849740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 0x55cf68e50c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f0257849740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e4d1d0 0x55cf68ec5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f0257849740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b5230 con 0x55cf68e4d1d0 2024-08-19T01:36:10.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f024effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e4d1d0 0x55cf68ec5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f024effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e50720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54662/0 (socket says 172.21.15.103:54662) 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f024effd640 1 -- 172.21.15.103:0/3337191787 learned_addr learned my addr 172.21.15.103:0/3337191787 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.853+0000 7f024e7fc640 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 0x55cf68e50c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b4b60 con 0x55cf68e42910 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b1da0 con 0x55cf68e4acf0 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f0257849740 10 client.0 _ll_register_callbacks cb 0x7f0240020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f024effd640 1 -- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 msgr2=0x55cf68e50c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f024effd640 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 0x55cf68e50c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f024effd640 1 -- 172.21.15.103:0/3337191787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e4d1d0 msgr2=0x55cf68ec5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f024effd640 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e4d1d0 0x55cf68ec5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f024effd640 1 -- 172.21.15.103:0/3337191787 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf68ec6400 con 0x55cf68e42910 2024-08-19T01:36:10.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f024effd640 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e50720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0240020040 tx=0x7f024000b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cfa280462cbc82 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.854+0000 7f0232ff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0232ff5640 1 -- 172.21.15.103:0/3337191787 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240035040 con 0x55cf68e42910 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0244031880 con 0x55cf68e42910 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0232ff5640 1 -- 172.21.15.103:0/3337191787 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f024003a620 con 0x55cf68e42910 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0232ff5640 1 -- 172.21.15.103:0/3337191787 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f024003a820 con 0x55cf68e42910 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 10 client.25019 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf68ec76a0 con 0x55cf68e42910 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0232ff5640 1 -- 172.21.15.103:0/3337191787 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f024001ce50 con 0x55cf68e42910 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0232ff5640 1 -- 172.21.15.103:0/3337191787 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0240042060 con 0x55cf68e42910 2024-08-19T01:36:10.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 10 client.25019 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:10.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 10 client.25019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:36:10.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 20 client.25019 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68ec9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 4 client.25019 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.855+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0244031880 con 0x55cf68e5d0c0 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.856+0000 7f024e7fc640 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68ec9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.856+0000 7f024e7fc640 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68ec9e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e5d1e504d721df server_cookie=23ccf8d43c95114d in_seq=0 out_seq=0 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.856+0000 7f0232ff5640 10 client.25019 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.856+0000 7f0232ff5640 1 -- 172.21.15.103:0/3337191787 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0244031880 con 0x55cf68e5d0c0 2024-08-19T01:36:10.860 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:10.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.856+0000 7f0232ff5640 10 client.25019 handle_command_reply: tid=0 2024-08-19T01:36:10.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.856+0000 7f0257849740 1 client.25019 shutdown 2024-08-19T01:36:10.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 msgr2=0x55cf68e50720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e50720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0240020040 tx=0x7f024000b980 comp rx=0 tx=0).stop 2024-08-19T01:36:10.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 shutdown_connections 2024-08-19T01:36:10.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55cf68e5d0c0 0x55cf68ec9e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf68e4d1d0 0x55cf68ec5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf68e4acf0 0x55cf68e50c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 --2- 172.21.15.103:0/3337191787 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cf68e42910 0x55cf68e50720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 >> 172.21.15.103:0/3337191787 conn(0x55cf68e3e580 msgr2=0x55cf68e497b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0232ff5640 0 client.25019 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 shutdown_connections 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 1 -- 172.21.15.103:0/3337191787 wait complete. 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.857+0000 7f0257849740 20 client.25019 trim_cache size 0 max 16384 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02380a2c80 msgr2=0x7f02380a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02380a2c80 0x7f02380a5140 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f024001f000 tx=0x7f0240017000 comp rx=0 tx=0).stop 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02501a03a0 msgr2=0x7f02501aa9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02501a03a0 0x7f02501aa9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f024800e940 tx=0x7f024800ee00 comp rx=0 tx=0).stop 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 shutdown_connections 2024-08-19T01:36:10.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02380a2c80 0x7f02380a5140 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02501a03a0 0x7f02501aa9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0250107fd0 0x7f025019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 --2- 172.21.15.103:0/3447151129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0250102a60 0x7f025019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:10.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 >> 172.21.15.103:0/3447151129 conn(0x7f02500fc500 msgr2=0x7f0250100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:10.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 shutdown_connections 2024-08-19T01:36:10.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:10.858+0000 7f025544b640 1 -- 172.21.15.103:0/3447151129 wait complete. 2024-08-19T01:36:11.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:36:11.863 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:12.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:11 smithi190 ceph-mon[22032]: pgmap v351: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-19T01:36:12.148 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:11 smithi103 ceph-mon[18436]: pgmap v351: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-19T01:36:12.154 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:11 smithi146 ceph-mon[22098]: pgmap v351: 97 pgs: 97 active+clean; 664 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.427+0000 7fd486b0a640 1 -- 172.21.15.103:0/56782867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd480101090 msgr2=0x7fd48010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.427+0000 7fd486b0a640 1 --2- 172.21.15.103:0/56782867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd480101090 0x7fd48010dcc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009900 tx=0x7fd46c02a410 comp rx=0 tx=0).stop 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.428+0000 7fd486b0a640 1 -- 172.21.15.103:0/56782867 shutdown_connections 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.428+0000 7fd486b0a640 1 --2- 172.21.15.103:0/56782867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd48010e200 0x7fd480110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.428+0000 7fd486b0a640 1 --2- 172.21.15.103:0/56782867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd480101090 0x7fd48010dcc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.428+0000 7fd486b0a640 1 --2- 172.21.15.103:0/56782867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4801006e0 0x7fd480100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.428+0000 7fd486b0a640 1 -- 172.21.15.103:0/56782867 >> 172.21.15.103:0/56782867 conn(0x7fd4800fc500 msgr2=0x7fd4800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.428+0000 7fd486b0a640 1 -- 172.21.15.103:0/56782867 shutdown_connections 2024-08-19T01:36:12.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.428+0000 7fd486b0a640 1 -- 172.21.15.103:0/56782867 wait complete. 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.429+0000 7fd486b0a640 1 Processor -- start 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.429+0000 7fd486b0a640 1 -- start start 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.429+0000 7fd486b0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4801006e0 0x7fd48019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd486b0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480101090 0x7fd48019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd486b0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48010e200 0x7fd48019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd486b0a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113e20 con 0x7fd48010e200 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48010e200 0x7fd48019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48010e200 0x7fd48019ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47934/0 (socket says 172.21.15.103:47934) 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 -- 172.21.15.103:0/1581978437 learned_addr learned my addr 172.21.15.103:0/1581978437 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113ca0 con 0x7fd480101090 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113fa0 con 0x7fd4801006e0 2024-08-19T01:36:12.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4848a6640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4801006e0 0x7fd48019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd47ffff640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480101090 0x7fd48019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 -- 172.21.15.103:0/1581978437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4801006e0 msgr2=0x7fd48019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4801006e0 0x7fd48019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 -- 172.21.15.103:0/1581978437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480101090 msgr2=0x7fd48019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480101090 0x7fd48019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd4850a7640 1 -- 172.21.15.103:0/1581978437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019f170 con 0x7fd48010e200 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.430+0000 7fd47ffff640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480101090 0x7fd48019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.431+0000 7fd4850a7640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48010e200 0x7fd48019ea90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd478009fd0 tx=0x7fd47800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.431+0000 7fd47dffb640 1 -- 172.21.15.103:0/1581978437 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478017070 con 0x7fd48010e200 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.431+0000 7fd47dffb640 1 -- 172.21.15.103:0/1581978437 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd478003e20 con 0x7fd48010e200 2024-08-19T01:36:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.431+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019f460 con 0x7fd48010e200 2024-08-19T01:36:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.431+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a6d40 con 0x7fd48010e200 2024-08-19T01:36:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.432+0000 7fd47dffb640 1 -- 172.21.15.103:0/1581978437 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478010460 con 0x7fd48010e200 2024-08-19T01:36:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.435+0000 7fd47dffb640 1 -- 172.21.15.103:0/1581978437 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd478003970 con 0x7fd48010e200 2024-08-19T01:36:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd47dffb640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd4700a2c80 0x7fd4700a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd4848a6640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd4700a2c80 0x7fd4700a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd47dffb640 1 -- 172.21.15.103:0/1581978437 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd4780ceda0 con 0x7fd48010e200 2024-08-19T01:36:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 Processor -- start 2024-08-19T01:36:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 -- start start 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d741dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741e120 0x5618d74284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d742add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618d737aa40 con 0x5618d741d800 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5618d73129d0 con 0x5618d74289e0 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.436+0000 7fd488f08740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5618d7376a00 con 0x5618d741e120 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd4848a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d741dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd4848a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d741dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47950/0 (socket says 172.21.15.103:47950) 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd4848a6640 1 -- 172.21.15.103:0/772851191 learned_addr learned my addr 172.21.15.103:0/772851191 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd47ffff640 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741e120 0x5618d74284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd4850a7640 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d742add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd47ffff640 1 -- 172.21.15.103:0/772851191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 msgr2=0x5618d742add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd47ffff640 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d742add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd47ffff640 1 -- 172.21.15.103:0/772851191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 msgr2=0x5618d741dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd47ffff640 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d741dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd47ffff640 1 -- 172.21.15.103:0/772851191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d7319650 con 0x5618d741e120 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd4848a6640 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d741dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.437+0000 7fd4848a6640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd4700a2c80 0x7fd4700a5140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd474005fd0 tx=0x7fd47400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.438+0000 7fd47ffff640 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741e120 0x5618d74284a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd46c02a930 tx=0x7fd46c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca8600da9d2f6ee3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.438+0000 7fd46affd640 1 -- 172.21.15.103:0/772851191 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c03f020 con 0x5618d741e120 2024-08-19T01:36:12.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.438+0000 7fd46affd640 1 -- 172.21.15.103:0/772851191 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46c032e50 con 0x5618d741e120 2024-08-19T01:36:12.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.438+0000 7fd46affd640 1 -- 172.21.15.103:0/772851191 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c035070 con 0x5618d741e120 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 -- 172.21.15.103:0/772851191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741e120 msgr2=0x5618d74284a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741e120 0x5618d74284a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd46c02a930 tx=0x7fd46c031400 comp rx=0 tx=0).stop 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 -- 172.21.15.103:0/772851191 shutdown_connections 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d742add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741e120 0x5618d74284a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 --2- 172.21.15.103:0/772851191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d741dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 -- 172.21.15.103:0/772851191 >> 172.21.15.103:0/772851191 conn(0x5618d7419d90 msgr2=0x5618d741a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 -- 172.21.15.103:0/772851191 shutdown_connections 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 -- 172.21.15.103:0/772851191 wait complete. 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 -- start start 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.439+0000 7fd488f08740 1 Processor -- start 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd488f08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7425870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd488f08740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7425db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd488f08740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd488f08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618d743c5c0 con 0x5618d741d800 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd488f08740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5618d7335b20 con 0x5618d741e120 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd488f08740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5618d743c740 con 0x5618d74289e0 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd488f08740 10 client.0 _ll_register_callbacks cb 0x5618d7333bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd4848a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7425870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd4850a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd4850a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:46928/0 (socket says 172.21.15.103:46928) 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd4850a7640 1 -- 172.21.15.103:0/1301151014 learned_addr learned my addr 172.21.15.103:0/1301151014 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd47ffff640 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7425db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.440+0000 7fd4850a7640 1 -- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 msgr2=0x5618d7425db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd4850a7640 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7425db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd4850a7640 1 -- 172.21.15.103:0/1301151014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 msgr2=0x5618d7425870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd4850a7640 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7425870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd4850a7640 1 -- 172.21.15.103:0/1301151014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d70ec3c0 con 0x5618d74289e0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd4848a6640 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7425870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd4850a7640 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74262f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd47809b2d0 tx=0x7fd47809aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f11d9333d4f9bea server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd45ffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd45ffff640 1 -- 172.21.15.103:0/1301151014 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780b4060 con 0x5618d74289e0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd45ffff640 1 -- 172.21.15.103:0/1301151014 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd478097c40 con 0x5618d74289e0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46c032e50 con 0x5618d74289e0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.441+0000 7fd45ffff640 1 -- 172.21.15.103:0/1301151014 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd478020040 con 0x5618d74289e0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 10 client.25021 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618d7448170 con 0x5618d74289e0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd45ffff640 1 -- 172.21.15.103:0/1301151014 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46c032e50 con 0x5618d74289e0 2024-08-19T01:36:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd45ffff640 1 -- 172.21.15.103:0/1301151014 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd478084b60 con 0x5618d74289e0 2024-08-19T01:36:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 10 client.25021 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 10 client.25021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 20 client.25021 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d744b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 4 client.25021 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.442+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d7387fb0 con 0x5618d7449580 2024-08-19T01:36:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.443+0000 7fd4848a6640 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d744b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.443+0000 7fd4848a6640 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d744b990 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e265d2cb0c508a0b server_cookie=c1566f271de06d14 in_seq=0 out_seq=0 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.443+0000 7fd45ffff640 10 client.25021 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.444+0000 7fd45ffff640 1 -- 172.21.15.103:0/1301151014 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618d7387fb0 con 0x5618d7449580 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.444+0000 7fd45ffff640 10 client.25021 handle_command_reply: tid=0 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.444+0000 7fd488f08740 1 client.25021 shutdown 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.444+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 msgr2=0x5618d74262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.444+0000 7fd488f08740 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74262f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd47809b2d0 tx=0x7fd47809aff0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.444+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 shutdown_connections 2024-08-19T01:36:12.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d744b990 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd45ffff640 0 client.25021 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:12.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74262f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7425db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 1 --2- 172.21.15.103:0/1301151014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7425870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 >> 172.21.15.103:0/1301151014 conn(0x5618d7419d90 msgr2=0x5618d741fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:12.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 shutdown_connections 2024-08-19T01:36:12.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 1 -- 172.21.15.103:0/1301151014 wait complete. 2024-08-19T01:36:12.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.445+0000 7fd488f08740 20 client.25021 trim_cache size 0 max 16384 2024-08-19T01:36:12.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.470+0000 7fd488f08740 1 Processor -- start 2024-08-19T01:36:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.470+0000 7fd488f08740 1 -- start start 2024-08-19T01:36:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd488f08740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741d800 0x5618d7420260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd488f08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741e120 0x5618d7424140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd488f08740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d7426140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd488f08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd48006f890 con 0x5618d741e120 2024-08-19T01:36:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd488f08740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800713d0 con 0x5618d74289e0 2024-08-19T01:36:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd488f08740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd48006ff60 con 0x5618d741d800 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741e120 0x5618d7424140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741e120 0x5618d7424140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47966/0 (socket says 172.21.15.103:47966) 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 -- 172.21.15.103:0/1865362141 learned_addr learned my addr 172.21.15.103:0/1865362141 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd4848a6640 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741d800 0x5618d7420260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd4850a7640 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d7426140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 -- 172.21.15.103:0/1865362141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741d800 msgr2=0x5618d7420260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741d800 0x5618d7420260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 -- 172.21.15.103:0/1865362141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 msgr2=0x5618d7426140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d7426140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.471+0000 7fd47ffff640 1 -- 172.21.15.103:0/1865362141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d741df50 con 0x5618d741e120 2024-08-19T01:36:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.472+0000 7fd47ffff640 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741e120 0x5618d7424140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00b230 tx=0x7fd46c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79042a3a1cd1e8ca server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.472+0000 7fd46a7fc640 1 -- 172.21.15.103:0/1865362141 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c04c020 con 0x5618d741e120 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.472+0000 7fd46a7fc640 1 -- 172.21.15.103:0/1865362141 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd46c009520 con 0x5618d741e120 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.472+0000 7fd46a7fc640 1 -- 172.21.15.103:0/1865362141 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c035070 con 0x5618d741e120 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 -- 172.21.15.103:0/1865362141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741e120 msgr2=0x5618d7424140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741e120 0x5618d7424140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00b230 tx=0x7fd46c0097a0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 -- 172.21.15.103:0/1865362141 shutdown_connections 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d74289e0 0x5618d7426140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741e120 0x5618d7424140 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 --2- 172.21.15.103:0/1865362141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d741d800 0x5618d7420260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 -- 172.21.15.103:0/1865362141 >> 172.21.15.103:0/1865362141 conn(0x5618d7419d90 msgr2=0x5618d74277e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 -- 172.21.15.103:0/1865362141 shutdown_connections 2024-08-19T01:36:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.473+0000 7fd488f08740 1 -- 172.21.15.103:0/1865362141 wait complete. 2024-08-19T01:36:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.474+0000 7fd488f08740 1 -- start start 2024-08-19T01:36:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.474+0000 7fd488f08740 1 Processor -- start 2024-08-19T01:36:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.474+0000 7fd488f08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7441c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.474+0000 7fd4848a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7441c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.474+0000 7fd4848a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7441c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47980/0 (socket says 172.21.15.103:47980) 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.474+0000 7fd4848a6640 1 -- 172.21.15.103:0/2083022829 learned_addr learned my addr 172.21.15.103:0/2083022829 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7442160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd480070d00 con 0x5618d741d800 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd480070630 con 0x5618d741e120 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd48006d8b0 con 0x5618d74289e0 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd47ffff640 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7442160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd488f08740 10 client.0 _ll_register_callbacks cb 0x7fd4780092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd4848a6640 1 -- 172.21.15.103:0/2083022829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 msgr2=0x5618d74220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd4848a6640 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd4848a6640 1 -- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 msgr2=0x5618d7442160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd4848a6640 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7442160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd4848a6640 1 -- 172.21.15.103:0/2083022829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d74228f0 con 0x5618d741d800 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd4848a6640 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7441c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd47400b600 tx=0x7fd474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea484daab8ae9cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.475+0000 7fd45ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd45ffff640 1 -- 172.21.15.103:0/2083022829 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47402b070 con 0x5618d741d800 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd45ffff640 1 -- 172.21.15.103:0/2083022829 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd474005020 con 0x5618d741d800 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd46c009520 con 0x5618d741d800 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd45ffff640 1 -- 172.21.15.103:0/2083022829 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd474036050 con 0x5618d741d800 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618d74b3d40 con 0x5618d741d800 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd45ffff640 1 -- 172.21.15.103:0/2083022829 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4740047a0 con 0x5618d741d800 2024-08-19T01:36:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd45ffff640 1 -- 172.21.15.103:0/2083022829 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd4740275a0 con 0x5618d741d800 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 20 client.15450 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d74b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 4 client.15450 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.476+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd46c009520 con 0x5618d7449580 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.477+0000 7fd47ffff640 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d74b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.477+0000 7fd45ffff640 10 client.15450 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.477+0000 7fd47ffff640 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d74b6520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05664c6ee8e5cb7 server_cookie=bf800e4f6259a31a in_seq=0 out_seq=0 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.477+0000 7fd45ffff640 1 -- 172.21.15.103:0/2083022829 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd46c009520 con 0x5618d7449580 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.477+0000 7fd45ffff640 10 client.15450 handle_command_reply: tid=0 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.477+0000 7fd488f08740 1 client.15450 shutdown 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 msgr2=0x5618d7441c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7441c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd47400b600 tx=0x7fd474004730 comp rx=0 tx=0).stop 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 shutdown_connections 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5618d7449580 0x5618d74b6520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5618d74289e0 0x5618d74220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd45ffff640 0 client.15450 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:12.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5618d741e120 0x5618d7442160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 --2- 172.21.15.103:0/2083022829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d741d800 0x5618d7441c20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 >> 172.21.15.103:0/2083022829 conn(0x5618d7419d90 msgr2=0x5618d7444810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:12.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 shutdown_connections 2024-08-19T01:36:12.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 1 -- 172.21.15.103:0/2083022829 wait complete. 2024-08-19T01:36:12.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.478+0000 7fd488f08740 20 client.15450 trim_cache size 0 max 16384 2024-08-19T01:36:12.482 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd4700a2c80 msgr2=0x7fd4700a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd4700a2c80 0x7fd4700a5140 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd474005fd0 tx=0x7fd47400b390 comp rx=0 tx=0).stop 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48010e200 msgr2=0x7fd48019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48010e200 0x7fd48019ea90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd478009fd0 tx=0x7fd47800b260 comp rx=0 tx=0).stop 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 shutdown_connections 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd4700a2c80 0x7fd4700a5140 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd48010e200 0x7fd48019ea90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd480101090 0x7fd48019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 --2- 172.21.15.103:0/1581978437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4801006e0 0x7fd48019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 >> 172.21.15.103:0/1581978437 conn(0x7fd4800fc500 msgr2=0x7fd4800fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 shutdown_connections 2024-08-19T01:36:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:12.479+0000 7fd486b0a640 1 -- 172.21.15.103:0/1581978437 wait complete. 2024-08-19T01:36:14.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:13 smithi190 ceph-mon[22032]: pgmap v352: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:14.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:13 smithi146 ceph-mon[22098]: pgmap v352: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:14.251 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:36:14.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:36:14.282 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:13 smithi103 ceph-mon[18436]: pgmap v352: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:14.622 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:15.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 -- 172.21.15.103:0/3702009019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 msgr2=0x7ff06810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3702009019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 0x7ff06810e340 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff060009900 tx=0x7ff06002a480 comp rx=0 tx=0).stop 2024-08-19T01:36:15.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 -- 172.21.15.103:0/3702009019 shutdown_connections 2024-08-19T01:36:15.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3702009019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 0x7ff06810e340 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3702009019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff068103430 0x7ff06810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3702009019 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068102b10 0x7ff068102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 -- 172.21.15.103:0/3702009019 >> 172.21.15.103:0/3702009019 conn(0x7ff0680fc4f0 msgr2=0x7ff0680fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 -- 172.21.15.103:0/3702009019 shutdown_connections 2024-08-19T01:36:15.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.230+0000 7ff0705f7640 1 -- 172.21.15.103:0/3702009019 wait complete. 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 Processor -- start 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 -- start start 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff068102b10 0x7ff06819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068103430 0x7ff06819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 0x7ff06819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff068111b80 con 0x7ff06810beb0 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff068111a00 con 0x7ff068103430 2024-08-19T01:36:15.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.231+0000 7ff0705f7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff068111d00 con 0x7ff068102b10 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068103430 0x7ff06819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06eb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 0x7ff06819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068103430 0x7ff06819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:45604/0 (socket says 172.21.15.103:45604) 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 -- 172.21.15.103:0/3324288795 learned_addr learned my addr 172.21.15.103:0/3324288795 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 -- 172.21.15.103:0/3324288795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff068102b10 msgr2=0x7ff06819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff068102b10 0x7ff06819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 -- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 msgr2=0x7ff06819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 0x7ff06819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 -- 172.21.15.103:0/3324288795 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06819f200 con 0x7ff068103430 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06eb94640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 0x7ff06819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:15.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.232+0000 7ff06db92640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068103430 0x7ff06819a790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00e9f0 tx=0x7ff05c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.233+0000 7ff0677fe640 1 -- 172.21.15.103:0/3324288795 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c00cd80 con 0x7ff068103430 2024-08-19T01:36:15.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.233+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06819f4f0 con 0x7ff068103430 2024-08-19T01:36:15.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.233+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0681a6dd0 con 0x7ff068103430 2024-08-19T01:36:15.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.234+0000 7ff0677fe640 1 -- 172.21.15.103:0/3324288795 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff05c0054d0 con 0x7ff068103430 2024-08-19T01:36:15.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.234+0000 7ff0677fe640 1 -- 172.21.15.103:0/3324288795 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c010490 con 0x7ff068103430 2024-08-19T01:36:15.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.235+0000 7ff0677fe640 1 -- 172.21.15.103:0/3324288795 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff05c010770 con 0x7ff068103430 2024-08-19T01:36:15.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.236+0000 7ff0677fe640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0500a2c80 0x7ff0500a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.236+0000 7ff0729f5740 1 Processor -- start 2024-08-19T01:36:15.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.236+0000 7ff06e393640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0500a2c80 0x7ff0500a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.236+0000 7ff0677fe640 1 -- 172.21.15.103:0/3324288795 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff05c0cf220 con 0x7ff068103430 2024-08-19T01:36:15.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.236+0000 7ff0729f5740 1 -- start start 2024-08-19T01:36:15.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5738a10 0x558ee5738df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 0x558ee57436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5743bf0 0x558ee5745fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff0729f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ee56f3e40 con 0x558ee5743bf0 2024-08-19T01:36:15.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff0729f5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558ee56957f0 con 0x558ee5739330 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff0729f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ee563cfb0 con 0x558ee5738a10 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06e393640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0500a2c80 0x7ff0500a5140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff058005c40 tx=0x7ff058005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5743bf0 0x558ee5745fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5743bf0 0x558ee5745fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48000/0 (socket says 172.21.15.103:48000) 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 -- 172.21.15.103:0/28847214 learned_addr learned my addr 172.21.15.103:0/28847214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06e393640 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5738a10 0x558ee5738df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 -- 172.21.15.103:0/28847214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5738a10 msgr2=0x558ee5738df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5738a10 0x558ee5738df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 -- 172.21.15.103:0/28847214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 msgr2=0x558ee57436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 0x558ee57436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.237+0000 7ff06eb94640 1 -- 172.21.15.103:0/28847214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee54076a0 con 0x558ee5743bf0 2024-08-19T01:36:15.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.238+0000 7ff06eb94640 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5743bf0 0x558ee5745fe0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff06002aa70 tx=0x7ff060004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a6a176e4cf8b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.238+0000 7ff064ff9640 1 -- 172.21.15.103:0/28847214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060040070 con 0x558ee5743bf0 2024-08-19T01:36:15.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.238+0000 7ff064ff9640 1 -- 172.21.15.103:0/28847214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0600047a0 con 0x558ee5743bf0 2024-08-19T01:36:15.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.238+0000 7ff064ff9640 1 -- 172.21.15.103:0/28847214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060035040 con 0x558ee5743bf0 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 -- 172.21.15.103:0/28847214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5743bf0 msgr2=0x558ee5745fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5743bf0 0x558ee5745fe0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff06002aa70 tx=0x7ff060004430 comp rx=0 tx=0).stop 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 -- 172.21.15.103:0/28847214 shutdown_connections 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5743bf0 0x558ee5745fe0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 0x558ee57436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 --2- 172.21.15.103:0/28847214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5738a10 0x558ee5738df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 -- 172.21.15.103:0/28847214 >> 172.21.15.103:0/28847214 conn(0x558ee5734fa0 msgr2=0x558ee5735380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 -- 172.21.15.103:0/28847214 shutdown_connections 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.239+0000 7ff0729f5740 1 -- 172.21.15.103:0/28847214 wait complete. 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 -- start start 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 Processor -- start 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5740b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee5741090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5743bf0 0x558ee57415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ee5650cb0 con 0x558ee5738a10 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558ee56917b0 con 0x558ee5743bf0 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ee5746810 con 0x558ee5739330 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff06e393640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5740b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff0729f5740 10 client.0 _ll_register_callbacks cb 0x558ee53a4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff06e393640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5740b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48014/0 (socket says 172.21.15.103:48014) 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff06e393640 1 -- 172.21.15.103:0/1826272852 learned_addr learned my addr 172.21.15.103:0/1826272852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff06db92640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee5741090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.240+0000 7ff06eb94640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5743bf0 0x558ee57415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff06db92640 1 -- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5743bf0 msgr2=0x558ee57415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff06db92640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5743bf0 0x558ee57415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff06db92640 1 -- 172.21.15.103:0/1826272852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 msgr2=0x558ee5740b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff06db92640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5740b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff06db92640 1 -- 172.21.15.103:0/1826272852 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee564fbc0 con 0x558ee5739330 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff06e393640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5740b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff06db92640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee5741090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0cccc0 tx=0x7ff05c09b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42336cd6f3908401 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff04dffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff04dffb640 1 -- 172.21.15.103:0/1826272852 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0a4430 con 0x558ee5739330 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0600047a0 con 0x558ee5739330 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff04dffb640 1 -- 172.21.15.103:0/1826272852 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff05c005000 con 0x558ee5739330 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff04dffb640 1 -- 172.21.15.103:0/1826272852 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff05c09fe00 con 0x558ee5739330 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff0729f5740 10 client.25033 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.241+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ee572c210 con 0x558ee5739330 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.242+0000 7ff04dffb640 1 -- 172.21.15.103:0/1826272852 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0600047a0 con 0x558ee5739330 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.242+0000 7ff04dffb640 1 -- 172.21.15.103:0/1826272852 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff05c0046c0 con 0x558ee5739330 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.242+0000 7ff0729f5740 10 client.25033 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.242+0000 7ff0729f5740 10 client.25033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.242+0000 7ff0729f5740 20 client.25033 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.243+0000 7ff0729f5740 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee572d620 0x558ee572fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.243+0000 7ff0729f5740 4 client.25033 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.243+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee56a2d60 con 0x558ee572d620 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.243+0000 7ff06e393640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee572d620 0x558ee572fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.243+0000 7ff06e393640 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee572d620 0x558ee572fa10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955c4e77e8d87b70 server_cookie=7b8f9530b0dafa27 in_seq=0 out_seq=0 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.243+0000 7ff04dffb640 10 client.25033 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff04dffb640 1 -- 172.21.15.103:0/1826272852 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ee56a2d60 con 0x558ee572d620 2024-08-19T01:36:15.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff04dffb640 10 client.25033 handle_command_reply: tid=0 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 client.25033 shutdown 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 msgr2=0x558ee5741090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee5741090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0cccc0 tx=0x7ff05c09b4e0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 shutdown_connections 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee572d620 0x558ee572fa10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5743bf0 0x558ee57415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee5741090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 --2- 172.21.15.103:0/1826272852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5740b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 >> 172.21.15.103:0/1826272852 conn(0x558ee5734fa0 msgr2=0x558ee572b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff04dffb640 0 client.25033 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:15.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 shutdown_connections 2024-08-19T01:36:15.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 1 -- 172.21.15.103:0/1826272852 wait complete. 2024-08-19T01:36:15.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.244+0000 7ff0729f5740 20 client.25033 trim_cache size 0 max 16384 2024-08-19T01:36:15.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.261+0000 7ff0729f5740 1 Processor -- start 2024-08-19T01:36:15.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.261+0000 7ff0729f5740 1 -- start start 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5734fa0 0x558ee5741420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5738a10 0x558ee573ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee573ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff0729f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff05400e1a0 con 0x558ee5734fa0 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff0729f5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff05400fce0 con 0x558ee5738a10 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff0729f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff05400e870 con 0x558ee5739330 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5734fa0 0x558ee5741420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5734fa0 0x558ee5741420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48028/0 (socket says 172.21.15.103:48028) 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 -- 172.21.15.103:0/944557128 learned_addr learned my addr 172.21.15.103:0/944557128 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06db92640 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5738a10 0x558ee573ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 -- 172.21.15.103:0/944557128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 msgr2=0x558ee573ddb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee573ddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 -- 172.21.15.103:0/944557128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5738a10 msgr2=0x558ee573ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5738a10 0x558ee573ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06e393640 1 -- 172.21.15.103:0/944557128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee573fb00 con 0x558ee5734fa0 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.262+0000 7ff06db92640 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5738a10 0x558ee573ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.263+0000 7ff06e393640 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5734fa0 0x558ee5741420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff058005fd0 tx=0x7ff058015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c736fd8bb9a709a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.263+0000 7ff04ffff640 1 -- 172.21.15.103:0/944557128 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05802e030 con 0x558ee5734fa0 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.263+0000 7ff04ffff640 1 -- 172.21.15.103:0/944557128 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff058010de0 con 0x558ee5734fa0 2024-08-19T01:36:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.263+0000 7ff04ffff640 1 -- 172.21.15.103:0/944557128 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05802ac50 con 0x558ee5734fa0 2024-08-19T01:36:15.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 -- 172.21.15.103:0/944557128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5734fa0 msgr2=0x558ee5741420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5734fa0 0x558ee5741420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff058005fd0 tx=0x7ff058015000 comp rx=0 tx=0).stop 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 -- 172.21.15.103:0/944557128 shutdown_connections 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5739330 0x558ee573ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5738a10 0x558ee573ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 --2- 172.21.15.103:0/944557128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5734fa0 0x558ee5741420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 -- 172.21.15.103:0/944557128 >> 172.21.15.103:0/944557128 conn(0x558ee5734640 msgr2=0x558ee573a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 -- 172.21.15.103:0/944557128 shutdown_connections 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 -- 172.21.15.103:0/944557128 wait complete. 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 -- start start 2024-08-19T01:36:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.264+0000 7ff0729f5740 1 Processor -- start 2024-08-19T01:36:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5734fa0 0x558ee5765ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5766400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff0729f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 0x558ee57bf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06db92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5766400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06db92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5766400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48044/0 (socket says 172.21.15.103:48044) 2024-08-19T01:36:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06db92640 1 -- 172.21.15.103:0/2745450305 learned_addr learned my addr 172.21.15.103:0/2745450305 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff05400f610 con 0x558ee5738a10 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff05400ef40 con 0x558ee5739330 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff05400c230 con 0x558ee5734fa0 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06e393640 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5734fa0 0x558ee5765ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06e393640 1 -- 172.21.15.103:0/2745450305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 msgr2=0x558ee57bf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06e393640 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 0x558ee57bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06e393640 1 -- 172.21.15.103:0/2745450305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 msgr2=0x558ee5766400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06e393640 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5766400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.265+0000 7ff06e393640 1 -- 172.21.15.103:0/2745450305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee57bfeb0 con 0x558ee5734fa0 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff06db92640 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5766400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff0729f5740 10 client.0 _ll_register_callbacks cb 0x7ff05c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff058010de0 con 0x558ee5734fa0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff06e393640 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5734fa0 0x558ee5765ec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff058013040 tx=0x7ff05801d9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e4d86de6e844cf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff04dffb640 10 client.25036 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff04dffb640 1 -- 172.21.15.103:0/2745450305 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058039070 con 0x558ee5734fa0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff04dffb640 1 -- 172.21.15.103:0/2745450305 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff05802a9d0 con 0x558ee5734fa0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff04dffb640 1 -- 172.21.15.103:0/2745450305 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff05802e040 con 0x558ee5734fa0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff0729f5740 10 client.25036 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.266+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=27}) -- 0x558ee57c7050 con 0x558ee5734fa0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff04dffb640 1 -- 172.21.15.103:0/2745450305 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff058010930 con 0x558ee5734fa0 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff0729f5740 10 client.25036 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff0729f5740 10 client.25036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff0729f5740 20 client.25036 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff0729f5740 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee5743bf0 0x558ee57c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff0729f5740 4 client.25036 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:36:15.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ee5735b50 con 0x558ee5743bf0 2024-08-19T01:36:15.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff06db92640 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee5743bf0 0x558ee57c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:15.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff04dffb640 10 client.25036 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:15.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.267+0000 7ff06db92640 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee5743bf0 0x558ee57c9740 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6f317efe7aa74e server_cookie=8a2190ac1abe2ef3 in_seq=0 out_seq=0 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.268+0000 7ff04dffb640 1 -- 172.21.15.103:0/2745450305 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ee5735b50 con 0x558ee5743bf0 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.268+0000 7ff04dffb640 10 client.25036 handle_command_reply: tid=0 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.268+0000 7ff0729f5740 1 client.25036 shutdown 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5734fa0 msgr2=0x558ee5765ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5734fa0 0x558ee5765ec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff058013040 tx=0x7ff05801d9c0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 shutdown_connections 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ee5743bf0 0x558ee57c9740 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff04dffb640 0 client.25036 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ee5739330 0x558ee57bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ee5738a10 0x558ee5766400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 --2- 172.21.15.103:0/2745450305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ee5734fa0 0x558ee5765ec0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 >> 172.21.15.103:0/2745450305 conn(0x558ee5734640 msgr2=0x558ee572b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 shutdown_connections 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 1 -- 172.21.15.103:0/2745450305 wait complete. 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.269+0000 7ff0729f5740 20 client.25036 trim_cache size 0 max 16384 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "207b1101-4b24-4d29-8763-30982763604b", 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:36:15.276 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0500a2c80 msgr2=0x7ff0500a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0500a2c80 0x7ff0500a5140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff058005c40 tx=0x7ff058005bd0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068103430 msgr2=0x7ff06819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068103430 0x7ff06819a790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00e9f0 tx=0x7ff05c00edf0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 shutdown_connections 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0500a2c80 0x7ff0500a5140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff06810beb0 0x7ff06819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff068103430 0x7ff06819a790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 --2- 172.21.15.103:0/3324288795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff068102b10 0x7ff06819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 >> 172.21.15.103:0/3324288795 conn(0x7ff0680fc4f0 msgr2=0x7ff0681006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:15.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 shutdown_connections 2024-08-19T01:36:15.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:15.270+0000 7ff0705f7640 1 -- 172.21.15.103:0/3324288795 wait complete. 2024-08-19T01:36:16.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:15 smithi103 ceph-mon[18436]: pgmap v353: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:16.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:15 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:36:16.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:15 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:16.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:15 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:36:16.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:15 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:16.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:36:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:15 smithi190 ceph-mon[22032]: pgmap v353: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:15 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:36:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:15 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:15 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:36:16.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:15 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:16.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:15 smithi146 ceph-mon[22098]: pgmap v353: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:15 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:36:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:15 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:15 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:36:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:15 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:16.385 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:16.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 -- 172.21.15.103:0/895667620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c100b50 msgr2=0x7fc71c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 --2- 172.21.15.103:0/895667620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c100b50 0x7fc71c100f30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc718009900 tx=0x7fc71802a480 comp rx=0 tx=0).stop 2024-08-19T01:36:16.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 -- 172.21.15.103:0/895667620 shutdown_connections 2024-08-19T01:36:16.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 --2- 172.21.15.103:0/895667620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc71c102c20 0x7fc71c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 --2- 172.21.15.103:0/895667620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c1022a0 0x7fc71c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 --2- 172.21.15.103:0/895667620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c100b50 0x7fc71c100f30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 -- 172.21.15.103:0/895667620 >> 172.21.15.103:0/895667620 conn(0x7fc71c0fc500 msgr2=0x7fc71c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:16.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 -- 172.21.15.103:0/895667620 shutdown_connections 2024-08-19T01:36:16.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.987+0000 7fc723f52640 1 -- 172.21.15.103:0/895667620 wait complete. 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.988+0000 7fc723f52640 1 Processor -- start 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.988+0000 7fc723f52640 1 -- start start 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.988+0000 7fc723f52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c100b50 0x7fc71c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.988+0000 7fc723f52640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc71c1022a0 0x7fc71c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.988+0000 7fc723f52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102c20 0x7fc71c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc723f52640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c111cb0 con 0x7fc71c102c20 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc723f52640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c111b30 con 0x7fc71c1022a0 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc723f52640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c111e30 con 0x7fc71c100b50 2024-08-19T01:36:16.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c100b50 0x7fc71c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c100b50 0x7fc71c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:47026/0 (socket says 172.21.15.103:47026) 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 -- 172.21.15.103:0/3938814148 learned_addr learned my addr 172.21.15.103:0/3938814148 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc7224ef640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102c20 0x7fc71c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc7214ed640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc71c1022a0 0x7fc71c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 -- 172.21.15.103:0/3938814148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc71c1022a0 msgr2=0x7fc71c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc71c1022a0 0x7fc71c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 -- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102c20 msgr2=0x7fc71c19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102c20 0x7fc71c19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 -- 172.21.15.103:0/3938814148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c19d220 con 0x7fc71c100b50 2024-08-19T01:36:16.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc721cee640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c100b50 0x7fc71c198270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc71802aa70 tx=0x7fc718003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:16.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc712ffd640 1 -- 172.21.15.103:0/3938814148 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71803f070 con 0x7fc71c100b50 2024-08-19T01:36:16.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.989+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c19d4b0 con 0x7fc71c100b50 2024-08-19T01:36:16.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.990+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c1a4df0 con 0x7fc71c100b50 2024-08-19T01:36:16.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.990+0000 7fc712ffd640 1 -- 172.21.15.103:0/3938814148 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc71803bb20 con 0x7fc71c100b50 2024-08-19T01:36:16.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.990+0000 7fc712ffd640 1 -- 172.21.15.103:0/3938814148 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718035030 con 0x7fc71c100b50 2024-08-19T01:36:16.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.992+0000 7fc712ffd640 1 -- 172.21.15.103:0/3938814148 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc718042070 con 0x7fc71c100b50 2024-08-19T01:36:16.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.992+0000 7fc712ffd640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7080a2c00 0x7fc7080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.993+0000 7fc712ffd640 1 -- 172.21.15.103:0/3938814148 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc7180ef400 con 0x7fc71c100b50 2024-08-19T01:36:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.993+0000 7fc7214ed640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7080a2c00 0x7fc7080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.993+0000 7fc726350740 1 Processor -- start 2024-08-19T01:36:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.993+0000 7fc726350740 1 -- start start 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc7214ed640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7080a2c00 0x7fc7080a50c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ffcd0 tx=0x7fc70c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc726350740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f9717187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc726350740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971712dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc726350740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971713860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc726350740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f97166e980 con 0x55f971718d70 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc726350740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f971606a70 con 0x55f9717133e0 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc726350740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f97166a940 con 0x55f9717183c0 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f9717187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f9717187a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:47032/0 (socket says 172.21.15.103:47032) 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 -- 172.21.15.103:0/1137112131 learned_addr learned my addr 172.21.15.103:0/1137112131 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc7214ed640 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971712dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971713860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 -- 172.21.15.103:0/1137112131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 msgr2=0x55f971713860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971713860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 -- 172.21.15.103:0/1137112131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 msgr2=0x55f971712dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971712dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc721cee640 1 -- 172.21.15.103:0/1137112131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9716696e0 con 0x55f9717183c0 2024-08-19T01:36:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.994+0000 7fc7214ed640 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971712dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.995+0000 7fc721cee640 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f9717187a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7180b81a0 tx=0x7fc718009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6488cc3b900f2b11 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.995+0000 7fc707fff640 1 -- 172.21.15.103:0/1137112131 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718049070 con 0x55f9717183c0 2024-08-19T01:36:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.995+0000 7fc707fff640 1 -- 172.21.15.103:0/1137112131 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7180ba040 con 0x55f9717183c0 2024-08-19T01:36:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.995+0000 7fc707fff640 1 -- 172.21.15.103:0/1137112131 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7180c4470 con 0x55f9717183c0 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 -- 172.21.15.103:0/1137112131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 msgr2=0x55f9717187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f9717187a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7180b81a0 tx=0x7fc718009260 comp rx=0 tx=0).stop 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 -- 172.21.15.103:0/1137112131 shutdown_connections 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971713860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971712dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 --2- 172.21.15.103:0/1137112131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f9717187a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 -- 172.21.15.103:0/1137112131 >> 172.21.15.103:0/1137112131 conn(0x55f971705250 msgr2=0x55f971705630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 -- 172.21.15.103:0/1137112131 shutdown_connections 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.996+0000 7fc726350740 1 -- 172.21.15.103:0/1137112131 wait complete. 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.997+0000 7fc726350740 1 -- start start 2024-08-19T01:36:16.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.997+0000 7fc726350740 1 Processor -- start 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.997+0000 7fc726350740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f97171c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.997+0000 7fc726350740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9717183c0 0x55f97171c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9717183c0 0x55f97171c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9717183c0 0x55f97171c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48104/0 (socket says 172.21.15.103:48104) 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 -- 172.21.15.103:0/1250917605 learned_addr learned my addr 172.21.15.103:0/1250917605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc721cee640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f97171c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc726350740 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f971718d70 0x55f97171cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f97171f550 con 0x55f9717183c0 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f971629aa0 con 0x55f971718d70 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f97171f6d0 con 0x55f9717133e0 2024-08-19T01:36:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f971718d70 0x55f97171cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 -- 172.21.15.103:0/1250917605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 msgr2=0x55f97171c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f97171c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 -- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f971718d70 msgr2=0x55f97171cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f971718d70 0x55f97171cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc7214ed640 1 -- 172.21.15.103:0/1250917605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f97160d650 con 0x55f9717183c0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.998+0000 7fc721cee640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f97171c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc7214ed640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9717183c0 0x55f97171c760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc70c003970 tx=0x7fc70c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e15fb2771e13706 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc7057fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc726350740 10 client.0 _ll_register_callbacks cb 0x55f971618e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7180ba040 con 0x55f9717183c0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc7057fa640 1 -- 172.21.15.103:0/1250917605 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c01fdc0 con 0x55f9717183c0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc7057fa640 1 -- 172.21.15.103:0/1250917605 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc70c028440 con 0x55f9717183c0 2024-08-19T01:36:17.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc7057fa640 1 -- 172.21.15.103:0/1250917605 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc70c028640 con 0x55f9717183c0 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc726350740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55f9713e03c0 con 0x55f9717183c0 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:16.999+0000 7fc7057fa640 1 -- 172.21.15.103:0/1250917605 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc70c026070 con 0x55f9717183c0 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.000+0000 7fc726350740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.000+0000 7fc726350740 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.000+0000 7fc726350740 20 client.15480 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.000+0000 7fc726350740 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f97172c330 0x55f97172e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.000+0000 7fc726350740 4 client.15480 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.000+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f97167bef0 con 0x55f97172c330 2024-08-19T01:36:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.000+0000 7fc721cee640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f97172c330 0x55f97172e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.001+0000 7fc721cee640 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f97172c330 0x55f97172e7c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b043ab581440516e server_cookie=897d71c76f7e52be in_seq=0 out_seq=0 2024-08-19T01:36:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.001+0000 7fc7057fa640 10 client.15480 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.001+0000 7fc7057fa640 1 -- 172.21.15.103:0/1250917605 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f97167bef0 con 0x55f97172c330 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.001+0000 7fc7057fa640 10 client.15480 handle_command_reply: tid=0 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.001+0000 7fc726350740 1 client.15480 shutdown 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.001+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9717183c0 msgr2=0x55f97171c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.001+0000 7fc726350740 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9717183c0 0x55f97171c760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc70c003970 tx=0x7fc70c004730 comp rx=0 tx=0).stop 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 shutdown_connections 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f97172c330 0x55f97172e7c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f971718d70 0x55f97171cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9717183c0 0x55f97171c760 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 --2- 172.21.15.103:0/1250917605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f97171c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc7057fa640 0 client.15480 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 >> 172.21.15.103:0/1250917605 conn(0x55f971705250 msgr2=0x55f971704900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 shutdown_connections 2024-08-19T01:36:17.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 1 -- 172.21.15.103:0/1250917605 wait complete. 2024-08-19T01:36:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.002+0000 7fc726350740 20 client.15480 trim_cache size 0 max 16384 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.034+0000 7fc726350740 1 Processor -- start 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.034+0000 7fc726350740 1 -- start start 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc726350740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971712dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc726350740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f97171e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc726350740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971717c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971717c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971717c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48120/0 (socket says 172.21.15.103:48120) 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc70000e390 con 0x55f971718d70 2024-08-19T01:36:17.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 -- 172.21.15.103:0/1404037188 learned_addr learned my addr 172.21.15.103:0/1404037188 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc70000fed0 con 0x55f9717133e0 2024-08-19T01:36:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc70000ea60 con 0x55f9717183c0 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc721cee640 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971712dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 -- 172.21.15.103:0/1404037188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 msgr2=0x55f97171e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f97171e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 -- 172.21.15.103:0/1404037188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 msgr2=0x55f971712dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971712dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 -- 172.21.15.103:0/1404037188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f97171b230 con 0x55f971718d70 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.035+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971717c00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc714011570 tx=0x7fc714011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c88bf82b988ca62 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.036+0000 7fc7077fe640 1 -- 172.21.15.103:0/1404037188 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714020070 con 0x55f971718d70 2024-08-19T01:36:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.036+0000 7fc7077fe640 1 -- 172.21.15.103:0/1404037188 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc714010c00 con 0x55f971718d70 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.036+0000 7fc7077fe640 1 -- 172.21.15.103:0/1404037188 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71401bd10 con 0x55f971718d70 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 msgr2=0x55f971717c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971717c00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc714011570 tx=0x7fc714011a30 comp rx=0 tx=0).stop 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 shutdown_connections 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f971717c00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717183c0 0x55f97171e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 --2- 172.21.15.103:0/1404037188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717133e0 0x55f971712dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 >> 172.21.15.103:0/1404037188 conn(0x55f971705250 msgr2=0x55f9717040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 shutdown_connections 2024-08-19T01:36:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 -- 172.21.15.103:0/1404037188 wait complete. 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 -- start start 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.037+0000 7fc726350740 1 Processor -- start 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc726350740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f971733d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc726350740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717183c0 0x55f971734280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc726350740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f9717347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc726350740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc70000f800 con 0x55f971718d70 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc726350740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc70000f130 con 0x55f9717183c0 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f9717347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc726350740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc70000c370 con 0x55f9717133e0 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f9717347c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48130/0 (socket says 172.21.15.103:48130) 2024-08-19T01:36:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 -- 172.21.15.103:0/1114957323 learned_addr learned my addr 172.21.15.103:0/1114957323 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7214ed640 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717183c0 0x55f971734280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc721cee640 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f971733d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc726350740 10 client.0 _ll_register_callbacks cb 0x7fc70c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 -- 172.21.15.103:0/1114957323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 msgr2=0x55f971733d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f971733d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 -- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717183c0 msgr2=0x55f971734280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717183c0 0x55f971734280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.038+0000 7fc7224ef640 1 -- 172.21.15.103:0/1114957323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9717960f0 con 0x55f971718d70 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc7224ef640 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f9717347c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc714010fe0 tx=0x7fc714009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4dc058adea962b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc7057fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc7057fa640 1 -- 172.21.15.103:0/1114957323 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714029060 con 0x55f971718d70 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc7057fa640 1 -- 172.21.15.103:0/1114957323 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc714004c40 con 0x55f971718d70 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc714010c00 con 0x55f971718d70 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc7057fa640 1 -- 172.21.15.103:0/1114957323 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc714020040 con 0x55f971718d70 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc726350740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f9717973a0 con 0x55f971718d70 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.039+0000 7fc7057fa640 1 -- 172.21.15.103:0/1114957323 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc714004e40 con 0x55f971718d70 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc7057fa640 1 -- 172.21.15.103:0/1114957323 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc71401a070 con 0x55f971718d70 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc726350740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc726350740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc726350740 20 client.15486 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc726350740 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f971797990 0x55f971799db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc726350740 4 client.15486 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:36:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc714010c00 con 0x55f971797990 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.040+0000 7fc721cee640 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f971797990 0x55f971799db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.041+0000 7fc721cee640 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f971797990 0x55f971799db0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6d63fbafc94d25 server_cookie=bb26f1a786ed5e14 in_seq=0 out_seq=0 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.041+0000 7fc7057fa640 10 client.15486 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.041+0000 7fc7057fa640 1 -- 172.21.15.103:0/1114957323 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc714010c00 con 0x55f971797990 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.041+0000 7fc7057fa640 10 client.15486 handle_command_reply: tid=0 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.041+0000 7fc726350740 1 client.15486 shutdown 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.041+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 msgr2=0x55f9717347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.041+0000 7fc726350740 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f9717347c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc714010fe0 tx=0x7fc714009d60 comp rx=0 tx=0).stop 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 shutdown_connections 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f971797990 0x55f971799db0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc7057fa640 0 client.15486 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f971718d70 0x55f9717347c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9717183c0 0x55f971734280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 --2- 172.21.15.103:0/1114957323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9717133e0 0x55f971733d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 >> 172.21.15.103:0/1114957323 conn(0x55f971705250 msgr2=0x55f971704900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 shutdown_connections 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 1 -- 172.21.15.103:0/1114957323 wait complete. 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:36:17.049 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:36:17.050 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:36:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.042+0000 7fc726350740 20 client.15486 trim_cache size 0 max 16384 2024-08-19T01:36:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7080a2c00 msgr2=0x7fc7080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7080a2c00 0x7fc7080a50c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ffcd0 tx=0x7fc70c005f90 comp rx=0 tx=0).stop 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c100b50 msgr2=0x7fc71c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c100b50 0x7fc71c198270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc71802aa70 tx=0x7fc718003c10 comp rx=0 tx=0).stop 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 shutdown_connections 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7080a2c00 0x7fc7080a50c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102c20 0x7fc71c19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc71c1022a0 0x7fc71c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 --2- 172.21.15.103:0/3938814148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc71c100b50 0x7fc71c198270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 >> 172.21.15.103:0/3938814148 conn(0x7fc71c0fc500 msgr2=0x7fc71c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 shutdown_connections 2024-08-19T01:36:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:17.043+0000 7fc723f52640 1 -- 172.21.15.103:0/3938814148 wait complete. 2024-08-19T01:36:17.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:16 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:36:17.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:16 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:36:17.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:16 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:36:17.928 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:36:17.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:36:18.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:17 smithi146 ceph-mon[22098]: pgmap v354: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:18.205 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:18.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:17 smithi103 ceph-mon[18436]: pgmap v354: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:17 smithi190 ceph-mon[22032]: pgmap v354: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-19T01:36:18.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 -- 172.21.15.103:0/343126329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8103410 msgr2=0x7fa6f81078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/343126329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8103410 0x7fa6f81078c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009930 tx=0x7fa6e802a420 comp rx=0 tx=0).stop 2024-08-19T01:36:18.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 -- 172.21.15.103:0/343126329 shutdown_connections 2024-08-19T01:36:18.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/343126329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f8107fd0 0x7fa6f810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/343126329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8103410 0x7fa6f81078c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/343126329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8102a60 0x7fa6f8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 -- 172.21.15.103:0/343126329 >> 172.21.15.103:0/343126329 conn(0x7fa6f80fc500 msgr2=0x7fa6f80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 -- 172.21.15.103:0/343126329 shutdown_connections 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.731+0000 7fa6ffd73640 1 -- 172.21.15.103:0/343126329 wait complete. 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 Processor -- start 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 -- start start 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8102a60 0x7fa6f81005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8103410 0x7fa6f8100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f8107fd0 0x7fa6f8101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8111c80 con 0x7fa6f8102a60 2024-08-19T01:36:18.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8111b00 con 0x7fa6f8103410 2024-08-19T01:36:18.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6ffd73640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8111e00 con 0x7fa6f8107fd0 2024-08-19T01:36:18.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.732+0000 7fa6fd30e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8103410 0x7fa6f8100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8103410 0x7fa6f8100b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:45700/0 (socket says 172.21.15.103:45700) 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 -- 172.21.15.103:0/1860651419 learned_addr learned my addr 172.21.15.103:0/1860651419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fdb0f640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8102a60 0x7fa6f81005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fe310640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f8107fd0 0x7fa6f8101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 -- 172.21.15.103:0/1860651419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f8107fd0 msgr2=0x7fa6f8101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f8107fd0 0x7fa6f8101070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 -- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8102a60 msgr2=0x7fa6f81005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8102a60 0x7fa6f81005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 -- 172.21.15.103:0/1860651419 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f8072b70 con 0x7fa6f8103410 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.733+0000 7fa6fd30e640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8103410 0x7fa6f8100b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8031440 tx=0x7fa6e8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.734+0000 7fa6eeffd640 1 -- 172.21.15.103:0/1860651419 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e803f030 con 0x7fa6f8103410 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.734+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f8072e00 con 0x7fa6f8103410 2024-08-19T01:36:18.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.734+0000 7fa6eeffd640 1 -- 172.21.15.103:0/1860651419 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6e8003aa0 con 0x7fa6f8103410 2024-08-19T01:36:18.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.734+0000 7fa6eeffd640 1 -- 172.21.15.103:0/1860651419 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8035070 con 0x7fa6f8103410 2024-08-19T01:36:18.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.734+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f8073330 con 0x7fa6f8103410 2024-08-19T01:36:18.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.735+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f8102e40 con 0x7fa6f8103410 2024-08-19T01:36:18.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.736+0000 7fa6eeffd640 1 -- 172.21.15.103:0/1860651419 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa6e8042030 con 0x7fa6f8103410 2024-08-19T01:36:18.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.737+0000 7fa6eeffd640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa6e40a2b80 0x7fa6e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:18.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.737+0000 7fa6eeffd640 1 -- 172.21.15.103:0/1860651419 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa6e80ef680 con 0x7fa6f8103410 2024-08-19T01:36:18.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.737+0000 7fa6fdb0f640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa6e40a2b80 0x7fa6e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:18.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.738+0000 7fa6fdb0f640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa6e40a2b80 0x7fa6e40a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009bd0 tx=0x7fa6f4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:18.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:18.741+0000 7fa6eeffd640 1 -- 172.21.15.103:0/1860651419 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa6e80bbf60 con 0x7fa6f8103410 2024-08-19T01:36:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.159+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6f8073d50 con 0x7fa6f8103410 2024-08-19T01:36:19.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.160+0000 7fa6eeffd640 1 -- 172.21.15.103:0/1860651419 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7fa6e80c0260 con 0x7fa6f8103410 2024-08-19T01:36:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:36:19.163 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-08-19T01:35:19:962857+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:35:19.321960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:36:19.163 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-08-19T01:36:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa6e40a2b80 msgr2=0x7fa6e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa6e40a2b80 0x7fa6e40a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009bd0 tx=0x7fa6f4005f90 comp rx=0 tx=0).stop 2024-08-19T01:36:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8103410 msgr2=0x7fa6f8100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8103410 0x7fa6f8100b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8031440 tx=0x7fa6e8004270 comp rx=0 tx=0).stop 2024-08-19T01:36:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 shutdown_connections 2024-08-19T01:36:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa6e40a2b80 0x7fa6e40a5040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6f8107fd0 0x7fa6f8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa6f8103410 0x7fa6f8100b30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 --2- 172.21.15.103:0/1860651419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8102a60 0x7fa6f81005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 >> 172.21.15.103:0/1860651419 conn(0x7fa6f80fc500 msgr2=0x7fa6f8105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 shutdown_connections 2024-08-19T01:36:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:19.163+0000 7fa6ffd73640 1 -- 172.21.15.103:0/1860651419 wait complete. 2024-08-19T01:36:20.037 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:36:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:19 smithi146 ceph-mon[22098]: pgmap v355: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:20.244 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:19 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1860651419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:20.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:19 smithi103 ceph-mon[18436]: pgmap v355: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:20.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:19 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1860651419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:20.311 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:19 smithi190 ceph-mon[22032]: pgmap v355: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:20.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:19 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1860651419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:36:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 -- 172.21.15.103:0/2873563917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d0102b10 msgr2=0x7fa0d0102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/2873563917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d0102b10 0x7fa0d0102f90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009900 tx=0x7fa0b802a410 comp rx=0 tx=0).stop 2024-08-19T01:36:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 -- 172.21.15.103:0/2873563917 shutdown_connections 2024-08-19T01:36:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/2873563917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d01034d0 0x7fa0d0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/2873563917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d0102b10 0x7fa0d0102f90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/2873563917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d00ffb40 0x7fa0d00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 -- 172.21.15.103:0/2873563917 >> 172.21.15.103:0/2873563917 conn(0x7fa0d00fb660 msgr2=0x7fa0d00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 -- 172.21.15.103:0/2873563917 shutdown_connections 2024-08-19T01:36:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.571+0000 7fa0d48fc640 1 -- 172.21.15.103:0/2873563917 wait complete. 2024-08-19T01:36:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.572+0000 7fa0d48fc640 1 Processor -- start 2024-08-19T01:36:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.572+0000 7fa0d48fc640 1 -- start start 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0d48fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d00ffb40 0x7fa0d0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0d48fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0102b10 0x7fa0d0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0d48fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d01034d0 0x7fa0d019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d00ffb40 0x7fa0d0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d00ffb40 0x7fa0d0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48166/0 (socket says 172.21.15.103:48166) 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0ce59c640 1 -- 172.21.15.103:0/88365040 learned_addr learned my addr 172.21.15.103:0/88365040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0102b10 0x7fa0d0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d010d410 con 0x7fa0d00ffb40 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d010d290 con 0x7fa0d01034d0 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d010d590 con 0x7fa0d0102b10 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.573+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d01034d0 0x7fa0d019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c5d9b640 1 -- 172.21.15.103:0/88365040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d01034d0 msgr2=0x7fa0d019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d01034d0 0x7fa0d019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c5d9b640 1 -- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d00ffb40 msgr2=0x7fa0d0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d00ffb40 0x7fa0d0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c5d9b640 1 -- 172.21.15.103:0/88365040 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d019f640 con 0x7fa0d0102b10 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d00ffb40 0x7fa0d0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0102b10 0x7fa0d0197f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0b802a930 tx=0x7fa0b8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c7fff640 1 -- 172.21.15.103:0/88365040 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b803f070 con 0x7fa0d0102b10 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d019f8d0 con 0x7fa0d0102b10 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c7fff640 1 -- 172.21.15.103:0/88365040 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0b8031c40 con 0x7fa0d0102b10 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0c7fff640 1 -- 172.21.15.103:0/88365040 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8035030 con 0x7fa0d0102b10 2024-08-19T01:36:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.574+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d019fde0 con 0x7fa0d0102b10 2024-08-19T01:36:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.578+0000 7fa0c7fff640 1 -- 172.21.15.103:0/88365040 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa0b8042070 con 0x7fa0d0102b10 2024-08-19T01:36:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.578+0000 7fa0d76b2740 1 Processor -- start 2024-08-19T01:36:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0c7fff640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0bc0a2cb0 0x7fa0bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0bc0a2cb0 0x7fa0bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0d76b2740 1 -- start start 2024-08-19T01:36:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0c7fff640 1 -- 172.21.15.103:0/88365040 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa0b80ef5c0 con 0x7fa0d0102b10 2024-08-19T01:36:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5573541339d0 con 0x5573541d4a90 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5573540db820 con 0x5573541d53d0 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55735412f990 con 0x5573541da460 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.579+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541d4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48172/0 (socket says 172.21.15.103:48172) 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 -- 172.21.15.103:0/384451580 learned_addr learned my addr 172.21.15.103:0/384451580 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0bc0a2cb0 0x7fa0bc0a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005fd0 tx=0x7fa0c8009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 -- 172.21.15.103:0/384451580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 msgr2=0x5573541dc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541dc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 -- 172.21.15.103:0/384451580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 msgr2=0x5573541d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 -- 172.21.15.103:0/384451580 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557353e47210 con 0x5573541d4a90 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541d4e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00115c0 tx=0x7fa0c0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ff748d6cb728 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0c4d99640 1 -- 172.21.15.103:0/384451580 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0005020 con 0x5573541d4a90 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0c4d99640 1 -- 172.21.15.103:0/384451580 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c00047a0 con 0x5573541d4a90 2024-08-19T01:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.580+0000 7fa0c4d99640 1 -- 172.21.15.103:0/384451580 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000f920 con 0x5573541d4a90 2024-08-19T01:36:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.581+0000 7fa0d76b2740 1 -- 172.21.15.103:0/384451580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 msgr2=0x5573541d4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.581+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541d4e90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00115c0 tx=0x7fa0c0011a80 comp rx=0 tx=0).stop 2024-08-19T01:36:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 -- 172.21.15.103:0/384451580 shutdown_connections 2024-08-19T01:36:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/384451580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541d4e90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 -- 172.21.15.103:0/384451580 >> 172.21.15.103:0/384451580 conn(0x5573541cbcf0 msgr2=0x5573541cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 -- 172.21.15.103:0/384451580 shutdown_connections 2024-08-19T01:36:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 -- 172.21.15.103:0/384451580 wait complete. 2024-08-19T01:36:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 -- start start 2024-08-19T01:36:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.582+0000 7fa0d76b2740 1 Processor -- start 2024-08-19T01:36:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5573541dd430 con 0x5573541d4a90 2024-08-19T01:36:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5573540ef520 con 0x5573541d53d0 2024-08-19T01:36:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5573541dd5b0 con 0x5573541da460 2024-08-19T01:36:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0d76b2740 10 client.0 _ll_register_callbacks cb 0x557353e46a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541decb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48176/0 (socket says 172.21.15.103:48176) 2024-08-19T01:36:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 -- 172.21.15.103:0/3666771692 learned_addr learned my addr 172.21.15.103:0/3666771692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 -- 172.21.15.103:0/3666771692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 msgr2=0x5573541e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 -- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 msgr2=0x5573541df1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541df1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 -- 172.21.15.103:0/3666771692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557353f13620 con 0x5573541d4a90 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541decb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009350 tx=0x7fa0c00051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae84ebac773f480 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.583+0000 7fa0b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/3666771692 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c001d050 con 0x5573541d4a90 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/3666771692 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa0c000f660 con 0x5573541d4a90 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c00047a0 con 0x5573541d4a90 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/3666771692 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0c000f880 con 0x5573541d4a90 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573541e8f90 con 0x5573541d4a90 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/3666771692 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c00047a0 con 0x5573541d4a90 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/3666771692 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa0c0030a00 con 0x5573541d4a90 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 20 client.15504 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5573541eaa10 0x5573541ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 4 client.15504 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.584+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557354141100 con 0x5573541eaa10 2024-08-19T01:36:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.585+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5573541eaa10 0x5573541ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.585+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5573541eaa10 0x5573541ece90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e5df7b65cee6c3e server_cookie=6d645804f80db8e6 in_seq=0 out_seq=0 2024-08-19T01:36:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.585+0000 7fa0b5ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/3666771692 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557354141100 con 0x5573541eaa10 2024-08-19T01:36:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0b5ffb640 10 client.15504 handle_command_reply: tid=0 2024-08-19T01:36:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 client.15504 shutdown 2024-08-19T01:36:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 msgr2=0x5573541decb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541decb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009350 tx=0x7fa0c00051f0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 shutdown_connections 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5573541eaa10 0x5573541ece90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541da460 0x5573541e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541d53d0 0x5573541df1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3666771692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541d4a90 0x5573541decb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 >> 172.21.15.103:0/3666771692 conn(0x5573541cbcf0 msgr2=0x5573541cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.586+0000 7fa0b5ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.587+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 shutdown_connections 2024-08-19T01:36:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.587+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3666771692 wait complete. 2024-08-19T01:36:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.587+0000 7fa0d76b2740 20 client.15504 trim_cache size 0 max 16384 2024-08-19T01:36:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 Processor -- start 2024-08-19T01:36:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 -- start start 2024-08-19T01:36:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a6940 con 0x5573541c6920 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a8480 con 0x5573541da460 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a7010 con 0x5573541d53d0 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48188/0 (socket says 172.21.15.103:48188) 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0ced9d640 1 -- 172.21.15.103:0/3183582614 learned_addr learned my addr 172.21.15.103:0/3183582614 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.615+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0ced9d640 1 -- 172.21.15.103:0/3183582614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 msgr2=0x5573541df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0ced9d640 1 -- 172.21.15.103:0/3183582614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 msgr2=0x5573541e6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0ced9d640 1 -- 172.21.15.103:0/3183582614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573541d6130 con 0x5573541c6920 2024-08-19T01:36:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541ccb70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa0c801f6e0 tx=0x7fa0c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3894bc627765994f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0b7fff640 1 -- 172.21.15.103:0/3183582614 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c801f110 con 0x5573541c6920 2024-08-19T01:36:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0b7fff640 1 -- 172.21.15.103:0/3183582614 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c801f2b0 con 0x5573541c6920 2024-08-19T01:36:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.616+0000 7fa0b7fff640 1 -- 172.21.15.103:0/3183582614 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80044c0 con 0x5573541c6920 2024-08-19T01:36:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.617+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3183582614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 msgr2=0x5573541ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.617+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541ccb70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa0c801f6e0 tx=0x7fa0c8014000 comp rx=0 tx=0).stop 2024-08-19T01:36:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.618+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3183582614 shutdown_connections 2024-08-19T01:36:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.618+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541ccb70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.618+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.618+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/3183582614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.618+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3183582614 >> 172.21.15.103:0/3183582614 conn(0x5573541cbcf0 msgr2=0x5573541dea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3183582614 shutdown_connections 2024-08-19T01:36:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 -- 172.21.15.103:0/3183582614 wait complete. 2024-08-19T01:36:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 -- start start 2024-08-19T01:36:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 Processor -- start 2024-08-19T01:36:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a7db0 con 0x5573541c6920 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a76e0 con 0x5573541da460 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01a4940 con 0x5573541d53d0 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.620+0000 7fa0d76b2740 10 client.0 _ll_register_callbacks cb 0x7fa0c0011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541e3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48204/0 (socket says 172.21.15.103:48204) 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 -- 172.21.15.103:0/1017376115 learned_addr learned my addr 172.21.15.103:0/1017376115 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 -- 172.21.15.103:0/1017376115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 msgr2=0x5573541e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 -- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 msgr2=0x5573541e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 -- 172.21.15.103:0/1017376115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573541d4410 con 0x5573541c6920 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ced9d640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0ce59c640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541e3a70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0011ac0 tx=0x7fa0c0004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7b4d58c83b446c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.621+0000 7fa0b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/1017376115 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c001d040 con 0x5573541c6920 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/1017376115 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa0c0004040 con 0x5573541c6920 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c801f2b0 con 0x5573541c6920 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/1017376115 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0c0004e10 con 0x5573541c6920 2024-08-19T01:36:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0d76b2740 10 client.15510 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557354259f60 con 0x5573541c6920 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/1017376115 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c801f2b0 con 0x5573541c6920 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/1017376115 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa0c0004260 con 0x5573541c6920 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0d76b2740 10 client.15510 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.622+0000 7fa0d76b2740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.623+0000 7fa0d76b2740 20 client.15510 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.623+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55735425a6d0 0x55735425caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.623+0000 7fa0d76b2740 4 client.15510 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.623+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55735413b450 con 0x55735425a6d0 2024-08-19T01:36:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.623+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55735425a6d0 0x55735425caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.623+0000 7fa0b5ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.623+0000 7fa0c5d9b640 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55735425a6d0 0x55735425caf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1750b4f909f0148b server_cookie=7538638c3d3f6c99 in_seq=0 out_seq=0 2024-08-19T01:36:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0b5ffb640 1 -- 172.21.15.103:0/1017376115 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55735413b450 con 0x55735425a6d0 2024-08-19T01:36:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0b5ffb640 10 client.15510 handle_command_reply: tid=0 2024-08-19T01:36:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 client.15510 shutdown 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 msgr2=0x5573541e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541e3a70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0011ac0 tx=0x7fa0c0004cf0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 shutdown_connections 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55735425a6d0 0x55735425caf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5573541da460 0x5573541e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573541d53d0 0x5573541e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0b5ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 --2- 172.21.15.103:0/1017376115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573541c6920 0x5573541e3a70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 >> 172.21.15.103:0/1017376115 conn(0x5573541cbcf0 msgr2=0x5573541ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 shutdown_connections 2024-08-19T01:36:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.627+0000 7fa0d76b2740 1 -- 172.21.15.103:0/1017376115 wait complete. 2024-08-19T01:36:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d76b2740 20 client.15510 trim_cache size 0 max 16384 2024-08-19T01:36:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:36:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:36:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:36:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:36:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0bc0a2cb0 msgr2=0x7fa0bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0bc0a2cb0 0x7fa0bc0a5170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005fd0 tx=0x7fa0c8009500 comp rx=0 tx=0).stop 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0102b10 msgr2=0x7fa0d0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0102b10 0x7fa0d0197f60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0b802a930 tx=0x7fa0b8003c10 comp rx=0 tx=0).stop 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 shutdown_connections 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0bc0a2cb0 0x7fa0bc0a5170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d01034d0 0x7fa0d019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0d0102b10 0x7fa0d0197f60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 --2- 172.21.15.103:0/88365040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d00ffb40 0x7fa0d0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 >> 172.21.15.103:0/88365040 conn(0x7fa0d00fb660 msgr2=0x7fa0d0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 shutdown_connections 2024-08-19T01:36:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:20.628+0000 7fa0d48fc640 1 -- 172.21.15.103:0/88365040 wait complete. 2024-08-19T01:36:20.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 -- 172.21.15.103:0/1830584933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32401007d0 msgr2=0x7f324010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 --2- 172.21.15.103:0/1830584933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32401007d0 0x7f324010e750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3238009ca0 tx=0x7f323802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 -- 172.21.15.103:0/1830584933 shutdown_connections 2024-08-19T01:36:20.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 --2- 172.21.15.103:0/1830584933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32401007d0 0x7f324010e750 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 --2- 172.21.15.103:0/1830584933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32400ffe30 0x7f3240100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 --2- 172.21.15.103:0/1830584933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32401068b0 0x7f3240106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 -- 172.21.15.103:0/1830584933 >> 172.21.15.103:0/1830584933 conn(0x7f3240076f60 msgr2=0x7f32400fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 -- 172.21.15.103:0/1830584933 shutdown_connections 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.854+0000 7f3248642640 1 -- 172.21.15.103:0/1830584933 wait complete. 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.855+0000 7f3248642640 1 Processor -- start 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.855+0000 7f3248642640 1 -- start start 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3248642640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32400ffe30 0x7f324019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3248642640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32401007d0 0x7f32401a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3248642640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32401068b0 0x7f3240199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3248642640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3240101cf0 con 0x7f32400ffe30 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3248642640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3240101b70 con 0x7f32401007d0 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3248642640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3240101e70 con 0x7f32401068b0 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32400ffe30 0x7f324019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32400ffe30 0x7f324019fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48210/0 (socket says 172.21.15.103:48210) 2024-08-19T01:36:20.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 -- 172.21.15.103:0/309258185 learned_addr learned my addr 172.21.15.103:0/309258185 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3245bdd640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32401007d0 0x7f32401a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f3246bdf640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32401068b0 0x7f3240199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 -- 172.21.15.103:0/309258185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32401068b0 msgr2=0x7f3240199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32401068b0 0x7f3240199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 -- 172.21.15.103:0/309258185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32401007d0 msgr2=0x7f32401a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32401007d0 0x7f32401a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.856+0000 7f32463de640 1 -- 172.21.15.103:0/309258185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324019a770 con 0x7f32400ffe30 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.857+0000 7f32463de640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32400ffe30 0x7f324019fe70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f323c00da90 tx=0x7f323c00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.857+0000 7f32377fe640 1 -- 172.21.15.103:0/309258185 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c017070 con 0x7f32400ffe30 2024-08-19T01:36:20.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.857+0000 7f32377fe640 1 -- 172.21.15.103:0/309258185 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f323c009c00 con 0x7f32400ffe30 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.857+0000 7f32377fe640 1 -- 172.21.15.103:0/309258185 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00f6b0 con 0x7f32400ffe30 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.857+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324019aa60 con 0x7f32400ffe30 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.857+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401a6c90 con 0x7f32400ffe30 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 Processor -- start 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 -- start start 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a74b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a755c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a74b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a74b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48224/0 (socket says 172.21.15.103:48224) 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f3246bdf640 1 -- 172.21.15.103:0/4092497515 learned_addr learned my addr 172.21.15.103:0/4092497515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 0x556f8a7585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556f8a6b0a40 con 0x556f8a74aff0 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556f8a6489d0 con 0x556f8a7561d0 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.860+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556f8a6aca00 con 0x556f8a74b910 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.861+0000 7f32377fe640 1 -- 172.21.15.103:0/309258185 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f323c010460 con 0x7f32400ffe30 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.861+0000 7f3246bdf640 1 -- 172.21.15.103:0/4092497515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 msgr2=0x556f8a755c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.861+0000 7f3246bdf640 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a755c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.861+0000 7f3246bdf640 1 -- 172.21.15.103:0/4092497515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 msgr2=0x556f8a7585c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.861+0000 7f3246bdf640 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 0x556f8a7585c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.861+0000 7f3246bdf640 1 -- 172.21.15.103:0/4092497515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8a64f650 con 0x556f8a74aff0 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.861+0000 7f3246bdf640 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a74b3d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3238035000 tx=0x7f32380093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79221f7ff9fd5f7f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.862+0000 7f32377fe640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f322c0a2c00 0x7f322c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.862+0000 7f3234ff9640 1 -- 172.21.15.103:0/4092497515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238031c40 con 0x556f8a74aff0 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.862+0000 7f3234ff9640 1 -- 172.21.15.103:0/4092497515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3238036030 con 0x556f8a74aff0 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.862+0000 7f3234ff9640 1 -- 172.21.15.103:0/4092497515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32380042a0 con 0x556f8a74aff0 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.862+0000 7f32463de640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f322c0a2c00 0x7f322c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.862+0000 7f32377fe640 1 -- 172.21.15.103:0/309258185 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f323c0ceab0 con 0x7f32400ffe30 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 msgr2=0x556f8a74b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f32463de640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f322c0a2c00 0x7f322c0a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f323c0cc950 tx=0x7f323c065fc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a74b3d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3238035000 tx=0x7f32380093e0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 shutdown_connections 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 0x556f8a7585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a755c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 --2- 172.21.15.103:0/4092497515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a74b3d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 >> 172.21.15.103:0/4092497515 conn(0x556f8a747580 msgr2=0x556f8a747960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 shutdown_connections 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.863+0000 7f324aa40740 1 -- 172.21.15.103:0/4092497515 wait complete. 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 -- start start 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 Processor -- start 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74aff0 0x556f8a753070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a7535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a753af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556f8a758c20 con 0x556f8a7561d0 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556f8a66bb20 con 0x556f8a74aff0 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556f8a758da0 con 0x556f8a74b910 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f324aa40740 10 client.0 _ll_register_callbacks cb 0x556f8a669bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a753af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f3245bdd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a7535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a753af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48226/0 (socket says 172.21.15.103:48226) 2024-08-19T01:36:20.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f3246bdf640 1 -- 172.21.15.103:0/2393822727 learned_addr learned my addr 172.21.15.103:0/2393822727 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.864+0000 7f3245bdd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a7535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:47200/0 (socket says 172.21.15.103:47200) 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3245bdd640 1 -- 172.21.15.103:0/2393822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74aff0 msgr2=0x556f8a753070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3245bdd640 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74aff0 0x556f8a753070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3245bdd640 1 -- 172.21.15.103:0/2393822727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 msgr2=0x556f8a753af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3245bdd640 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a753af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3245bdd640 1 -- 172.21.15.103:0/2393822727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8a4223c0 con 0x556f8a74b910 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3246bdf640 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a753af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3245bdd640 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a7535b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x556f8a748900 tx=0x7f32300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a9f56659b5208d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3229ffb640 1 -- 172.21.15.103:0/2393822727 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230005020 con 0x556f8a74b910 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f3229ffb640 1 -- 172.21.15.103:0/2393822727 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f32300051c0 con 0x556f8a74b910 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.865+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3238036030 con 0x556f8a74b910 2024-08-19T01:36:20.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f3229ffb640 1 -- 172.21.15.103:0/2393822727 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32300053c0 con 0x556f8a74b910 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 10 client.25081 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f8a7648a0 con 0x556f8a74b910 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f3229ffb640 1 -- 172.21.15.103:0/2393822727 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3238036030 con 0x556f8a74b910 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f3229ffb640 1 -- 172.21.15.103:0/2393822727 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3230018070 con 0x556f8a74b910 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 10 client.25081 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 10 client.25081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 20 client.25081 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a768260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 4 client.25081 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f8a6bdfb0 con 0x556f8a765e00 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.866+0000 7f3246bdf640 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a768260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.867+0000 7f3246bdf640 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a768260 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f12b278ecbde047 server_cookie=4115f19e8afd3cad in_seq=0 out_seq=0 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.867+0000 7f3229ffb640 10 client.25081 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f3229ffb640 1 -- 172.21.15.103:0/2393822727 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f8a6bdfb0 con 0x556f8a765e00 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f3229ffb640 10 client.25081 handle_command_reply: tid=0 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 client.25081 shutdown 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 msgr2=0x556f8a7535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a7535b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x556f8a748900 tx=0x7f32300118d0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 shutdown_connections 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a768260 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a753af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f3229ffb640 0 client.25081 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:20.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a7535b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 --2- 172.21.15.103:0/2393822727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74aff0 0x556f8a753070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 >> 172.21.15.103:0/2393822727 conn(0x556f8a747580 msgr2=0x556f8a74d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 shutdown_connections 2024-08-19T01:36:20.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 1 -- 172.21.15.103:0/2393822727 wait complete. 2024-08-19T01:36:20.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.868+0000 7f324aa40740 20 client.25081 trim_cache size 0 max 16384 2024-08-19T01:36:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 Processor -- start 2024-08-19T01:36:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 -- start start 2024-08-19T01:36:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74aff0 0x556f8a747960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74b910 0x556f8a76db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a76e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a76e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a76e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48232/0 (socket says 172.21.15.103:48232) 2024-08-19T01:36:20.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f3246bdf640 1 -- 172.21.15.103:0/862037355 learned_addr learned my addr 172.21.15.103:0/862037355 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32401af280 con 0x556f8a7561d0 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32401b0dc0 con 0x556f8a74b910 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32401af950 con 0x556f8a74aff0 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.895+0000 7f3246bdf640 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74aff0 0x556f8a747960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f3245bdd640 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74b910 0x556f8a76db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f3246bdf640 1 -- 172.21.15.103:0/862037355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74aff0 msgr2=0x556f8a747960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f3246bdf640 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74aff0 0x556f8a747960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f3246bdf640 1 -- 172.21.15.103:0/862037355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74b910 msgr2=0x556f8a76db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f3246bdf640 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74b910 0x556f8a76db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f3246bdf640 1 -- 172.21.15.103:0/862037355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8a755b20 con 0x556f8a7561d0 2024-08-19T01:36:20.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f3246bdf640 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a76e070 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3238009bf0 tx=0x7f3238005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a528a9018c4be65 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f322bfff640 1 -- 172.21.15.103:0/862037355 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32380314d0 con 0x556f8a7561d0 2024-08-19T01:36:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f322bfff640 1 -- 172.21.15.103:0/862037355 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3238036040 con 0x556f8a7561d0 2024-08-19T01:36:20.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.896+0000 7f322bfff640 1 -- 172.21.15.103:0/862037355 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238004610 con 0x556f8a7561d0 2024-08-19T01:36:20.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 msgr2=0x556f8a76e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a76e070 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3238009bf0 tx=0x7f3238005fd0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 shutdown_connections 2024-08-19T01:36:20.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a7561d0 0x556f8a76e070 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a74b910 0x556f8a76db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 --2- 172.21.15.103:0/862037355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74aff0 0x556f8a747960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 >> 172.21.15.103:0/862037355 conn(0x556f8a747580 msgr2=0x556f8a7421b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.897+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 shutdown_connections 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 -- 172.21.15.103:0/862037355 wait complete. 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 -- start start 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 Processor -- start 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a75e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a75e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 0x556f8a74f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32401b06f0 con 0x556f8a74aff0 2024-08-19T01:36:20.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32401b0020 con 0x556f8a7561d0 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f324aa40740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32401ad260 con 0x556f8a74b910 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.898+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a75e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f324aa40740 10 client.0 _ll_register_callbacks cb 0x7f3230005a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a75e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48236/0 (socket says 172.21.15.103:48236) 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 -- 172.21.15.103:0/945049022 learned_addr learned my addr 172.21.15.103:0/945049022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 -- 172.21.15.103:0/945049022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 msgr2=0x556f8a75e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3245bdd640 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a75e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a75e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 -- 172.21.15.103:0/945049022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 msgr2=0x556f8a74f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 0x556f8a74f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 -- 172.21.15.103:0/945049022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8a74fc70 con 0x556f8a74aff0 2024-08-19T01:36:20.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3245bdd640 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a75e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:20.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3246bdf640 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a75e040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f323802f310 tx=0x7f323802f340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0aa56837b85b30a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:20.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:20.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3229ffb640 1 -- 172.21.15.103:0/945049022 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238031450 con 0x556f8a74aff0 2024-08-19T01:36:20.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f3229ffb640 1 -- 172.21.15.103:0/945049022 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f32380315f0 con 0x556f8a74aff0 2024-08-19T01:36:20.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.899+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3238036040 con 0x556f8a74aff0 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f3229ffb640 1 -- 172.21.15.103:0/945049022 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3238031c50 con 0x556f8a74aff0 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 10 client.15528 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f8a7d0340 con 0x556f8a74aff0 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f3229ffb640 1 -- 172.21.15.103:0/945049022 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3238036040 con 0x556f8a74aff0 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f3229ffb640 1 -- 172.21.15.103:0/945049022 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3238043070 con 0x556f8a74aff0 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 10 client.15528 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 10 client.15528 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 20 client.15528 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a7d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 4 client.15528 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:20.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.900+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f8a6b84c0 con 0x556f8a765e00 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f3245bdd640 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a7d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f3245bdd640 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a7d26b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=634f358d3d4e4fde server_cookie=4db0187f9174ff2 in_seq=0 out_seq=0 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f3229ffb640 10 client.15528 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f3229ffb640 1 -- 172.21.15.103:0/945049022 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f8a6b84c0 con 0x556f8a765e00 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f3229ffb640 10 client.15528 handle_command_reply: tid=0 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f324aa40740 1 client.15528 shutdown 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 msgr2=0x556f8a75e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.901+0000 7f324aa40740 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a75e040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f323802f310 tx=0x7f323802f340 comp rx=0 tx=0).stop 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 shutdown_connections 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x556f8a765e00 0x556f8a7d26b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f3229ffb640 0 client.15528 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556f8a7561d0 0x556f8a74f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556f8a74b910 0x556f8a75e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 --2- 172.21.15.103:0/945049022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8a74aff0 0x556f8a75e040 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 >> 172.21.15.103:0/945049022 conn(0x556f8a747580 msgr2=0x556f8a747960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 shutdown_connections 2024-08-19T01:36:20.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 1 -- 172.21.15.103:0/945049022 wait complete. 2024-08-19T01:36:20.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.902+0000 7f324aa40740 20 client.15528 trim_cache size 0 max 16384 2024-08-19T01:36:20.908 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f322c0a2c00 msgr2=0x7f322c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f322c0a2c00 0x7f322c0a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f323c0cc950 tx=0x7f323c065fc0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32400ffe30 msgr2=0x7f324019fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32400ffe30 0x7f324019fe70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f323c00da90 tx=0x7f323c00df50 comp rx=0 tx=0).stop 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 shutdown_connections 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f322c0a2c00 0x7f322c0a50c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32401068b0 0x7f3240199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32401007d0 0x7f32401a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 --2- 172.21.15.103:0/309258185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32400ffe30 0x7f324019fe70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 >> 172.21.15.103:0/309258185 conn(0x7f3240076f60 msgr2=0x7f3240103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:20.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 shutdown_connections 2024-08-19T01:36:20.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:20.903+0000 7f3248642640 1 -- 172.21.15.103:0/309258185 wait complete. 2024-08-19T01:36:21.697 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:36:21.987 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:22.160 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:21 smithi103 ceph-mon[18436]: pgmap v356: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:22.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:21 smithi146 ceph-mon[22098]: pgmap v356: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:21 smithi190 ceph-mon[22032]: pgmap v356: 97 pgs: 97 active+clean; 665 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-19T01:36:24.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:23 smithi146 ceph-mon[22098]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-08-19T01:36:24.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:23 smithi103 ceph-mon[18436]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-08-19T01:36:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:23 smithi190 ceph-mon[22032]: pgmap v357: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-08-19T01:36:24.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 -- 172.21.15.103:0/181770093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37181068b0 msgr2=0x7f3718106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 --2- 172.21.15.103:0/181770093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37181068b0 0x7f3718106c90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3708009900 tx=0x7f370802a410 comp rx=0 tx=0).stop 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 -- 172.21.15.103:0/181770093 shutdown_connections 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 --2- 172.21.15.103:0/181770093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37181007d0 0x7f371810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 --2- 172.21.15.103:0/181770093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37180ffe30 0x7f3718100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 --2- 172.21.15.103:0/181770093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37181068b0 0x7f3718106c90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 -- 172.21.15.103:0/181770093 >> 172.21.15.103:0/181770093 conn(0x7f3718076f60 msgr2=0x7f37180fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 -- 172.21.15.103:0/181770093 shutdown_connections 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.055+0000 7f371d7af640 1 -- 172.21.15.103:0/181770093 wait complete. 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 Processor -- start 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 -- start start 2024-08-19T01:36:24.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37180ffe30 0x7f3718198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37181007d0 0x7f37181985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37181068b0 0x7f371819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3718111c30 con 0x7f37180ffe30 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3718111ab0 con 0x7f37181068b0 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f371d7af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3718111db0 con 0x7f37181007d0 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f3716ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37180ffe30 0x7f3718198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f3716ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37180ffe30 0x7f3718198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37224/0 (socket says 172.21.15.103:37224) 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f3716ffd640 1 -- 172.21.15.103:0/328681442 learned_addr learned my addr 172.21.15.103:0/328681442 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f37177fe640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37181068b0 0x7f371819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.056+0000 7f370ffff640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37181007d0 0x7f37181985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3716ffd640 1 -- 172.21.15.103:0/328681442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37181007d0 msgr2=0x7f37181985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3716ffd640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37181007d0 0x7f37181985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3716ffd640 1 -- 172.21.15.103:0/328681442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37181068b0 msgr2=0x7f371819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3716ffd640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37181068b0 0x7f371819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3716ffd640 1 -- 172.21.15.103:0/328681442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371819d040 con 0x7f37180ffe30 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f37177fe640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37181068b0 0x7f371819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3716ffd640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37180ffe30 0x7f3718198090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3708005940 tx=0x7f37080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3714ff9640 1 -- 172.21.15.103:0/328681442 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370803f030 con 0x7f37180ffe30 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3714ff9640 1 -- 172.21.15.103:0/328681442 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3708003cb0 con 0x7f37180ffe30 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371819d2d0 con 0x7f37180ffe30 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f3714ff9640 1 -- 172.21.15.103:0/328681442 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708035070 con 0x7f37180ffe30 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.057+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37181a4c10 con 0x7f37180ffe30 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.059+0000 7f3714ff9640 1 -- 172.21.15.103:0/328681442 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3708042030 con 0x7f37180ffe30 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 Processor -- start 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 -- start start 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da31290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f8d996a40 con 0x559f8da317d0 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559f8d92e9d0 con 0x559f8da405e0 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f371fbad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559f8d992a00 con 0x559f8da30eb0 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f3716ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f3714ff9640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37040a2c00 0x7f37040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f3716ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37226/0 (socket says 172.21.15.103:37226) 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.060+0000 7f3716ffd640 1 -- 172.21.15.103:0/369137123 learned_addr learned my addr 172.21.15.103:0/369137123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37040a2c00 0x7f37040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da31290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f370ffff640 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f3714ff9640 1 -- 172.21.15.103:0/328681442 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f37080ef850 con 0x7f37180ffe30 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 -- 172.21.15.103:0/369137123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 msgr2=0x559f8da429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 -- 172.21.15.103:0/369137123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 msgr2=0x559f8da400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 -- 172.21.15.103:0/369137123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8d935650 con 0x559f8da30eb0 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da31290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f37181074f0 tx=0x7f37100107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975485aa7ed7498e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f37177fe640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37040a2c00 0x7f37040a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3710010ca0 tx=0x7f371001b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.061+0000 7f370d7fa640 1 -- 172.21.15.103:0/369137123 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371000ebe0 con 0x559f8da30eb0 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f370d7fa640 1 -- 172.21.15.103:0/369137123 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371000ed80 con 0x559f8da30eb0 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f370d7fa640 1 -- 172.21.15.103:0/369137123 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710004030 con 0x559f8da30eb0 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 -- 172.21.15.103:0/369137123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 msgr2=0x559f8da31290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da31290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f37181074f0 tx=0x7f37100107a0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 -- 172.21.15.103:0/369137123 shutdown_connections 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 --2- 172.21.15.103:0/369137123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da31290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 -- 172.21.15.103:0/369137123 >> 172.21.15.103:0/369137123 conn(0x559f8da2d580 msgr2=0x559f8da2d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 -- 172.21.15.103:0/369137123 shutdown_connections 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.062+0000 7f371fbad740 1 -- 172.21.15.103:0/369137123 wait complete. 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 -- start start 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 Processor -- start 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da30eb0 0x559f8da3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da317d0 0x559f8da3dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da405e0 0x559f8da3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f8da43150 con 0x559f8da405e0 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559f8d951b20 con 0x559f8da30eb0 2024-08-19T01:36:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559f8da432d0 con 0x559f8da317d0 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.063+0000 7f371fbad740 10 client.0 _ll_register_callbacks cb 0x559f8d94fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da405e0 0x559f8da3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da405e0 0x559f8da3e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37228/0 (socket says 172.21.15.103:37228) 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 -- 172.21.15.103:0/386736259 learned_addr learned my addr 172.21.15.103:0/386736259 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f3716ffd640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da30eb0 0x559f8da3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f370ffff640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da317d0 0x559f8da3dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 -- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da317d0 msgr2=0x559f8da3dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da317d0 0x559f8da3dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 -- 172.21.15.103:0/386736259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da30eb0 msgr2=0x559f8da3d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da30eb0 0x559f8da3d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 -- 172.21.15.103:0/386736259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8d7083c0 con 0x559f8da405e0 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f3716ffd640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da30eb0 0x559f8da3d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f370ffff640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da317d0 0x559f8da3dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f37177fe640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da405e0 0x559f8da3e020 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3710013690 tx=0x7f3710007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1533c715a2df5d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f36fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f36fe7fc640 1 -- 172.21.15.103:0/386736259 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371000ebe0 con 0x559f8da405e0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f36fe7fc640 1 -- 172.21.15.103:0/386736259 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3710003c10 con 0x559f8da405e0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371000ed80 con 0x559f8da405e0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f36fe7fc640 1 -- 172.21.15.103:0/386736259 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3710003e10 con 0x559f8da405e0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f371fbad740 10 client.15540 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.064+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f8da4ee70 con 0x559f8da405e0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f36fe7fc640 1 -- 172.21.15.103:0/386736259 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371000ed80 con 0x559f8da405e0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f36fe7fc640 1 -- 172.21.15.103:0/386736259 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f371001c050 con 0x559f8da405e0 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f371fbad740 10 client.15540 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f371fbad740 10 client.15540 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f371fbad740 20 client.15540 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f371fbad740 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8da50330 0x559f8da52720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f371fbad740 4 client.15540 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f8d9a3fb0 con 0x559f8da50330 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.065+0000 7f3716ffd640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8da50330 0x559f8da52720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.066+0000 7f3716ffd640 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8da50330 0x559f8da52720 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b72cbd64d7dc3747 server_cookie=2382ae2f22e90e4f in_seq=0 out_seq=0 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.066+0000 7f36fe7fc640 10 client.15540 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.066+0000 7f36fe7fc640 1 -- 172.21.15.103:0/386736259 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f8d9a3fb0 con 0x559f8da50330 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.066+0000 7f36fe7fc640 10 client.15540 handle_command_reply: tid=0 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.066+0000 7f371fbad740 1 client.15540 shutdown 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da405e0 msgr2=0x559f8da3e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da405e0 0x559f8da3e020 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3710013690 tx=0x7f3710007a40 comp rx=0 tx=0).stop 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 shutdown_connections 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8da50330 0x559f8da52720 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da405e0 0x559f8da3e020 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da317d0 0x559f8da3dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 --2- 172.21.15.103:0/386736259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da30eb0 0x559f8da3d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f36fe7fc640 0 client.15540 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 >> 172.21.15.103:0/386736259 conn(0x559f8da2d580 msgr2=0x559f8da332e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 shutdown_connections 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 1 -- 172.21.15.103:0/386736259 wait complete. 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.067+0000 7f371fbad740 20 client.15540 trim_cache size 0 max 16384 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 Processor -- start 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 -- start start 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da57f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da2bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37181aae20 con 0x559f8da317d0 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37181ac960 con 0x559f8da405e0 2024-08-19T01:36:24.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f371fbad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37181ab4f0 con 0x559f8da30eb0 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f370ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f370ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37238/0 (socket says 172.21.15.103:37238) 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.080+0000 7f370ffff640 1 -- 172.21.15.103:0/4266781135 learned_addr learned my addr 172.21.15.103:0/4266781135 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f37177fe640 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da2bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f3716ffd640 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da57f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370ffff640 1 -- 172.21.15.103:0/4266781135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 msgr2=0x559f8da57f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370ffff640 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da57f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370ffff640 1 -- 172.21.15.103:0/4266781135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 msgr2=0x559f8da2bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370ffff640 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da2bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370ffff640 1 -- 172.21.15.103:0/4266781135 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8da31600 con 0x559f8da317d0 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f3716ffd640 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da57f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f37177fe640 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370ffff640 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da584c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3700009bb0 tx=0x7f3700012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68e0fb55720ba2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370cff9640 1 -- 172.21.15.103:0/4266781135 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700021070 con 0x559f8da317d0 2024-08-19T01:36:24.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370cff9640 1 -- 172.21.15.103:0/4266781135 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37000035f0 con 0x559f8da317d0 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.081+0000 7f370cff9640 1 -- 172.21.15.103:0/4266781135 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370000f6b0 con 0x559f8da317d0 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 -- 172.21.15.103:0/4266781135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 msgr2=0x559f8da584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da584c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3700009bb0 tx=0x7f3700012820 comp rx=0 tx=0).stop 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 -- 172.21.15.103:0/4266781135 shutdown_connections 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da584c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 --2- 172.21.15.103:0/4266781135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da57f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 -- 172.21.15.103:0/4266781135 >> 172.21.15.103:0/4266781135 conn(0x559f8da2d580 msgr2=0x559f8da281b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 -- 172.21.15.103:0/4266781135 shutdown_connections 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 -- 172.21.15.103:0/4266781135 wait complete. 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 -- start start 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.082+0000 7f371fbad740 1 Processor -- start 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f371fbad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f371fbad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da48c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f371fbad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da43b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f371fbad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37181ac290 con 0x559f8da317d0 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f371fbad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37181abbc0 con 0x559f8da405e0 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f371fbad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37181a8e00 con 0x559f8da30eb0 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f371fbad740 10 client.0 _ll_register_callbacks cb 0x7f37080bbeb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da48c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da48c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37252/0 (socket says 172.21.15.103:37252) 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 -- 172.21.15.103:0/2534246962 learned_addr learned my addr 172.21.15.103:0/2534246962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f37177fe640 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da43b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f3716ffd640 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 -- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 msgr2=0x559f8da48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da48730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 -- 172.21.15.103:0/2534246962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 msgr2=0x559f8da43b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:24.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da43b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 -- 172.21.15.103:0/2534246962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8da44320 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f370ffff640 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da48c70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3700009b80 tx=0x7f3700009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a96139f9b6e002e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.083+0000 7f36fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f36fe7fc640 1 -- 172.21.15.103:0/2534246962 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700021070 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f36fe7fc640 1 -- 172.21.15.103:0/2534246962 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3700003bb0 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37000035f0 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f36fe7fc640 1 -- 172.21.15.103:0/2534246962 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3700027420 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 10 client.15546 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f8daba9d0 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f36fe7fc640 1 -- 172.21.15.103:0/2534246962 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37000035f0 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f36fe7fc640 1 -- 172.21.15.103:0/2534246962 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f370001b030 con 0x559f8da317d0 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 10 client.15546 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:24.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 10 client.15546 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 20 client.15546 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8dabb0c0 0x559f8dabd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 4 client.15546 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f8d99e4c0 con 0x559f8dabb0c0 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.084+0000 7f3716ffd640 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8dabb0c0 0x559f8dabd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f3716ffd640 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8dabb0c0 0x559f8dabd4e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64a6815f4c56b1f2 server_cookie=c5f2de167fdd118a in_seq=0 out_seq=0 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f36fe7fc640 10 client.15546 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f36fe7fc640 1 -- 172.21.15.103:0/2534246962 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f8d99e4c0 con 0x559f8dabb0c0 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f36fe7fc640 10 client.15546 handle_command_reply: tid=0 2024-08-19T01:36:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 client.15546 shutdown 2024-08-19T01:36:24.909 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 msgr2=0x559f8da48c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da48c70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3700009b80 tx=0x7f3700009990 comp rx=0 tx=0).stop 2024-08-19T01:36:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 shutdown_connections 2024-08-19T01:36:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x559f8dabb0c0 0x559f8dabd4e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f8da405e0 0x559f8da43b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f36fe7fc640 0 client.15546 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f8da317d0 0x559f8da48c70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 --2- 172.21.15.103:0/2534246962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559f8da30eb0 0x559f8da48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 >> 172.21.15.103:0/2534246962 conn(0x559f8da2d580 msgr2=0x559f8da2bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 shutdown_connections 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 1 -- 172.21.15.103:0/2534246962 wait complete. 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.085+0000 7f371fbad740 20 client.15546 trim_cache size 0 max 16384 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37040a2c00 msgr2=0x7f37040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37040a2c00 0x7f37040a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3710010ca0 tx=0x7f371001b000 comp rx=0 tx=0).stop 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37180ffe30 msgr2=0x7f3718198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37180ffe30 0x7f3718198090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3708005940 tx=0x7f37080042f0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 shutdown_connections 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37040a2c00 0x7f37040a50c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37181068b0 0x7f371819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37181007d0 0x7f37181985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 --2- 172.21.15.103:0/328681442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37180ffe30 0x7f3718198090 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 >> 172.21.15.103:0/328681442 conn(0x7f3718076f60 msgr2=0x7f3718104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 shutdown_connections 2024-08-19T01:36:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:24.086+0000 7f371d7af640 1 -- 172.21.15.103:0/328681442 wait complete. 2024-08-19T01:36:25.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:36:25.977 INFO:tasks.workunit.client.0.smithi103.stderr:Updating files: 95% (12271/12849) Updating files: 96% (12336/12849) Updating files: 97% (12464/12849) Updating files: 98% (12593/12849) Updating files: 99% (12721/12849) Updating files: 100% (12849/12849) Updating files: 100% (12849/12849), done. 2024-08-19T01:36:26.035 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:26.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:25 smithi103 ceph-mon[18436]: pgmap v358: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-19T01:36:26.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:25 smithi146 ceph-mon[22098]: pgmap v358: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-19T01:36:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:25 smithi190 ceph-mon[22032]: pgmap v358: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-19T01:36:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/2688968659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 msgr2=0x7fbf741056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/2688968659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741056b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009900 tx=0x7fbf6402a3f0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/2688968659 shutdown_connections 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/2688968659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf74105bf0 0x7fbf74106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/2688968659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741056b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/2688968659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf741005f0 0x7fbf741009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/2688968659 >> 172.21.15.103:0/2688968659 conn(0x7fbf740fc500 msgr2=0x7fbf740fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/2688968659 shutdown_connections 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.572+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/2688968659 wait complete. 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 Processor -- start 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 -- start start 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf741005f0 0x7fbf741a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf74105bf0 0x7fbf7419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74118270 con 0x7fbf74100f10 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741180f0 con 0x7fbf74105bf0 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7bdf4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741183f0 con 0x7fbf741005f0 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7938f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7938f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37268/0 (socket says 172.21.15.103:37268) 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7938f640 1 -- 172.21.15.103:0/3328875502 learned_addr learned my addr 172.21.15.103:0/3328875502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.573+0000 7fbf7a391640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf74105bf0 0x7fbf7419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf741005f0 0x7fbf741a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7938f640 1 -- 172.21.15.103:0/3328875502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf741005f0 msgr2=0x7fbf741a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf741005f0 0x7fbf741a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7938f640 1 -- 172.21.15.103:0/3328875502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf74105bf0 msgr2=0x7fbf7419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf74105bf0 0x7fbf7419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7938f640 1 -- 172.21.15.103:0/3328875502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7419ed30 con 0x7fbf74100f10 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7a391640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf74105bf0 0x7fbf7419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741a4980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf64006010 tx=0x7fbf640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf6affd640 1 -- 172.21.15.103:0/3328875502 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6403f070 con 0x7fbf74100f10 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf6affd640 1 -- 172.21.15.103:0/3328875502 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf64003aa0 con 0x7fbf74100f10 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7419efc0 con 0x7fbf74100f10 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf6affd640 1 -- 172.21.15.103:0/3328875502 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64035030 con 0x7fbf74100f10 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.574+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741ab310 con 0x7fbf74100f10 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 Processor -- start 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 -- start start 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb18910 0x55c85eb22c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c85ea7da40 con 0x55c85eb17ff0 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c85ea159d0 con 0x55c85eb18910 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c85ea79a00 con 0x55c85eb231d0 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.577+0000 7fbf6affd640 1 -- 172.21.15.103:0/3328875502 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbf64042070 con 0x7fbf74100f10 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb183d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37274/0 (socket says 172.21.15.103:37274) 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 -- 172.21.15.103:0/1265135958 learned_addr learned my addr 172.21.15.103:0/1265135958 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf79b90640 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb18910 0x55c85eb22c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7938f640 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 -- 172.21.15.103:0/1265135958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 msgr2=0x55c85eb255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 -- 172.21.15.103:0/1265135958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb18910 msgr2=0x55c85eb22c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb18910 0x55c85eb22c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 -- 172.21.15.103:0/1265135958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c85ea1c650 con 0x55c85eb17ff0 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf6affd640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf600a2c00 0x7fbf600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf7a391640 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb183d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007f40 tx=0x7fbf6c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b173f603a28176e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf600a2c00 0x7fbf600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf6affd640 1 -- 172.21.15.103:0/3328875502 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbf640ef940 con 0x7fbf74100f10 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf5ffff640 1 -- 172.21.15.103:0/1265135958 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c014070 con 0x55c85eb17ff0 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf5ffff640 1 -- 172.21.15.103:0/1265135958 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6c004d20 con 0x55c85eb17ff0 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.578+0000 7fbf5ffff640 1 -- 172.21.15.103:0/1265135958 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c005520 con 0x55c85eb17ff0 2024-08-19T01:36:26.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/1265135958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 msgr2=0x55c85eb183d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb183d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007f40 tx=0x7fbf6c00e7e0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf600a2c00 0x7fbf600a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c85eb153e0 tx=0x7fbf70008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/1265135958 shutdown_connections 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb18910 0x55c85eb22c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/1265135958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb183d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/1265135958 >> 172.21.15.103:0/1265135958 conn(0x55c85eb14580 msgr2=0x55c85eb14960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/1265135958 shutdown_connections 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.579+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/1265135958 wait complete. 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 -- start start 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 Processor -- start 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb231d0 0x55c85eb20b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c85eb25da0 con 0x55c85eb17ff0 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c85ea38b20 con 0x55c85eb231d0 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c85eb25f20 con 0x55c85eb18910 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf7e1f2740 10 client.0 _ll_register_callbacks cb 0x55c85ea36bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf79b90640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf79b90640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37288/0 (socket says 172.21.15.103:37288) 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.580+0000 7fbf79b90640 1 -- 172.21.15.103:0/3455427474 learned_addr learned my addr 172.21.15.103:0/3455427474 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf7a391640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb231d0 0x55c85eb20b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf79b90640 1 -- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 msgr2=0x55c85eb205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf79b90640 1 -- 172.21.15.103:0/3455427474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb231d0 msgr2=0x55c85eb20b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb231d0 0x55c85eb20b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf79b90640 1 -- 172.21.15.103:0/3455427474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c85e7ef3c0 con 0x55c85eb17ff0 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf7a391640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb231d0 0x55c85eb20b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb200a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf70016a00 tx=0x7fbf700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b04162606ac47f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/3455427474 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7001c050 con 0x55c85eb17ff0 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf6c004d20 con 0x55c85eb17ff0 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/3455427474 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbf70014b40 con 0x55c85eb17ff0 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/3455427474 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf70010ea0 con 0x55c85eb17ff0 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf7e1f2740 10 client.15558 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:26.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c85eb31a00 con 0x55c85eb17ff0 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.581+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/3455427474 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6c004d20 con 0x55c85eb17ff0 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/3455427474 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf700064d0 con 0x55c85eb17ff0 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7e1f2740 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7e1f2740 10 client.15558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7e1f2740 20 client.15558 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb32ec0 0x55c85eb35340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7e1f2740 4 client.15558 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c85ea8afb0 con 0x55c85eb32ec0 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb32ec0 0x55c85eb35340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb32ec0 0x55c85eb35340 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=447d2d06da3aa43a server_cookie=8ff5bea6dd9af07c in_seq=0 out_seq=0 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.582+0000 7fbf5d7fa640 10 client.15558 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/3455427474 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c85ea8afb0 con 0x55c85eb32ec0 2024-08-19T01:36:26.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf5d7fa640 10 client.15558 handle_command_reply: tid=0 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 client.15558 shutdown 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 msgr2=0x55c85eb200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb200a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf70016a00 tx=0x7fbf700046c0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 shutdown_connections 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb32ec0 0x55c85eb35340 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb231d0 0x55c85eb20b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3455427474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb17ff0 0x55c85eb200a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf5d7fa640 0 client.15558 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 >> 172.21.15.103:0/3455427474 conn(0x55c85eb14580 msgr2=0x55c85eb1a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 shutdown_connections 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.583+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3455427474 wait complete. 2024-08-19T01:36:26.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.584+0000 7fbf7e1f2740 20 client.15558 trim_cache size 0 max 16384 2024-08-19T01:36:26.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 Processor -- start 2024-08-19T01:36:26.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 -- start start 2024-08-19T01:36:26.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb1aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb134c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb231d0 0x55c85eb3ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b3780 con 0x55c85eb231d0 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b52c0 con 0x55c85eb17ff0 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b3e50 con 0x55c85eb18910 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7a391640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb231d0 0x55c85eb3ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7a391640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb231d0 0x55c85eb3ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37294/0 (socket says 172.21.15.103:37294) 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7a391640 1 -- 172.21.15.103:0/3797818872 learned_addr learned my addr 172.21.15.103:0/3797818872 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7938f640 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb1aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7a391640 1 -- 172.21.15.103:0/3797818872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 msgr2=0x55c85eb134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7a391640 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7a391640 1 -- 172.21.15.103:0/3797818872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 msgr2=0x55c85eb1aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.597+0000 7fbf7a391640 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb1aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.598+0000 7fbf7a391640 1 -- 172.21.15.103:0/3797818872 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c85eb1e5f0 con 0x55c85eb231d0 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.598+0000 7fbf79b90640 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb1aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.598+0000 7fbf7a391640 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb231d0 0x55c85eb3ac60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007750 tx=0x7fbf6c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3e7781cd37d737 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.598+0000 7fbf5f7fe640 1 -- 172.21.15.103:0/3797818872 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c014070 con 0x55c85eb231d0 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.598+0000 7fbf5f7fe640 1 -- 172.21.15.103:0/3797818872 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6c010af0 con 0x55c85eb231d0 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.598+0000 7fbf5f7fe640 1 -- 172.21.15.103:0/3797818872 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c005590 con 0x55c85eb231d0 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3797818872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb231d0 msgr2=0x55c85eb3ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb231d0 0x55c85eb3ac60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007750 tx=0x7fbf6c007ad0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3797818872 shutdown_connections 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb231d0 0x55c85eb3ac60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb18910 0x55c85eb134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/3797818872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb1aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3797818872 >> 172.21.15.103:0/3797818872 conn(0x55c85eb14580 msgr2=0x55c85eb12db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3797818872 shutdown_connections 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/3797818872 wait complete. 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 -- start start 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.599+0000 7fbf7e1f2740 1 Processor -- start 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb2b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb18910 0x55c85eb2b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7e1f2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b4bf0 con 0x55c85eb18910 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b4520 con 0x55c85eb17ff0 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7e1f2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b1760 con 0x55c85eb231d0 2024-08-19T01:36:26.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7e1f2740 10 client.0 _ll_register_callbacks cb 0x7fbf7000b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7938f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb18910 0x55c85eb2b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7938f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb18910 0x55c85eb2b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37300/0 (socket says 172.21.15.103:37300) 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7938f640 1 -- 172.21.15.103:0/2944404267 learned_addr learned my addr 172.21.15.103:0/2944404267 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf79b90640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb2b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7a391640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7938f640 1 -- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 msgr2=0x55c85eb267d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7938f640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb267d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.600+0000 7fbf7938f640 1 -- 172.21.15.103:0/2944404267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 msgr2=0x55c85eb2b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7938f640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb2b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7938f640 1 -- 172.21.15.103:0/2944404267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c85eb27000 con 0x55c85eb18910 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7a391640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf79b90640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb2b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:26.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7938f640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb18910 0x55c85eb2b980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbf640bbed0 tx=0x7fbf640319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09f11e1742558a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/2944404267 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64049040 con 0x55c85eb18910 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/2944404267 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbf640d5160 con 0x55c85eb18910 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf6c010af0 con 0x55c85eb18910 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/2944404267 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf640bc8d0 con 0x55c85eb18910 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 10 client.15564 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c85eb9d810 con 0x55c85eb18910 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/2944404267 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6c010af0 con 0x55c85eb18910 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/2944404267 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf640b8840 con 0x55c85eb18910 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 10 client.15564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 20 client.15564 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb9df90 0x55c85eba03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:26.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.601+0000 7fbf7e1f2740 4 client.15564 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.602+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c85ea854c0 con 0x55c85eb9df90 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.602+0000 7fbf79b90640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb9df90 0x55c85eba03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.602+0000 7fbf5d7fa640 10 client.15564 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.602+0000 7fbf79b90640 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb9df90 0x55c85eba03b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a942978e00f41ab server_cookie=4c118f417963420f in_seq=0 out_seq=0 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf5d7fa640 1 -- 172.21.15.103:0/2944404267 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c85ea854c0 con 0x55c85eb9df90 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf5d7fa640 10 client.15564 handle_command_reply: tid=0 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 client.15564 shutdown 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb18910 msgr2=0x55c85eb2b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb18910 0x55c85eb2b980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbf640bbed0 tx=0x7fbf640319b0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 shutdown_connections 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55c85eb9df90 0x55c85eba03b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c85eb231d0 0x55c85eb267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c85eb18910 0x55c85eb2b980 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 --2- 172.21.15.103:0/2944404267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c85eb17ff0 0x55c85eb2b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf5d7fa640 0 client.15564 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:36:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 >> 172.21.15.103:0/2944404267 conn(0x55c85eb14580 msgr2=0x55c85eb2e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 shutdown_connections 2024-08-19T01:36:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 1 -- 172.21.15.103:0/2944404267 wait complete. 2024-08-19T01:36:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.603+0000 7fbf7e1f2740 20 client.15564 trim_cache size 0 max 16384 2024-08-19T01:36:26.615 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf600a2c00 msgr2=0x7fbf600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf600a2c00 0x7fbf600a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c85eb153e0 tx=0x7fbf70008040 comp rx=0 tx=0).stop 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 msgr2=0x7fbf741a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741a4980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf64006010 tx=0x7fbf640042f0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 shutdown_connections 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbf600a2c00 0x7fbf600a50c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf74105bf0 0x7fbf7419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf74100f10 0x7fbf741a4980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 --2- 172.21.15.103:0/3328875502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf741005f0 0x7fbf741a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 >> 172.21.15.103:0/3328875502 conn(0x7fbf740fc500 msgr2=0x7fbf7410f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 shutdown_connections 2024-08-19T01:36:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:26.604+0000 7fbf7bdf4640 1 -- 172.21.15.103:0/3328875502 wait complete. 2024-08-19T01:36:26.927 INFO:tasks.workunit.client.0.smithi103.stderr:Note: switching to '78215f5a1cf5eaf321d506a66539942e12851f34'. 2024-08-19T01:36:26.927 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T01:36:26.927 INFO:tasks.workunit.client.0.smithi103.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-19T01:36:26.927 INFO:tasks.workunit.client.0.smithi103.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-19T01:36:26.927 INFO:tasks.workunit.client.0.smithi103.stderr:state without impacting any branches by switching back to a branch. 2024-08-19T01:36:26.927 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T01:36:26.927 INFO:tasks.workunit.client.0.smithi103.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr: git switch -c 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr:Or undo this operation with: 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr: git switch - 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-08-19T01:36:26.928 INFO:tasks.workunit.client.0.smithi103.stderr:HEAD is now at 78215f5a1cf Merge pull request #59291 from zdover23/wip-doc-2024-08-19-backport-59290-to-squid 2024-08-19T01:36:26.932 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-19T01:36:26.964 INFO:tasks.workunit.client.0.smithi103.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-19T01:36:26.966 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T01:36:26.966 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-19T01:36:27.710 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:28.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:27 smithi103 ceph-mon[18436]: pgmap v359: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-19T01:36:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:27 smithi146 ceph-mon[22098]: pgmap v359: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-19T01:36:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:27 smithi190 ceph-mon[22032]: pgmap v359: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-19T01:36:28.412 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-19T01:36:28.496 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-19T01:36:28.536 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-19T01:36:28.537 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T01:36:28.537 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-19T01:36:28.578 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-19T01:36:28.580 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:36:28.580 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-19T01:36:28.596 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-08-19T01:36:28.597 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-08-19T01:36:28.598 DEBUG:teuthology.orchestra.run.smithi103:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=78215f5a1cf5eaf321d506a66539942e12851f34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-08-19T01:36:28.666 INFO:tasks.workunit.client.0.smithi103.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-08-19T01:36:28.666 INFO:tasks.workunit.client.0.smithi103.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-08-19T01:36:28.666 INFO:tasks.workunit.client.0.smithi103.stderr:+ trap -- retry EXIT 2024-08-19T01:36:28.666 INFO:tasks.workunit.client.0.smithi103.stderr:+ rm -rf ffsb 2024-08-19T01:36:28.667 INFO:tasks.workunit.client.0.smithi103.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-08-19T01:36:28.672 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into 'ffsb'... 2024-08-19T01:36:28.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.642+0000 7f7b6673b640 1 -- 172.21.15.103:0/2489463906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b60102b00 msgr2=0x7f7b60102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.642+0000 7f7b6673b640 1 --2- 172.21.15.103:0/2489463906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b60102b00 0x7f7b60102ee0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009930 tx=0x7f7b5002a420 comp rx=0 tx=0).stop 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- 172.21.15.103:0/2489463906 shutdown_connections 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 --2- 172.21.15.103:0/2489463906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b60108030 0x7f7b6010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 --2- 172.21.15.103:0/2489463906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b601034b0 0x7f7b601078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 --2- 172.21.15.103:0/2489463906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b60102b00 0x7f7b60102ee0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- 172.21.15.103:0/2489463906 >> 172.21.15.103:0/2489463906 conn(0x7f7b600fc500 msgr2=0x7f7b600fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- 172.21.15.103:0/2489463906 shutdown_connections 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- 172.21.15.103:0/2489463906 wait complete. 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 Processor -- start 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- start start 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60102b00 0x7f7b601982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b601034b0 0x7f7b60198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b60108030 0x7f7b6019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b60111cb0 con 0x7f7b601034b0 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b60111b30 con 0x7f7b60102b00 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.643+0000 7f7b6673b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b60111e30 con 0x7f7b60108030 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b601034b0 0x7f7b60198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b601034b0 0x7f7b60198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37312/0 (socket says 172.21.15.103:37312) 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 -- 172.21.15.103:0/3214350649 learned_addr learned my addr 172.21.15.103:0/3214350649 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60102b00 0x7f7b601982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b64cd8640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b60108030 0x7f7b6019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 -- 172.21.15.103:0/3214350649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b60108030 msgr2=0x7f7b6019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b60108030 0x7f7b6019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 -- 172.21.15.103:0/3214350649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60102b00 msgr2=0x7f7b601982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60102b00 0x7f7b601982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 -- 172.21.15.103:0/3214350649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6019d2a0 con 0x7f7b601034b0 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b57fff640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b601034b0 0x7f7b60198830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800d740 tx=0x7f7b4800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:28.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b5dffb640 1 -- 172.21.15.103:0/3214350649 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48017070 con 0x7f7b601034b0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b5dffb640 1 -- 172.21.15.103:0/3214350649 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b4800de50 con 0x7f7b601034b0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b5dffb640 1 -- 172.21.15.103:0/3214350649 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4800f620 con 0x7f7b601034b0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.644+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6019d590 con 0x7f7b601034b0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.645+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b601a4e70 con 0x7f7b601034b0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b5dffb640 1 -- 172.21.15.103:0/3214350649 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7b4801a070 con 0x7f7b601034b0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 Processor -- start 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 -- start start 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c359f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2c3504a40 con 0x55f2c359f910 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f2c349c9d0 con 0x55f2c35aa1d0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.647+0000 7f7b68b39740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f2c3500a00 con 0x55f2c359eff0 2024-08-19T01:36:28.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5dffb640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7b4c0a2b80 0x7f7b4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37322/0 (socket says 172.21.15.103:37322) 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 -- 172.21.15.103:0/3335477024 learned_addr learned my addr 172.21.15.103:0/3335477024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b64cd8640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7b4c0a2b80 0x7f7b4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b64cd8640 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c359f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5dffb640 1 -- 172.21.15.103:0/3214350649 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7b480ce970 con 0x7f7b601034b0 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b57fff640 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 -- 172.21.15.103:0/3335477024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 msgr2=0x55f2c359f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c359f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 -- 172.21.15.103:0/3335477024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 msgr2=0x55f2c35ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 -- 172.21.15.103:0/3335477024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c34a3650 con 0x55f2c359f910 2024-08-19T01:36:28.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b57fff640 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b64cd8640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7b4c0a2b80 0x7f7b4c0a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7b580098b0 tx=0x7f7b58009af0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35a9c90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b5002a940 tx=0x7f7b50005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13a49182e43bfa1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b567fc640 1 -- 172.21.15.103:0/3335477024 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50040070 con 0x55f2c359f910 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.648+0000 7f7b567fc640 1 -- 172.21.15.103:0/3335477024 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b50004030 con 0x55f2c359f910 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b567fc640 1 -- 172.21.15.103:0/3335477024 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50035040 con 0x55f2c359f910 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 -- 172.21.15.103:0/3335477024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 msgr2=0x55f2c35a9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35a9c90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b5002a940 tx=0x7f7b50005c00 comp rx=0 tx=0).stop 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 -- 172.21.15.103:0/3335477024 shutdown_connections 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35a9c90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 --2- 172.21.15.103:0/3335477024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c359f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 -- 172.21.15.103:0/3335477024 >> 172.21.15.103:0/3335477024 conn(0x55f2c359b580 msgr2=0x55f2c359b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 -- 172.21.15.103:0/3335477024 shutdown_connections 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.649+0000 7f7b68b39740 1 -- 172.21.15.103:0/3335477024 wait complete. 2024-08-19T01:36:28.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 -- start start 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 Processor -- start 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359eff0 0x55f2c35a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359f910 0x55f2c35a7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2c35acda0 con 0x55f2c359eff0 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f2c34bfb20 con 0x55f2c35aa1d0 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f2c35acf20 con 0x55f2c359f910 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b68b39740 10 client.0 _ll_register_callbacks cb 0x55f2c34bdbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359eff0 0x55f2c35a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b5ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359eff0 0x55f2c35a7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37330/0 (socket says 172.21.15.103:37330) 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.650+0000 7f7b5ffff640 1 -- 172.21.15.103:0/272375255 learned_addr learned my addr 172.21.15.103:0/272375255 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b57fff640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359f910 0x55f2c35a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b64cd8640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b5ffff640 1 -- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359f910 msgr2=0x55f2c35a7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359f910 0x55f2c35a7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b5ffff640 1 -- 172.21.15.103:0/272375255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 msgr2=0x55f2c35a7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b5ffff640 1 -- 172.21.15.103:0/272375255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c32763c0 con 0x55f2c359eff0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b57fff640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359f910 0x55f2c35a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359eff0 0x55f2c35a7010 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7b50031440 tx=0x7f7b50005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c8e9a1f1a4250c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/272375255 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50040030 con 0x55f2c359eff0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/272375255 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7b500314a0 con 0x55f2c359eff0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b50004030 con 0x55f2c359eff0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/272375255 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b50035040 con 0x55f2c359eff0 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b68b39740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:28.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2c35b88b0 con 0x55f2c359eff0 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.651+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/272375255 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b50004030 con 0x55f2c359eff0 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/272375255 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b500035f0 con 0x55f2c359eff0 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b68b39740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b68b39740 10 client.15576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b68b39740 20 client.15576 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b68b39740 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c35b9c00 0x55f2c35bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b68b39740 4 client.15576 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2c3511fb0 con 0x55f2c35b9c00 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b57fff640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c35b9c00 0x55f2c35bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b57fff640 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c35b9c00 0x55f2c35bbff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f311d48862ee1b1 server_cookie=c2d0435162a940bb in_seq=0 out_seq=0 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.652+0000 7f7b3f7fe640 10 client.15576 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/272375255 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2c3511fb0 con 0x55f2c35b9c00 2024-08-19T01:36:28.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b3f7fe640 10 client.15576 handle_command_reply: tid=0 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 client.15576 shutdown 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359eff0 msgr2=0x55f2c35a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359eff0 0x55f2c35a7010 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7b50031440 tx=0x7f7b50005590 comp rx=0 tx=0).stop 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 shutdown_connections 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c35b9c00 0x55f2c35bbff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359f910 0x55f2c35a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b3f7fe640 0 client.15576 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 --2- 172.21.15.103:0/272375255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359eff0 0x55f2c35a7010 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.653+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 >> 172.21.15.103:0/272375255 conn(0x55f2c359b580 msgr2=0x55f2c35a1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.654+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 shutdown_connections 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.654+0000 7f7b68b39740 1 -- 172.21.15.103:0/272375255 wait complete. 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.654+0000 7f7b68b39740 20 client.15576 trim_cache size 0 max 16384 2024-08-19T01:36:28.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 Processor -- start 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 -- start start 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359eff0 0x55f2c35c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601ad1e0 con 0x55f2c359f910 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601aed20 con 0x55f2c359eff0 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b68b39740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601ad8b0 con 0x55f2c35aa1d0 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37344/0 (socket says 172.21.15.103:37344) 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 -- 172.21.15.103:0/1147191627 learned_addr learned my addr 172.21.15.103:0/1147191627 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359eff0 0x55f2c35c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 -- 172.21.15.103:0/1147191627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c35aa1d0 msgr2=0x55f2c35a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 -- 172.21.15.103:0/1147191627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359eff0 msgr2=0x55f2c35c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359eff0 0x55f2c35c15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b57fff640 1 -- 172.21.15.103:0/1147191627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c359c710 con 0x55f2c359f910 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.668+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359eff0 0x55f2c35c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.669+0000 7f7b57fff640 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35c1b30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7b480cc4a0 tx=0x7f7b48010b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5fce267830a2a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.669+0000 7f7b55ffb640 1 -- 172.21.15.103:0/1147191627 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48021040 con 0x55f2c359f910 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.669+0000 7f7b55ffb640 1 -- 172.21.15.103:0/1147191627 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b480a3e70 con 0x55f2c359f910 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.669+0000 7f7b55ffb640 1 -- 172.21.15.103:0/1147191627 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b480a39e0 con 0x55f2c359f910 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 -- 172.21.15.103:0/1147191627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 msgr2=0x55f2c35c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35c1b30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7b480cc4a0 tx=0x7f7b48010b00 comp rx=0 tx=0).stop 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 -- 172.21.15.103:0/1147191627 shutdown_connections 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c359f910 0x55f2c35c1b30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 --2- 172.21.15.103:0/1147191627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359eff0 0x55f2c35c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 -- 172.21.15.103:0/1147191627 >> 172.21.15.103:0/1147191627 conn(0x55f2c359b580 msgr2=0x55f2c3599d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:28.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 -- 172.21.15.103:0/1147191627 shutdown_connections 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 -- 172.21.15.103:0/1147191627 wait complete. 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 -- start start 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 Processor -- start 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c35b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359f910 0x55f2c35b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.670+0000 7f7b68b39740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b68b39740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601ae650 con 0x55f2c35aa1d0 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b68b39740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601adf80 con 0x55f2c359f910 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b68b39740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b601ab1c0 con 0x55f2c359eff0 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b68b39740 10 client.0 _ll_register_callbacks cb 0x7f7b50031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b64cd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b64cd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37356/0 (socket says 172.21.15.103:37356) 2024-08-19T01:36:28.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b64cd8640 1 -- 172.21.15.103:0/2396398889 learned_addr learned my addr 172.21.15.103:0/2396398889 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b57fff640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359f910 0x55f2c35b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c35b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b57fff640 1 -- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 msgr2=0x55f2c35b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b57fff640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c35b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b57fff640 1 -- 172.21.15.103:0/2396398889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c35aa1d0 msgr2=0x55f2c35a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b57fff640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b57fff640 1 -- 172.21.15.103:0/2396398889 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c35a42d0 con 0x55f2c359f910 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b64cd8640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b57fff640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359f910 0x55f2c35b29d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b480cc4a0 tx=0x7f7b48097650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecbbc1de827eaac8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/2396398889 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48021060 con 0x55f2c359f910 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/2396398889 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7b480a39e0 con 0x55f2c359f910 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.671+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b480a3e70 con 0x55f2c359f910 2024-08-19T01:36:28.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/2396398889 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b4809f370 con 0x55f2c359f910 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 10 client.25139 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2c35b1da0 con 0x55f2c359f910 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/2396398889 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b480a3e70 con 0x55f2c359f910 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/2396398889 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b4809a300 con 0x55f2c359f910 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 10 client.25139 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 10 client.25139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 20 client.25139 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c36247c0 0x55f2c3626bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 4 client.25139 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.672+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2c350c4c0 con 0x55f2c36247c0 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c36247c0 0x55f2c3626bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b3f7fe640 10 client.25139 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b5ffff640 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c36247c0 0x55f2c3626bb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88f9b26c5e7d70e9 server_cookie=45749b37bf239262 in_seq=0 out_seq=0 2024-08-19T01:36:28.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b3f7fe640 1 -- 172.21.15.103:0/2396398889 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2c350c4c0 con 0x55f2c36247c0 2024-08-19T01:36:28.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b3f7fe640 10 client.25139 handle_command_reply: tid=0 2024-08-19T01:36:28.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 client.25139 shutdown 2024-08-19T01:36:28.697 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:28.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359f910 msgr2=0x55f2c35b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359f910 0x55f2c35b29d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b480cc4a0 tx=0x7f7b48097650 comp rx=0 tx=0).stop 2024-08-19T01:36:28.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 shutdown_connections 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55f2c36247c0 0x55f2c3626bb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2c35aa1d0 0x55f2c35a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2c359f910 0x55f2c35b29d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b68b39740 1 --2- 172.21.15.103:0/2396398889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2c359eff0 0x55f2c35b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.673+0000 7f7b3f7fe640 0 client.25139 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 >> 172.21.15.103:0/2396398889 conn(0x55f2c359b580 msgr2=0x55f2c35a8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 shutdown_connections 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b68b39740 1 -- 172.21.15.103:0/2396398889 wait complete. 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b68b39740 20 client.25139 trim_cache size 0 max 16384 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7b4c0a2b80 msgr2=0x7f7b4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7b4c0a2b80 0x7f7b4c0a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7b580098b0 tx=0x7f7b58009af0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b601034b0 msgr2=0x7f7b60198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:28.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b601034b0 0x7f7b60198830 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800d740 tx=0x7f7b4800dc00 comp rx=0 tx=0).stop 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 shutdown_connections 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7b4c0a2b80 0x7f7b4c0a5040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b60108030 0x7f7b6019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b601034b0 0x7f7b60198830 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 --2- 172.21.15.103:0/3214350649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60102b00 0x7f7b601982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 >> 172.21.15.103:0/3214350649 conn(0x7f7b600fc500 msgr2=0x7f7b60105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 shutdown_connections 2024-08-19T01:36:28.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:28.674+0000 7f7b6673b640 1 -- 172.21.15.103:0/3214350649 wait complete. 2024-08-19T01:36:29.516 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:36:29.830 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:30.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:29 smithi103 ceph-mon[18436]: pgmap v360: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-08-19T01:36:30.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:29 smithi146 ceph-mon[22098]: pgmap v360: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-08-19T01:36:30.272 INFO:tasks.workunit.client.0.smithi103.stderr:+ trap - EXIT 2024-08-19T01:36:30.272 INFO:tasks.workunit.client.0.smithi103.stderr:+ cd ffsb 2024-08-19T01:36:30.272 INFO:tasks.workunit.client.0.smithi103.stderr:+ ./configure 2024-08-19T01:36:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:29 smithi190 ceph-mon[22032]: pgmap v360: 97 pgs: 97 active+clean; 666 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-08-19T01:36:30.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 -- 172.21.15.103:0/3281031195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01067a0 msgr2=0x7f54b0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/3281031195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01067a0 0x7f54b0106b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009930 tx=0x7f54a002a420 comp rx=0 tx=0).stop 2024-08-19T01:36:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 -- 172.21.15.103:0/3281031195 shutdown_connections 2024-08-19T01:36:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/3281031195 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54b0101140 0x7f54b010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/3281031195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b01007a0 0x7f54b0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/3281031195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01067a0 0x7f54b0106b80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 -- 172.21.15.103:0/3281031195 >> 172.21.15.103:0/3281031195 conn(0x7f54b00fc500 msgr2=0x7f54b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 -- 172.21.15.103:0/3281031195 shutdown_connections 2024-08-19T01:36:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.373+0000 7f54b4c5a640 1 -- 172.21.15.103:0/3281031195 wait complete. 2024-08-19T01:36:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 Processor -- start 2024-08-19T01:36:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 -- start start 2024-08-19T01:36:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01007a0 0x7f54b0198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0101140 0x7f54b0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54b01067a0 0x7f54b01a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54b0110a50 con 0x7f54b01007a0 2024-08-19T01:36:30.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01108d0 con 0x7f54b01067a0 2024-08-19T01:36:30.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54b4c5a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54b0110bd0 con 0x7f54b0101140 2024-08-19T01:36:30.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01007a0 0x7f54b0198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01007a0 0x7f54b0198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37370/0 (socket says 172.21.15.103:37370) 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 -- 172.21.15.103:0/2020122061 learned_addr learned my addr 172.21.15.103:0/2020122061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54b01067a0 0x7f54b01a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0101140 0x7f54b0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 -- 172.21.15.103:0/2020122061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0101140 msgr2=0x7f54b0199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0101140 0x7f54b0199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 -- 172.21.15.103:0/2020122061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54b01067a0 msgr2=0x7f54b01a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54b01067a0 0x7f54b01a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.374+0000 7f54ae59c640 1 -- 172.21.15.103:0/2020122061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b01a43e0 con 0x7f54b01007a0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.375+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01007a0 0x7f54b0198e20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f54a0005940 tx=0x7f54a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.375+0000 7f54a77fe640 1 -- 172.21.15.103:0/2020122061 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a003f030 con 0x7f54b01007a0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.375+0000 7f54a77fe640 1 -- 172.21.15.103:0/2020122061 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a0003cb0 con 0x7f54b01007a0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.375+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b01a45b0 con 0x7f54b01007a0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.375+0000 7f54a77fe640 1 -- 172.21.15.103:0/2020122061 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0035070 con 0x7f54b01007a0 2024-08-19T01:36:30.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.375+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b01a4a00 con 0x7f54b01007a0 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.377+0000 7f54a77fe640 1 -- 172.21.15.103:0/2020122061 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f54a0042030 con 0x7f54b01007a0 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 Processor -- start 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 -- start start 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6deac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6ded5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e3a6d45a40 con 0x55e3a6ddfff0 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e3a6cdd9d0 con 0x55e3a6de0910 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54b7058740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e3a6d41a00 con 0x55e3a6deb1d0 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54a77fe640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f549c0a2b80 0x7f549c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54aed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37378/0 (socket says 172.21.15.103:37378) 2024-08-19T01:36:30.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54aed9d640 1 -- 172.21.15.103:0/2976339406 learned_addr learned my addr 172.21.15.103:0/2976339406 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6deac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6ded5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f549c0a2b80 0x7f549c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54a77fe640 1 -- 172.21.15.103:0/2020122061 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f54a00ef4b0 con 0x7f54b01007a0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54aed9d640 1 -- 172.21.15.103:0/2976339406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 msgr2=0x55e3a6ded5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.378+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6ded5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.379+0000 7f54aed9d640 1 -- 172.21.15.103:0/2976339406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 msgr2=0x55e3a6deac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.379+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6deac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.379+0000 7f54aed9d640 1 -- 172.21.15.103:0/2976339406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a6ce4650 con 0x55e3a6ddfff0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.379+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de03d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54a80139d0 tx=0x7f54a8013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b8b18f62417ee0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.379+0000 7f54a4ff9640 1 -- 172.21.15.103:0/2976339406 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8013050 con 0x55e3a6ddfff0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.379+0000 7f54a4ff9640 1 -- 172.21.15.103:0/2976339406 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a8022070 con 0x55e3a6ddfff0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.379+0000 7f54a4ff9640 1 -- 172.21.15.103:0/2976339406 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a80126b0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 -- 172.21.15.103:0/2976339406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 msgr2=0x55e3a6de03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f549c0a2b80 0x7f549c0a5040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f54a8021000 tx=0x7f54a801f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de03d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54a80139d0 tx=0x7f54a8013e90 comp rx=0 tx=0).stop 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 -- 172.21.15.103:0/2976339406 shutdown_connections 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6ded5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6deac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 --2- 172.21.15.103:0/2976339406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de03d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 -- 172.21.15.103:0/2976339406 >> 172.21.15.103:0/2976339406 conn(0x55e3a6ddc580 msgr2=0x55e3a6ddc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 -- 172.21.15.103:0/2976339406 shutdown_connections 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 -- 172.21.15.103:0/2976339406 wait complete. 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 -- start start 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.380+0000 7f54b7058740 1 Processor -- start 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6de0910 0x55e3a6de8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e3a6dedda0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e3a6d00b20 con 0x55e3a6deb1d0 2024-08-19T01:36:30.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e3a6dedf20 con 0x55e3a6de0910 2024-08-19T01:36:30.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 10 client.0 _ll_register_callbacks cb 0x55e3a6cfebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:30.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37390/0 (socket says 172.21.15.103:37390) 2024-08-19T01:36:30.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 -- 172.21.15.103:0/2871819156 learned_addr learned my addr 172.21.15.103:0/2871819156 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:30.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6de0910 0x55e3a6de8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 -- 172.21.15.103:0/2871819156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6de0910 msgr2=0x55e3a6de8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6de0910 0x55e3a6de8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 -- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 msgr2=0x55e3a6de8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 -- 172.21.15.103:0/2871819156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a6ab73c0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6de0910 0x55e3a6de8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:30.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:30.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de8050 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f54a00bbcd0 tx=0x7f54a002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e9acb7e6eec274 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:30.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f5495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:30.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f5495ffb640 1 -- 172.21.15.103:0/2871819156 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0049070 con 0x55e3a6ddfff0 2024-08-19T01:36:30.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f5495ffb640 1 -- 172.21.15.103:0/2871819156 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f54a00b83b0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a8022070 con 0x55e3a6ddfff0 2024-08-19T01:36:30.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f5495ffb640 1 -- 172.21.15.103:0/2871819156 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a00eb020 con 0x55e3a6ddfff0 2024-08-19T01:36:30.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 10 client.15594 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:30.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.381+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3a6df9950 con 0x55e3a6ddfff0 2024-08-19T01:36:30.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f5495ffb640 1 -- 172.21.15.103:0/2871819156 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a8022070 con 0x55e3a6ddfff0 2024-08-19T01:36:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f5495ffb640 1 -- 172.21.15.103:0/2871819156 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54a00ef0b0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54b7058740 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54b7058740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54b7058740 20 client.15594 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54b7058740 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6dfadb0 0x55e3a6dfd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54b7058740 4 client.15594 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:30.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3a6d52fb0 con 0x55e3a6dfadb0 2024-08-19T01:36:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6dfadb0 0x55e3a6dfd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f5495ffb640 10 client.15594 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.382+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6dfadb0 0x55e3a6dfd200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6439e27c9a6412 server_cookie=637bf7c8b1a2e7bf in_seq=0 out_seq=0 2024-08-19T01:36:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f5495ffb640 1 -- 172.21.15.103:0/2871819156 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3a6d52fb0 con 0x55e3a6dfadb0 2024-08-19T01:36:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f5495ffb640 10 client.15594 handle_command_reply: tid=0 2024-08-19T01:36:30.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 client.15594 shutdown 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 msgr2=0x55e3a6de8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de8050 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f54a00bbcd0 tx=0x7f54a002a420 comp rx=0 tx=0).stop 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 shutdown_connections 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6dfadb0 0x55e3a6dfd200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6de0910 0x55e3a6de8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f5495ffb640 0 client.15594 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 --2- 172.21.15.103:0/2871819156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de8050 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 >> 172.21.15.103:0/2871819156 conn(0x55e3a6ddc580 msgr2=0x55e3a6de22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 shutdown_connections 2024-08-19T01:36:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 1 -- 172.21.15.103:0/2871819156 wait complete. 2024-08-19T01:36:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.383+0000 7f54b7058740 20 client.15594 trim_cache size 0 max 16384 2024-08-19T01:36:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 Processor -- start 2024-08-19T01:36:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 -- start start 2024-08-19T01:36:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6de0910 0x55e3a6e02b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6e03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01acf30 con 0x55e3a6de0910 2024-08-19T01:36:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01aea70 con 0x55e3a6deb1d0 2024-08-19T01:36:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54b7058740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01ad600 con 0x55e3a6ddfff0 2024-08-19T01:36:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.396+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6de0910 0x55e3a6e02b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6de0910 0x55e3a6e02b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37400/0 (socket says 172.21.15.103:37400) 2024-08-19T01:36:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 -- 172.21.15.103:0/2408653009 learned_addr learned my addr 172.21.15.103:0/2408653009 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54ae59c640 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 -- 172.21.15.103:0/2408653009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6ddfff0 msgr2=0x55e3a6de2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 -- 172.21.15.103:0/2408653009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 msgr2=0x55e3a6e03050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54aed9d640 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6e03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6e03050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 -- 172.21.15.103:0/2408653009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a6dfd880 con 0x55e3a6de0910 2024-08-19T01:36:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f54a7fff640 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6de0910 0x55e3a6e02b10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f549800a5c0 tx=0x7f5498010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbc3b15e0a82525 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f5497fff640 1 -- 172.21.15.103:0/2408653009 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498005020 con 0x55e3a6de0910 2024-08-19T01:36:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f5497fff640 1 -- 172.21.15.103:0/2408653009 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54980047a0 con 0x55e3a6de0910 2024-08-19T01:36:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.397+0000 7f5497fff640 1 -- 172.21.15.103:0/2408653009 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549801c680 con 0x55e3a6de0910 2024-08-19T01:36:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 -- 172.21.15.103:0/2408653009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6de0910 msgr2=0x55e3a6e02b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6de0910 0x55e3a6e02b10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f549800a5c0 tx=0x7f5498010800 comp rx=0 tx=0).stop 2024-08-19T01:36:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 -- 172.21.15.103:0/2408653009 shutdown_connections 2024-08-19T01:36:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6deb1d0 0x55e3a6e03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6de0910 0x55e3a6e02b10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 --2- 172.21.15.103:0/2408653009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6ddfff0 0x55e3a6de2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 -- 172.21.15.103:0/2408653009 >> 172.21.15.103:0/2408653009 conn(0x55e3a6ddc580 msgr2=0x55e3a6de6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:30.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 -- 172.21.15.103:0/2408653009 shutdown_connections 2024-08-19T01:36:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 -- 172.21.15.103:0/2408653009 wait complete. 2024-08-19T01:36:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 -- start start 2024-08-19T01:36:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.398+0000 7f54b7058740 1 Processor -- start 2024-08-19T01:36:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6df33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6df3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01ae3a0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01adcd0 con 0x55e3a6de0910 2024-08-19T01:36:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01aaf10 con 0x55e3a6deb1d0 2024-08-19T01:36:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 10 client.0 _ll_register_callbacks cb 0x7f54a0009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6df33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6df33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37410/0 (socket says 172.21.15.103:37410) 2024-08-19T01:36:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 -- 172.21.15.103:0/784750763 learned_addr learned my addr 172.21.15.103:0/784750763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:30.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54a7fff640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6df3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 -- 172.21.15.103:0/784750763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 msgr2=0x55e3a6de4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54aed9d640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 -- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 msgr2=0x55e3a6df3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:30.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6df3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 -- 172.21.15.103:0/784750763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a6de5090 con 0x55e3a6ddfff0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54a7fff640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6df3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54aed9d640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54ae59c640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6df33c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f54a0038020 tx=0x7f54a00c4aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84efc92e21450533 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f5495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f5495ffb640 1 -- 172.21.15.103:0/784750763 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0049040 con 0x55e3a6ddfff0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f5495ffb640 1 -- 172.21.15.103:0/784750763 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f54a009abb0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.399+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54980047a0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f5495ffb640 1 -- 172.21.15.103:0/784750763 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54a00bc440 con 0x55e3a6ddfff0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 10 client.15600 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3a6e655a0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f5495ffb640 1 -- 172.21.15.103:0/784750763 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54980047a0 con 0x55e3a6ddfff0 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f5495ffb640 1 -- 172.21.15.103:0/784750763 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f54a00bb190 con 0x55e3a6ddfff0 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 20 client.15600 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6e65a70 0x55e3a6e67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 4 client.15600 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3a6d4d4c0 con 0x55e3a6e65a70 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.400+0000 7f54a7fff640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6e65a70 0x55e3a6e67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f5495ffb640 10 client.15600 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:30.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54a7fff640 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6e65a70 0x55e3a6e67e60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1759151955c0cfd server_cookie=abda2a79d0cf72bc in_seq=0 out_seq=0 2024-08-19T01:36:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f5495ffb640 1 -- 172.21.15.103:0/784750763 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3a6d4d4c0 con 0x55e3a6e65a70 2024-08-19T01:36:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f5495ffb640 10 client.15600 handle_command_reply: tid=0 2024-08-19T01:36:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 client.15600 shutdown 2024-08-19T01:36:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 msgr2=0x55e3a6df33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6df33c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f54a0038020 tx=0x7f54a00c4aa0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.435 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:36:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 shutdown_connections 2024-08-19T01:36:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3a6e65a70 0x55e3a6e67e60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3a6deb1d0 0x55e3a6de4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3a6de0910 0x55e3a6df3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f5495ffb640 0 client.15600 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 --2- 172.21.15.103:0/784750763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3a6ddfff0 0x55e3a6df33c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 >> 172.21.15.103:0/784750763 conn(0x55e3a6ddc580 msgr2=0x55e3a6de8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 shutdown_connections 2024-08-19T01:36:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 1 -- 172.21.15.103:0/784750763 wait complete. 2024-08-19T01:36:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.401+0000 7f54b7058740 20 client.15600 trim_cache size 0 max 16384 2024-08-19T01:36:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f549c0a2b80 msgr2=0x7f549c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f549c0a2b80 0x7f549c0a5040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f54a8021000 tx=0x7f54a801f000 comp rx=0 tx=0).stop 2024-08-19T01:36:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01007a0 msgr2=0x7f54b0198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01007a0 0x7f54b0198e20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f54a0005940 tx=0x7f54a00042f0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 shutdown_connections 2024-08-19T01:36:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f549c0a2b80 0x7f549c0a5040 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54b01067a0 0x7f54b01a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0101140 0x7f54b0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 --2- 172.21.15.103:0/2020122061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b01007a0 0x7f54b0198e20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:30.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 >> 172.21.15.103:0/2020122061 conn(0x7f54b00fc500 msgr2=0x7f54b00fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:30.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 shutdown_connections 2024-08-19T01:36:30.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:30.402+0000 7f54b4c5a640 1 -- 172.21.15.103:0/2020122061 wait complete. 2024-08-19T01:36:30.440 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-19T01:36:31.447 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether build environment is sane... yes 2024-08-19T01:36:31.461 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-08-19T01:36:31.461 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gawk... gawk 2024-08-19T01:36:31.482 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether make sets $(MAKE)... yes 2024-08-19T01:36:31.522 INFO:tasks.workunit.client.0.smithi103.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-19T01:36:31.522 INFO:tasks.workunit.client.0.smithi103.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-19T01:36:31.543 INFO:tasks.workunit.client.0.smithi103.stdout:checking for style of include used by make... GNU 2024-08-19T01:36:31.548 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc... gcc 2024-08-19T01:36:31.632 INFO:tasks.workunit.client.0.smithi103.stdout:checking for C compiler default output file name... a.out 2024-08-19T01:36:31.635 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether the C compiler works... yes 2024-08-19T01:36:31.638 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are cross compiling... no 2024-08-19T01:36:31.689 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of executables... 2024-08-19T01:36:31.730 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of object files... o 2024-08-19T01:36:31.770 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are using the GNU C compiler... yes 2024-08-19T01:36:31.809 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc accepts -g... yes 2024-08-19T01:36:31.915 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to accept ISO C89... none needed 2024-08-19T01:36:32.056 INFO:tasks.workunit.client.0.smithi103.stdout:checking dependency style of gcc... gcc3 2024-08-19T01:36:32.172 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to run the C preprocessor... gcc -E 2024-08-19T01:36:32.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:31 smithi146 ceph-mon[22098]: pgmap v361: 97 pgs: 97 active+clean; 709 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-08-19T01:36:32.217 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:36:32.217 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:36:32.229 INFO:tasks.workunit.client.0.smithi103.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-08-19T01:36:32.231 INFO:tasks.workunit.client.0.smithi103.stdout:checking for egrep... /usr/bin/grep -E 2024-08-19T01:36:32.241 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:31 smithi103 ceph-mon[18436]: pgmap v361: 97 pgs: 97 active+clean; 709 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-08-19T01:36:32.245 INFO:tasks.workunit.client.0.smithi103.stdout:checking for AIX... no 2024-08-19T01:36:32.269 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc... (cached) gcc 2024-08-19T01:36:32.283 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-08-19T01:36:32.283 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc accepts -g... (cached) yes 2024-08-19T01:36:32.284 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-08-19T01:36:32.285 INFO:tasks.workunit.client.0.smithi103.stdout:checking dependency style of gcc... (cached) gcc3 2024-08-19T01:36:32.358 INFO:tasks.workunit.client.0.smithi103.stdout:checking for main in -lm... yes 2024-08-19T01:36:32.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:31 smithi190 ceph-mon[22032]: pgmap v361: 97 pgs: 97 active+clean; 709 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-08-19T01:36:32.460 INFO:tasks.workunit.client.0.smithi103.stdout:checking for main in -lpthread... yes 2024-08-19T01:36:32.493 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:32.699 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ANSI C header files... yes 2024-08-19T01:36:32.823 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-08-19T01:36:32.879 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/types.h... yes 2024-08-19T01:36:32.946 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/stat.h... yes 2024-08-19T01:36:33.027 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdlib.h... yes 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 -- 172.21.15.103:0/906453232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201007d0 msgr2=0x7f7a2010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 --2- 172.21.15.103:0/906453232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201007d0 0x7f7a2010e750 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009ca0 tx=0x7f7a1802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 -- 172.21.15.103:0/906453232 shutdown_connections 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 --2- 172.21.15.103:0/906453232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201007d0 0x7f7a2010e750 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 --2- 172.21.15.103:0/906453232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a200ffe30 0x7f7a20100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 --2- 172.21.15.103:0/906453232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a201068b0 0x7f7a20106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 -- 172.21.15.103:0/906453232 >> 172.21.15.103:0/906453232 conn(0x7f7a20076f60 msgr2=0x7f7a200fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 -- 172.21.15.103:0/906453232 shutdown_connections 2024-08-19T01:36:33.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 -- 172.21.15.103:0/906453232 wait complete. 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.047+0000 7f7a284de640 1 Processor -- start 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a284de640 1 -- start start 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a284de640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a200ffe30 0x7f7a201980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a284de640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a201007d0 0x7f7a20198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a284de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201068b0 0x7f7a2019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a284de640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20111c80 con 0x7f7a201068b0 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a284de640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20111b00 con 0x7f7a201007d0 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a284de640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20111e00 con 0x7f7a200ffe30 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a200ffe30 0x7f7a201980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a26a7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201068b0 0x7f7a2019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a200ffe30 0x7f7a201980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:57146/0 (socket says 172.21.15.103:57146) 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 -- 172.21.15.103:0/2428486075 learned_addr learned my addr 172.21.15.103:0/2428486075 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a25a79640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a201007d0 0x7f7a20198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 -- 172.21.15.103:0/2428486075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a201007d0 msgr2=0x7f7a20198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a201007d0 0x7f7a20198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 -- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201068b0 msgr2=0x7f7a2019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201068b0 0x7f7a2019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a2627a640 1 -- 172.21.15.103:0/2428486075 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2019d070 con 0x7f7a200ffe30 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.048+0000 7f7a25a79640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a201007d0 0x7f7a20198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.049+0000 7f7a2627a640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a200ffe30 0x7f7a201980c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e9f0 tx=0x7f7a1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.049+0000 7f7a177fe640 1 -- 172.21.15.103:0/2428486075 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00cd80 con 0x7f7a200ffe30 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.049+0000 7f7a177fe640 1 -- 172.21.15.103:0/2428486075 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a1c0108e0 con 0x7f7a200ffe30 2024-08-19T01:36:33.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.049+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2019d360 con 0x7f7a200ffe30 2024-08-19T01:36:33.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.049+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a20100f40 con 0x7f7a200ffe30 2024-08-19T01:36:33.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.049+0000 7f7a177fe640 1 -- 172.21.15.103:0/2428486075 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c018920 con 0x7f7a200ffe30 2024-08-19T01:36:33.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.052+0000 7f7a177fe640 1 -- 172.21.15.103:0/2428486075 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7a1c018ac0 con 0x7f7a200ffe30 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.052+0000 7f7a2a8dc740 1 Processor -- start 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2a8dc740 1 -- start start 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598200 0x55b6df5985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6df55be40 con 0x55b6df5a33e0 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b6df4fd7f0 con 0x55b6df598b20 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6df4a4fb0 con 0x55b6df598200 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a26a7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598200 0x55b6df5985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a26a7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598200 0x55b6df5985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:57156/0 (socket says 172.21.15.103:57156) 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a26a7b640 1 -- 172.21.15.103:0/258398501 learned_addr learned my addr 172.21.15.103:0/258398501 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.053+0000 7f7a2627a640 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a25a79640 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a177fe640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a0c0a2b80 0x7f7a0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a2627a640 1 -- 172.21.15.103:0/258398501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598200 msgr2=0x55b6df5985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a2627a640 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598200 0x55b6df5985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a2627a640 1 -- 172.21.15.103:0/258398501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 msgr2=0x55b6df5a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a2627a640 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a2627a640 1 -- 172.21.15.103:0/258398501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6df26f6a0 con 0x55b6df598b20 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a26a7b640 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598200 0x55b6df5985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a26a7b640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a0c0a2b80 0x7f7a0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.054+0000 7f7a177fe640 1 -- 172.21.15.103:0/2428486075 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7a1c0cf380 con 0x7f7a200ffe30 2024-08-19T01:36:33.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.055+0000 7f7a2627a640 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a2ea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0ccdf0 tx=0x7f7a1c06bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef656936184df53 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.055+0000 7f7a14ff9640 1 -- 172.21.15.103:0/258398501 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c026030 con 0x55b6df598b20 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.055+0000 7f7a14ff9640 1 -- 172.21.15.103:0/258398501 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a1c010de0 con 0x55b6df598b20 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.055+0000 7f7a14ff9640 1 -- 172.21.15.103:0/258398501 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c09bc60 con 0x55b6df598b20 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/258398501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 msgr2=0x55b6df5a2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a26a7b640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a0c0a2b80 0x7f7a0c0a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a18004ad0 tx=0x7f7a180095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a2ea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0ccdf0 tx=0x7f7a1c06bc10 comp rx=0 tx=0).stop 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/258398501 shutdown_connections 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a2ea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/258398501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598200 0x55b6df5985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/258398501 >> 172.21.15.103:0/258398501 conn(0x55b6df594790 msgr2=0x55b6df594b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.056+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/258398501 shutdown_connections 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.057+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/258398501 wait complete. 2024-08-19T01:36:33.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.057+0000 7f7a2a8dc740 1 -- start start 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.057+0000 7f7a2a8dc740 1 Processor -- start 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.057+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.057+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.057+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df5a33e0 0x55b6df5a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6df4b8cb0 con 0x55b6df598200 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b6df4f97b0 con 0x55b6df598b20 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b6df5a5f50 con 0x55b6df5a33e0 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2a8dc740 10 client.0 _ll_register_callbacks cb 0x55b6df20cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34862/0 (socket says 172.21.15.103:34862) 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 -- 172.21.15.103:0/4051318028 learned_addr learned my addr 172.21.15.103:0/4051318028 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a25a79640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 -- 172.21.15.103:0/4051318028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df5a33e0 msgr2=0x55b6df5a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a26a7b640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df5a33e0 0x55b6df5a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df5a33e0 0x55b6df5a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 -- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 msgr2=0x55b6df5a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 -- 172.21.15.103:0/4051318028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6df4b7bc0 con 0x55b6df598200 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a25a79640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a2627a640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a02b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0a42f0 tx=0x7f7a1c09bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e107c6c906fc0a10 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.058+0000 7f7a09ffb640 1 -- 172.21.15.103:0/4051318028 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c09b7a0 con 0x55b6df598200 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.059+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6df50ad60 con 0x55b6df598200 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.059+0000 7f7a09ffb640 1 -- 172.21.15.103:0/4051318028 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7a1c099d00 con 0x55b6df598200 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.059+0000 7f7a09ffb640 1 -- 172.21.15.103:0/4051318028 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c09b5c0 con 0x55b6df598200 2024-08-19T01:36:33.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.059+0000 7f7a2a8dc740 10 client.15612 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.059+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b6df5b1df0 con 0x55b6df598200 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.059+0000 7f7a09ffb640 1 -- 172.21.15.103:0/4051318028 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a1c06f040 con 0x55b6df598200 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a09ffb640 1 -- 172.21.15.103:0/4051318028 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a1c0a4390 con 0x55b6df598200 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a2a8dc740 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a2a8dc740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a2a8dc740 20 client.15612 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df5b3140 0x55b6df5b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a2a8dc740 4 client.15612 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6df50ad60 con 0x55b6df5b3140 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a25a79640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df5b3140 0x55b6df5b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a09ffb640 10 client.15612 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.060+0000 7f7a25a79640 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df5b3140 0x55b6df5b5530 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76824ddaefbc007 server_cookie=e266e69f99c602f9 in_seq=0 out_seq=0 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a09ffb640 1 -- 172.21.15.103:0/4051318028 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6df50ad60 con 0x55b6df5b3140 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a09ffb640 10 client.15612 handle_command_reply: tid=0 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 client.15612 shutdown 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 msgr2=0x55b6df5a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a02b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0a42f0 tx=0x7f7a1c09bb40 comp rx=0 tx=0).stop 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 shutdown_connections 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df5b3140 0x55b6df5b5530 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df5a33e0 0x55b6df5a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598b20 0x55b6df5a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/4051318028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a02b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a09ffb640 0 client.15612 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.061+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 >> 172.21.15.103:0/4051318028 conn(0x55b6df594790 msgr2=0x55b6df59a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.062+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 shutdown_connections 2024-08-19T01:36:33.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.062+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/4051318028 wait complete. 2024-08-19T01:36:33.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.062+0000 7f7a2a8dc740 20 client.15612 trim_cache size 0 max 16384 2024-08-19T01:36:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.080+0000 7f7a2a8dc740 1 Processor -- start 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.080+0000 7f7a2a8dc740 1 -- start start 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598200 0x55b6df59f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201aaef0 con 0x55b6df5a33e0 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201aca30 con 0x55b6df598200 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ab5c0 con 0x55b6df598b20 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a26a7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a26a7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5bab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34864/0 (socket says 172.21.15.103:34864) 2024-08-19T01:36:33.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a26a7b640 1 -- 172.21.15.103:0/1705580444 learned_addr learned my addr 172.21.15.103:0/1705580444 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a2627a640 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598200 0x55b6df59f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a25a79640 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a25a79640 1 -- 172.21.15.103:0/1705580444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598200 msgr2=0x55b6df59f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a25a79640 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598200 0x55b6df59f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a25a79640 1 -- 172.21.15.103:0/1705580444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 msgr2=0x55b6df5bab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a25a79640 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5bab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.081+0000 7f7a25a79640 1 -- 172.21.15.103:0/1705580444 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6df598910 con 0x55b6df598b20 2024-08-19T01:36:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.082+0000 7f7a25a79640 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a0b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b6df5b5b00 tx=0x7f7a10005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cc34bd1c040bfdc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.082+0000 7f7a0bfff640 1 -- 172.21.15.103:0/1705580444 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10004030 con 0x55b6df598b20 2024-08-19T01:36:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.082+0000 7f7a0bfff640 1 -- 172.21.15.103:0/1705580444 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a1000f890 con 0x55b6df598b20 2024-08-19T01:36:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.082+0000 7f7a0bfff640 1 -- 172.21.15.103:0/1705580444 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10003850 con 0x55b6df598b20 2024-08-19T01:36:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/1705580444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 msgr2=0x55b6df5a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a0b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55b6df5b5b00 tx=0x7f7a10005f90 comp rx=0 tx=0).stop 2024-08-19T01:36:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/1705580444 shutdown_connections 2024-08-19T01:36:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df5a33e0 0x55b6df5bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a0b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/1705580444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df598200 0x55b6df59f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/1705580444 >> 172.21.15.103:0/1705580444 conn(0x55b6df594790 msgr2=0x55b6df5a2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/1705580444 shutdown_connections 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/1705580444 wait complete. 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.083+0000 7f7a2a8dc740 1 -- start start 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 1 Processor -- start 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df5a33e0 0x55b6df5a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ac360 con 0x55b6df598200 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201abc90 con 0x55b6df5a33e0 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201a8ed0 con 0x55b6df598b20 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2a8dc740 10 client.0 _ll_register_callbacks cb 0x7f7a1c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a25a79640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a25a79640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:57192/0 (socket says 172.21.15.103:57192) 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a25a79640 1 -- 172.21.15.103:0/729485847 learned_addr learned my addr 172.21.15.103:0/729485847 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a26a7b640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df5a33e0 0x55b6df5a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.084+0000 7f7a2627a640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a25a79640 1 -- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df5a33e0 msgr2=0x55b6df5a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a25a79640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df5a33e0 0x55b6df5a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a25a79640 1 -- 172.21.15.103:0/729485847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 msgr2=0x55b6df5a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a25a79640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a25a79640 1 -- 172.21.15.103:0/729485847 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6df61cdd0 con 0x55b6df598b20 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a2627a640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a25a79640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a76f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7a100058c0 tx=0x7f7a100122e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5fc0541f1c466a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.085+0000 7f7a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a09ffb640 1 -- 172.21.15.103:0/729485847 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000fe50 con 0x55b6df598b20 2024-08-19T01:36:33.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a09ffb640 1 -- 172.21.15.103:0/729485847 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7a10005070 con 0x55b6df598b20 2024-08-19T01:36:33.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a1000f890 con 0x55b6df598b20 2024-08-19T01:36:33.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a09ffb640 1 -- 172.21.15.103:0/729485847 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a1003b050 con 0x55b6df598b20 2024-08-19T01:36:33.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 10 client.25141 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b6df61e110 con 0x55b6df598b20 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a09ffb640 1 -- 172.21.15.103:0/729485847 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a100046c0 con 0x55b6df598b20 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a09ffb640 1 -- 172.21.15.103:0/729485847 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a10010550 con 0x55b6df598b20 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 10 client.25141 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 10 client.25141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 20 client.25141 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df61e850 0x55b6df620c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 4 client.25141 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.086+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7a1000f890 con 0x55b6df61e850 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.087+0000 7f7a2627a640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df61e850 0x55b6df620c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.087+0000 7f7a09ffb640 10 client.25141 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.087+0000 7f7a2627a640 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df61e850 0x55b6df620c70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da30e66a980ecc3b server_cookie=f7356baa4b8d2e7f in_seq=0 out_seq=0 2024-08-19T01:36:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.088+0000 7f7a09ffb640 1 -- 172.21.15.103:0/729485847 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7a1000f890 con 0x55b6df61e850 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.088+0000 7f7a09ffb640 10 client.25141 handle_command_reply: tid=0 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.088+0000 7f7a2a8dc740 1 client.25141 shutdown 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.088+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 msgr2=0x55b6df5a76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.088+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a76f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7a100058c0 tx=0x7f7a100122e0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 shutdown_connections 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b6df61e850 0x55b6df620c70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b6df5a33e0 0x55b6df5a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b6df598b20 0x55b6df5a76f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a09ffb640 0 client.25141 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 --2- 172.21.15.103:0/729485847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6df598200 0x55b6df5a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 >> 172.21.15.103:0/729485847 conn(0x55b6df594790 msgr2=0x55b6df5930d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 shutdown_connections 2024-08-19T01:36:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 1 -- 172.21.15.103:0/729485847 wait complete. 2024-08-19T01:36:33.094 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:36:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:36:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "34aed164-435f-48fd-a97e-65d939a99c2c", 2024-08-19T01:36:33.094 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:36:33.094 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.089+0000 7f7a2a8dc740 20 client.25141 trim_cache size 0 max 16384 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a0c0a2b80 msgr2=0x7f7a0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a0c0a2b80 0x7f7a0c0a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a18004ad0 tx=0x7f7a180095c0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a200ffe30 msgr2=0x7f7a201980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a200ffe30 0x7f7a201980c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e9f0 tx=0x7f7a1c00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 shutdown_connections 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7a0c0a2b80 0x7f7a0c0a5040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a201068b0 0x7f7a2019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a201007d0 0x7f7a20198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 --2- 172.21.15.103:0/2428486075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a200ffe30 0x7f7a201980c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 >> 172.21.15.103:0/2428486075 conn(0x7f7a20076f60 msgr2=0x7f7a200fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 shutdown_connections 2024-08-19T01:36:33.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:33.090+0000 7f7a284de640 1 -- 172.21.15.103:0/2428486075 wait complete. 2024-08-19T01:36:33.110 INFO:tasks.workunit.client.0.smithi103.stdout:checking for string.h... yes 2024-08-19T01:36:33.195 INFO:tasks.workunit.client.0.smithi103.stdout:checking for memory.h... yes 2024-08-19T01:36:33.263 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strings.h... yes 2024-08-19T01:36:33.340 INFO:tasks.workunit.client.0.smithi103.stdout:checking for inttypes.h... yes 2024-08-19T01:36:33.422 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdint.h... yes 2024-08-19T01:36:33.508 INFO:tasks.workunit.client.0.smithi103.stdout:checking for unistd.h... yes 2024-08-19T01:36:33.616 INFO:tasks.workunit.client.0.smithi103.stdout:checking pthread.h usability... yes 2024-08-19T01:36:33.647 INFO:tasks.workunit.client.0.smithi103.stdout:checking pthread.h presence... yes 2024-08-19T01:36:33.647 INFO:tasks.workunit.client.0.smithi103.stdout:checking for pthread.h... yes 2024-08-19T01:36:33.722 INFO:tasks.workunit.client.0.smithi103.stdout:checking fcntl.h usability... yes 2024-08-19T01:36:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 -- 172.21.15.103:0/1000618983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e80ffb80 msgr2=0x7f51e80fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 --2- 172.21.15.103:0/1000618983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e80ffb80 0x7f51e80fff80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009900 tx=0x7f51d802a410 comp rx=0 tx=0).stop 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 -- 172.21.15.103:0/1000618983 shutdown_connections 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 --2- 172.21.15.103:0/1000618983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e81035b0 0x7f51e8109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 --2- 172.21.15.103:0/1000618983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51e8102bf0 0x7f51e8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 --2- 172.21.15.103:0/1000618983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e80ffb80 0x7f51e80fff80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 -- 172.21.15.103:0/1000618983 >> 172.21.15.103:0/1000618983 conn(0x7f51e80fb6a0 msgr2=0x7f51e80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 -- 172.21.15.103:0/1000618983 shutdown_connections 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.731+0000 7f51eea37640 1 -- 172.21.15.103:0/1000618983 wait complete. 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 Processor -- start 2024-08-19T01:36:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 -- start start 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51e80ffb80 0x7f51e8102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e8102bf0 0x7f51e8100ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e81035b0 0x7f51e81010e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51e810d3e0 con 0x7f51e8102bf0 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51e810d260 con 0x7f51e80ffb80 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51eea37640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51e810d560 con 0x7f51e81035b0 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e8102bf0 0x7f51e8100ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e8102bf0 0x7f51e8100ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34880/0 (socket says 172.21.15.103:34880) 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.732+0000 7f51e77fe640 1 -- 172.21.15.103:0/3243880455 learned_addr learned my addr 172.21.15.103:0/3243880455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e7fff640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51e80ffb80 0x7f51e8102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e81035b0 0x7f51e81010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e77fe640 1 -- 172.21.15.103:0/3243880455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e81035b0 msgr2=0x7f51e81010e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e77fe640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e81035b0 0x7f51e81010e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e77fe640 1 -- 172.21.15.103:0/3243880455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51e80ffb80 msgr2=0x7f51e8102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e77fe640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51e80ffb80 0x7f51e8102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e77fe640 1 -- 172.21.15.103:0/3243880455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e8101970 con 0x7f51e8102bf0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e77fe640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e8102bf0 0x7f51e8100ba0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ea60 tx=0x7f51dc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e57fa640 1 -- 172.21.15.103:0/3243880455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00cde0 con 0x7f51e8102bf0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e81a1c00 con 0x7f51e8102bf0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e57fa640 1 -- 172.21.15.103:0/3243880455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51dc0035f0 con 0x7f51e8102bf0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51e57fa640 1 -- 172.21.15.103:0/3243880455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc010690 con 0x7f51e8102bf0 2024-08-19T01:36:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.733+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e81a21a0 con 0x7f51e8102bf0 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e57fa640 1 -- 172.21.15.103:0/3243880455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f51dc010830 con 0x7f51e8102bf0 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 Processor -- start 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 -- start start 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb5fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb62510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560aadabd9d0 con 0x560aadb5afc0 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560aada65820 con 0x560aadb5a680 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51f17fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560aadab9990 con 0x560aadb60050 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb5fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb5fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34890/0 (socket says 172.21.15.103:34890) 2024-08-19T01:36:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e7fff640 1 -- 172.21.15.103:0/530592392 learned_addr learned my addr 172.21.15.103:0/530592392 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e77fe640 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb62510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e57fa640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51d00a2cb0 0x7f51d00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e7fff640 1 -- 172.21.15.103:0/530592392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 msgr2=0x560aadb62510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e7fff640 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb62510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e7fff640 1 -- 172.21.15.103:0/530592392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 msgr2=0x560aadb5aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.736+0000 7f51e7fff640 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb5aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.737+0000 7f51e7fff640 1 -- 172.21.15.103:0/530592392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aad7d1210 con 0x560aadb5afc0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.737+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51d00a2cb0 0x7f51d00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.737+0000 7f51e57fa640 1 -- 172.21.15.103:0/3243880455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f51dc0cf280 con 0x7f51e8102bf0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.737+0000 7f51e7fff640 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb5fa80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f51e8069d90 tx=0x7f51d8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79afef3fb44aad91 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.737+0000 7f51ce7fc640 1 -- 172.21.15.103:0/530592392 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8040070 con 0x560aadb5afc0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.737+0000 7f51ce7fc640 1 -- 172.21.15.103:0/530592392 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d8004030 con 0x560aadb5afc0 2024-08-19T01:36:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.737+0000 7f51ce7fc640 1 -- 172.21.15.103:0/530592392 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8035040 con 0x560aadb5afc0 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 -- 172.21.15.103:0/530592392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 msgr2=0x560aadb5fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51d00a2cb0 0x7f51d00a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009fd0 tx=0x7f51e0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb5fa80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f51e8069d90 tx=0x7f51d8005e40 comp rx=0 tx=0).stop 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 -- 172.21.15.103:0/530592392 shutdown_connections 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb62510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb5fa80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 --2- 172.21.15.103:0/530592392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb5aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 -- 172.21.15.103:0/530592392 >> 172.21.15.103:0/530592392 conn(0x560aadb55cf0 msgr2=0x560aadb560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 -- 172.21.15.103:0/530592392 shutdown_connections 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 -- 172.21.15.103:0/530592392 wait complete. 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.738+0000 7f51f17fc740 1 -- start start 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 1 Processor -- start 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb5a680 0x560aadb64860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb64da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb60050 0x560aadb6bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560aadb62fc0 con 0x560aadb5afc0 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560aada79520 con 0x560aadb60050 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560aadb63140 con 0x560aadb5a680 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51f17fc740 10 client.0 _ll_register_callbacks cb 0x560aad7d0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb5a680 0x560aadb64860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb64da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb5a680 0x560aadb64860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:57232/0 (socket says 172.21.15.103:57232) 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 -- 172.21.15.103:0/3292880056 learned_addr learned my addr 172.21.15.103:0/3292880056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 -- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb60050 msgr2=0x560aadb6bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb60050 0x560aadb6bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb60050 0x560aadb6bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 -- 172.21.15.103:0/3292880056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 msgr2=0x560aadb64da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb64da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e7fff640 1 -- 172.21.15.103:0/3292880056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aad89d620 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.739+0000 7f51e77fe640 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb64da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51e7fff640 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb5a680 0x560aadb64860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f51d802a930 tx=0x7f51d8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f658d868ac75fe2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51c77fe640 1 -- 172.21.15.103:0/3292880056 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8040070 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51c77fe640 1 -- 172.21.15.103:0/3292880056 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f51d8031580 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d8004030 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51c77fe640 1 -- 172.21.15.103:0/3292880056 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51d8035040 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 10 client.25153 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560aadb62fc0 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51c77fe640 1 -- 172.21.15.103:0/3292880056 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d8004030 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51c77fe640 1 -- 172.21.15.103:0/3292880056 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f51d8004c40 con 0x560aadb5a680 2024-08-19T01:36:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 10 client.25153 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 10 client.25153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 20 client.25153 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadb6fdf0 0x560aadb721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 4 client.25153 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.740+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aadacb100 con 0x560aadb6fdf0 2024-08-19T01:36:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.741+0000 7f51e77fe640 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadb6fdf0 0x560aadb721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.741+0000 7f51e77fe640 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadb6fdf0 0x560aadb721e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6fef93a9f617c2 server_cookie=197fe57251624668 in_seq=0 out_seq=0 2024-08-19T01:36:33.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.741+0000 7f51c77fe640 10 client.25153 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51c77fe640 1 -- 172.21.15.103:0/3292880056 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560aadacb100 con 0x560aadb6fdf0 2024-08-19T01:36:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51c77fe640 10 client.25153 handle_command_reply: tid=0 2024-08-19T01:36:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 client.25153 shutdown 2024-08-19T01:36:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb5a680 msgr2=0x560aadb64860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb5a680 0x560aadb64860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f51d802a930 tx=0x7f51d8009580 comp rx=0 tx=0).stop 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 shutdown_connections 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadb6fdf0 0x560aadb721e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb60050 0x560aadb6bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb64da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51c77fe640 0 client.25153 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 --2- 172.21.15.103:0/3292880056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb5a680 0x560aadb64860 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 >> 172.21.15.103:0/3292880056 conn(0x560aadb55cf0 msgr2=0x560aadb57500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 shutdown_connections 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 1 -- 172.21.15.103:0/3292880056 wait complete. 2024-08-19T01:36:33.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.742+0000 7f51f17fc740 20 client.25153 trim_cache size 0 max 16384 2024-08-19T01:36:33.752 INFO:tasks.workunit.client.0.smithi103.stdout:checking fcntl.h presence... yes 2024-08-19T01:36:33.753 INFO:tasks.workunit.client.0.smithi103.stdout:checking for fcntl.h... yes 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 Processor -- start 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 -- start start 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5a680 0x560aadb64e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5afc0 0x560aadb6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb54d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81aae00 con 0x560aadb5a680 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81ac940 con 0x560aadb5afc0 2024-08-19T01:36:33.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51f17fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81ab4d0 con 0x560aadb60050 2024-08-19T01:36:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5a680 0x560aadb64e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5a680 0x560aadb64e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34902/0 (socket says 172.21.15.103:34902) 2024-08-19T01:36:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5afc0 0x560aadb6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51e7fff640 1 -- 172.21.15.103:0/1231024650 learned_addr learned my addr 172.21.15.103:0/1231024650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb54d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51e77fe640 1 -- 172.21.15.103:0/1231024650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 msgr2=0x560aadb54d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.758+0000 7f51e77fe640 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb54d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.759+0000 7f51e77fe640 1 -- 172.21.15.103:0/1231024650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5a680 msgr2=0x560aadb64e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.759+0000 7f51e77fe640 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5a680 0x560aadb64e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.759+0000 7f51e77fe640 1 -- 172.21.15.103:0/1231024650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aadb64400 con 0x560aadb5afc0 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.759+0000 7f51e77fe640 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5afc0 0x560aadb6bda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0cce00 tx=0x7f51dc0cf670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91fbd35b8ff54c3c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.759+0000 7f51cdffb640 1 -- 172.21.15.103:0/1231024650 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc09b650 con 0x560aadb5afc0 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.759+0000 7f51cdffb640 1 -- 172.21.15.103:0/1231024650 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51dc0b4070 con 0x560aadb5afc0 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.759+0000 7f51cdffb640 1 -- 172.21.15.103:0/1231024650 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc09b470 con 0x560aadb5afc0 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 -- 172.21.15.103:0/1231024650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5afc0 msgr2=0x560aadb6bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5afc0 0x560aadb6bda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0cce00 tx=0x7f51dc0cf670 comp rx=0 tx=0).stop 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 -- 172.21.15.103:0/1231024650 shutdown_connections 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb54d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5afc0 0x560aadb6bda0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1231024650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5a680 0x560aadb64e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 -- 172.21.15.103:0/1231024650 >> 172.21.15.103:0/1231024650 conn(0x560aadb55cf0 msgr2=0x560aadb54680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 -- 172.21.15.103:0/1231024650 shutdown_connections 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 -- 172.21.15.103:0/1231024650 wait complete. 2024-08-19T01:36:33.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.760+0000 7f51f17fc740 1 -- start start 2024-08-19T01:36:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 1 Processor -- start 2024-08-19T01:36:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb6a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81ac270 con 0x560aadb5afc0 2024-08-19T01:36:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81abba0 con 0x560aadb5a680 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81a8de0 con 0x560aadb60050 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51f17fc740 10 client.0 _ll_register_callbacks cb 0x7f51e0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb693a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39876/0 (socket says 172.21.15.103:39876) 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34914/0 (socket says 172.21.15.103:34914) 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 -- 172.21.15.103:0/1535028832 learned_addr learned my addr 172.21.15.103:0/1535028832 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb6a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 -- 172.21.15.103:0/1535028832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 msgr2=0x560aadb6a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb6a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 -- 172.21.15.103:0/1535028832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 msgr2=0x560aadb698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e7fff640 1 -- 172.21.15.103:0/1535028832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aadb6af00 con 0x560aadb5a680 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51e77fe640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.761+0000 7f51ecfd4640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb6a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51e7fff640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb693a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51d802a450 tx=0x7f51d8032b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe8da36d8fcfd05 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51c77fe640 1 -- 172.21.15.103:0/1535028832 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8040050 con 0x560aadb5a680 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51dc0b4070 con 0x560aadb5a680 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51c77fe640 1 -- 172.21.15.103:0/1535028832 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f51d8003bb0 con 0x560aadb5a680 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51c77fe640 1 -- 172.21.15.103:0/1535028832 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51d8003db0 con 0x560aadb5a680 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 10 client.25181 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560aadbdaac0 con 0x560aadb5a680 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51c77fe640 1 -- 172.21.15.103:0/1535028832 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51dc0b4070 con 0x560aadb5a680 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51c77fe640 1 -- 172.21.15.103:0/1535028832 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f51d80047e0 con 0x560aadb5a680 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 10 client.25181 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 10 client.25181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 20 client.25181 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadbdb0d0 0x560aadbdd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 4 client.25181 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.762+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560aadac5450 con 0x560aadbdb0d0 2024-08-19T01:36:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.763+0000 7f51e77fe640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadbdb0d0 0x560aadbdd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.763+0000 7f51c77fe640 10 client.25181 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.763+0000 7f51e77fe640 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadbdb0d0 0x560aadbdd4f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ecefc29f309e8c server_cookie=249092ce8ceac3ff in_seq=0 out_seq=0 2024-08-19T01:36:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51c77fe640 1 -- 172.21.15.103:0/1535028832 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560aadac5450 con 0x560aadbdb0d0 2024-08-19T01:36:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51c77fe640 10 client.25181 handle_command_reply: tid=0 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 client.25181 shutdown 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 msgr2=0x560aadb693a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb693a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51d802a450 tx=0x7f51d8032b00 comp rx=0 tx=0).stop 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 shutdown_connections 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560aadbdb0d0 0x560aadbdd4f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560aadb60050 0x560aadb6a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560aadb5afc0 0x560aadb698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51c77fe640 0 client.25181 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 --2- 172.21.15.103:0/1535028832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560aadb5a680 0x560aadb693a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 >> 172.21.15.103:0/1535028832 conn(0x560aadb55cf0 msgr2=0x560aadb54ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 shutdown_connections 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 1 -- 172.21.15.103:0/1535028832 wait complete. 2024-08-19T01:36:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.845+0000 7f51f17fc740 20 client.25181 trim_cache size 0 max 16384 2024-08-19T01:36:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:36:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:36:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:36:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:36:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51d00a2cb0 msgr2=0x7f51d00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51d00a2cb0 0x7f51d00a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009fd0 tx=0x7f51e0004a40 comp rx=0 tx=0).stop 2024-08-19T01:36:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e8102bf0 msgr2=0x7f51e8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e8102bf0 0x7f51e8100ba0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ea60 tx=0x7f51dc00ef20 comp rx=0 tx=0).stop 2024-08-19T01:36:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 shutdown_connections 2024-08-19T01:36:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51d00a2cb0 0x7f51d00a5170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e81035b0 0x7f51e81010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e8102bf0 0x7f51e8100ba0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 --2- 172.21.15.103:0/3243880455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51e80ffb80 0x7f51e8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 >> 172.21.15.103:0/3243880455 conn(0x7f51e80fb6a0 msgr2=0x7f51e8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 shutdown_connections 2024-08-19T01:36:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:33.846+0000 7f51eea37640 1 -- 172.21.15.103:0/3243880455 wait complete. 2024-08-19T01:36:33.877 INFO:tasks.workunit.client.0.smithi103.stdout:checking limits.h usability... yes 2024-08-19T01:36:33.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: pgmap v362: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-19T01:36:33.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:36:33.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:36:33.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:33.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:36:33.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:33.903 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:36:33.903 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:36:33.904 INFO:tasks.workunit.client.0.smithi103.stdout:checking limits.h presence... yes 2024-08-19T01:36:33.905 INFO:tasks.workunit.client.0.smithi103.stdout:checking for limits.h... yes 2024-08-19T01:36:33.930 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdint.h... (cached) yes 2024-08-19T01:36:34.030 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h usability... yes 2024-08-19T01:36:34.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:36:34.059 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h presence... yes 2024-08-19T01:36:34.060 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/time.h... yes 2024-08-19T01:36:34.083 INFO:tasks.workunit.client.0.smithi103.stdout:checking for unistd.h... (cached) yes 2024-08-19T01:36:34.160 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/vfs.h usability... yes 2024-08-19T01:36:34.189 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/vfs.h presence... yes 2024-08-19T01:36:34.190 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/vfs.h... yes 2024-08-19T01:36:34.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: pgmap v362: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:36:34.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:36:34.257 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/limits.h usability... no 2024-08-19T01:36:34.298 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/limits.h presence... no 2024-08-19T01:36:34.299 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/limits.h... no 2024-08-19T01:36:34.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:36:34.351 INFO:tasks.workunit.client.0.smithi103.stdout:checking for an ANSI C-conforming const... yes 2024-08-19T01:36:34.352 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:36:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: pgmap v362: 97 pgs: 97 active+clean; 2.8 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-19T01:36:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:36:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:36:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:36:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:36:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:36:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:36:34.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:36:34.400 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-19T01:36:34.402 INFO:tasks.workunit.client.0.smithi103.stdout:checking for function prototypes... yes 2024-08-19T01:36:34.484 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether setvbuf arguments are reversed... no 2024-08-19T01:36:34.567 INFO:tasks.workunit.client.0.smithi103.stdout:checking for system... yes 2024-08-19T01:36:34.652 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gettimeofday... yes 2024-08-19T01:36:34.757 INFO:tasks.workunit.client.0.smithi103.stdout:checking for mkdir... yes 2024-08-19T01:36:34.843 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strerror... yes 2024-08-19T01:36:34.932 INFO:tasks.workunit.client.0.smithi103.stdout:checking for open64... yes 2024-08-19T01:36:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.936+0000 7ff145509640 1 -- 172.21.15.103:0/732038104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff140101160 msgr2=0x7ff14010d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.936+0000 7ff145509640 1 --2- 172.21.15.103:0/732038104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff140101160 0x7ff14010d690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff138009cd0 tx=0x7ff13802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.936+0000 7ff145509640 1 -- 172.21.15.103:0/732038104 shutdown_connections 2024-08-19T01:36:34.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.936+0000 7ff145509640 1 --2- 172.21.15.103:0/732038104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff140101160 0x7ff14010d690 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.936+0000 7ff145509640 1 --2- 172.21.15.103:0/732038104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1401007c0 0x7ff140100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.936+0000 7ff145509640 1 --2- 172.21.15.103:0/732038104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1401067c0 0x7ff140106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.937+0000 7ff145509640 1 -- 172.21.15.103:0/732038104 >> 172.21.15.103:0/732038104 conn(0x7ff1400fc500 msgr2=0x7ff1400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.937+0000 7ff145509640 1 -- 172.21.15.103:0/732038104 shutdown_connections 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.937+0000 7ff145509640 1 -- 172.21.15.103:0/732038104 wait complete. 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.937+0000 7ff145509640 1 Processor -- start 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.937+0000 7ff145509640 1 -- start start 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.937+0000 7ff145509640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1401007c0 0x7ff1401980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff145509640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140101160 0x7ff1401985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff145509640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1401067c0 0x7ff14019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff145509640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff140110a70 con 0x7ff1401067c0 2024-08-19T01:36:34.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff145509640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff1401108f0 con 0x7ff140101160 2024-08-19T01:36:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff145509640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff140110bf0 con 0x7ff1401007c0 2024-08-19T01:36:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1401067c0 0x7ff14019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1401067c0 0x7ff14019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34928/0 (socket says 172.21.15.103:34928) 2024-08-19T01:36:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 -- 172.21.15.103:0/2192258371 learned_addr learned my addr 172.21.15.103:0/2192258371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13effd640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1401007c0 0x7ff1401980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff137fff640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140101160 0x7ff1401985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 -- 172.21.15.103:0/2192258371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1401007c0 msgr2=0x7ff1401980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1401007c0 0x7ff1401980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 -- 172.21.15.103:0/2192258371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140101160 msgr2=0x7ff1401985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140101160 0x7ff1401985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 -- 172.21.15.103:0/2192258371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14019d050 con 0x7ff1401067c0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13f7fe640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1401067c0 0x7ff14019c970 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff138033040 tx=0x7ff138009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff13cff9640 1 -- 172.21.15.103:0/2192258371 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138004040 con 0x7ff1401067c0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.938+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14019d2e0 con 0x7ff1401067c0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.939+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401a4c20 con 0x7ff1401067c0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.939+0000 7ff13cff9640 1 -- 172.21.15.103:0/2192258371 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff138004690 con 0x7ff1401067c0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.939+0000 7ff13cff9640 1 -- 172.21.15.103:0/2192258371 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138030da0 con 0x7ff1401067c0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 Processor -- start 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff13cff9640 1 -- 172.21.15.103:0/2192258371 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff13803b400 con 0x7ff1401067c0 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 -- start start 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc509980 con 0x55c0fc5a49f0 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc4a1a70 con 0x55c0fc5b3800 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff147907740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc505940 con 0x55c0fc5a40d0 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff13effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff13effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34942/0 (socket says 172.21.15.103:34942) 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff13effd640 1 -- 172.21.15.103:0/1910705428 learned_addr learned my addr 172.21.15.103:0/1910705428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.942+0000 7ff137fff640 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff13cff9640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff12c0a2b80 0x7ff12c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff137fff640 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff13f7fe640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff12c0a2b80 0x7ff12c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff137fff640 1 -- 172.21.15.103:0/1910705428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 msgr2=0x55c0fc5a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff137fff640 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff137fff640 1 -- 172.21.15.103:0/1910705428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 msgr2=0x55c0fc5b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff13cff9640 1 -- 172.21.15.103:0/2192258371 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff13802e030 con 0x7ff1401067c0 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff137fff640 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff137fff640 1 -- 172.21.15.103:0/1910705428 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fc5046e0 con 0x55c0fc5b3800 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff13effd640 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff137fff640 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b5bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff128009730 tx=0x7ff1280096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8acee5f4003fd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff13f7fe640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff12c0a2b80 0x7ff12c0a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff138009df0 tx=0x7ff1380871a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff1357fa640 1 -- 172.21.15.103:0/1910705428 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128004050 con 0x55c0fc5b3800 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff1357fa640 1 -- 172.21.15.103:0/1910705428 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1280054d0 con 0x55c0fc5b3800 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.943+0000 7ff1357fa640 1 -- 172.21.15.103:0/1910705428 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128004c60 con 0x55c0fc5b3800 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 -- 172.21.15.103:0/1910705428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 msgr2=0x55c0fc5b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b5bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff128009730 tx=0x7ff1280096c0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 -- 172.21.15.103:0/1910705428 shutdown_connections 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b5bf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 --2- 172.21.15.103:0/1910705428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 -- 172.21.15.103:0/1910705428 >> 172.21.15.103:0/1910705428 conn(0x55c0fc5a07a0 msgr2=0x55c0fc5a0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.944+0000 7ff147907740 1 -- 172.21.15.103:0/1910705428 shutdown_connections 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 -- 172.21.15.103:0/1910705428 wait complete. 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 -- start start 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 Processor -- start 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc5b63d0 con 0x55c0fc5a40d0 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc4c4aa0 con 0x55c0fc5b3800 2024-08-19T01:36:34.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c0fc5b6550 con 0x55c0fc5a49f0 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff147907740 10 client.0 _ll_register_callbacks cb 0x55c0fc4b3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34956/0 (socket says 172.21.15.103:34956) 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.945+0000 7ff137fff640 1 -- 172.21.15.103:0/2742171051 learned_addr learned my addr 172.21.15.103:0/2742171051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff137fff640 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff13effd640 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff13effd640 1 -- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 msgr2=0x55c0fc5b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff13effd640 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff13effd640 1 -- 172.21.15.103:0/2742171051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a40d0 msgr2=0x55c0fc5b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff13effd640 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff13effd640 1 -- 172.21.15.103:0/2742171051 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fc4a8650 con 0x55c0fc5a49f0 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff13effd640 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b0cb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff130012730 tx=0x7ff130012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e687e839af224bb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:34.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff1267fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff1267fc640 1 -- 172.21.15.103:0/2742171051 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130004050 con 0x55c0fc5a49f0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1280054d0 con 0x55c0fc5a49f0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.946+0000 7ff1267fc640 1 -- 172.21.15.103:0/2742171051 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff1300041f0 con 0x55c0fc5a49f0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff1267fc640 1 -- 172.21.15.103:0/2742171051 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff130023070 con 0x55c0fc5a49f0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 10 client.25171 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0fc27b3c0 con 0x55c0fc5a49f0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff1267fc640 1 -- 172.21.15.103:0/2742171051 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1280054d0 con 0x55c0fc5a49f0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff1267fc640 1 -- 172.21.15.103:0/2742171051 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff130003950 con 0x55c0fc5a49f0 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 10 client.25171 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 10 client.25171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 20 client.25171 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc5c3880 0x55c0fc5c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 4 client.25171 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0fc516ef0 con 0x55c0fc5c3880 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.947+0000 7ff137fff640 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc5c3880 0x55c0fc5c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff1267fc640 10 client.25171 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff137fff640 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc5c3880 0x55c0fc5c5ce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29179fc2f7c8ee9 server_cookie=bf25420bd1940cba in_seq=0 out_seq=0 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff1267fc640 1 -- 172.21.15.103:0/2742171051 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0fc516ef0 con 0x55c0fc5c3880 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff1267fc640 10 client.25171 handle_command_reply: tid=0 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 client.25171 shutdown 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 msgr2=0x55c0fc5b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b0cb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff130012730 tx=0x7ff130012bf0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 shutdown_connections 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc5c3880 0x55c0fc5c5ce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b0cb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff1267fc640 0 client.25171 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:34.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 --2- 172.21.15.103:0/2742171051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.948+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 >> 172.21.15.103:0/2742171051 conn(0x55c0fc5a07a0 msgr2=0x55c0fc5a6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:34.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.949+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 shutdown_connections 2024-08-19T01:36:34.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.949+0000 7ff147907740 1 -- 172.21.15.103:0/2742171051 wait complete. 2024-08-19T01:36:34.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.949+0000 7ff147907740 20 client.25171 trim_cache size 0 max 16384 2024-08-19T01:36:34.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 Processor -- start 2024-08-19T01:36:34.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 -- start start 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff12000e380 con 0x55c0fc5a49f0 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff12000fec0 con 0x55c0fc5b3800 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff147907740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff12000ea50 con 0x55c0fc5a40d0 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff13effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff13effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34968/0 (socket says 172.21.15.103:34968) 2024-08-19T01:36:34.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff137fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff13effd640 1 -- 172.21.15.103:0/2923554244 learned_addr learned my addr 172.21.15.103:0/2923554244 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.969+0000 7ff137fff640 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff137fff640 1 -- 172.21.15.103:0/2923554244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 msgr2=0x55c0fc5ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff137fff640 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff137fff640 1 -- 172.21.15.103:0/2923554244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 msgr2=0x55c0fc5cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff137fff640 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff137fff640 1 -- 172.21.15.103:0/2923554244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0fc5a6c10 con 0x55c0fc5a40d0 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff137fff640 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5cb5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1280058c0 tx=0x7ff128005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca97833b17f6805a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff134ff9640 1 -- 172.21.15.103:0/2923554244 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128028070 con 0x55c0fc5a40d0 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff134ff9640 1 -- 172.21.15.103:0/2923554244 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff128029040 con 0x55c0fc5a40d0 2024-08-19T01:36:34.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.970+0000 7ff134ff9640 1 -- 172.21.15.103:0/2923554244 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12801f6a0 con 0x55c0fc5a40d0 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 -- 172.21.15.103:0/2923554244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 msgr2=0x55c0fc5cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5cb5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1280058c0 tx=0x7ff128005850 comp rx=0 tx=0).stop 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 -- 172.21.15.103:0/2923554244 shutdown_connections 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5b3800 0x55c0fc5ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 --2- 172.21.15.103:0/2923554244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5cb5f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 -- 172.21.15.103:0/2923554244 >> 172.21.15.103:0/2923554244 conn(0x55c0fc5a07a0 msgr2=0x55c0fc5b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 -- 172.21.15.103:0/2923554244 shutdown_connections 2024-08-19T01:36:34.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 -- 172.21.15.103:0/2923554244 wait complete. 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.971+0000 7ff147907740 1 -- start start 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 1 Processor -- start 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff12000f7f0 con 0x55c0fc5b3800 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff12000f120 con 0x55c0fc5a40d0 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff12000c360 con 0x55c0fc5a49f0 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff147907740 10 client.0 _ll_register_callbacks cb 0x7ff130009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34984/0 (socket says 172.21.15.103:34984) 2024-08-19T01:36:34.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 -- 172.21.15.103:0/1726360993 learned_addr learned my addr 172.21.15.103:0/1726360993 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff13effd640 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 -- 172.21.15.103:0/1726360993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 msgr2=0x55c0fc5b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 -- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5a40d0 msgr2=0x55c0fc5bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5bc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.972+0000 7ff137fff640 1 -- 172.21.15.103:0/1726360993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0fc62d5c0 con 0x55c0fc5b3800 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff137fff640 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b7be0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff1280047e0 tx=0x7ff128005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9515b1c93241e537 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff1267fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff1267fc640 1 -- 172.21.15.103:0/1726360993 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128028050 con 0x55c0fc5b3800 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff1267fc640 1 -- 172.21.15.103:0/1726360993 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff128031450 con 0x55c0fc5b3800 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff128029040 con 0x55c0fc5b3800 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff1267fc640 1 -- 172.21.15.103:0/1726360993 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff128031650 con 0x55c0fc5b3800 2024-08-19T01:36:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 10 client.15654 fetch_fsmap learned FSMap version 27 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0fc62e730 con 0x55c0fc5b3800 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff1267fc640 1 -- 172.21.15.103:0/1726360993 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff128029040 con 0x55c0fc5b3800 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff1267fc640 1 -- 172.21.15.103:0/1726360993 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff12802e040 con 0x55c0fc5b3800 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 10 client.15654 fetch_fsmap finished waiting for FSMap version 27 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 20 client.15654 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc62eb50 0x55c0fc630f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 4 client.15654 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.973+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c0fc511400 con 0x55c0fc62eb50 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff13effd640 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc62eb50 0x55c0fc630f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff13effd640 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc62eb50 0x55c0fc630f40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ba4a9b0409ad9 server_cookie=3e0b0584130328be in_seq=0 out_seq=0 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff1267fc640 10 client.15654 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:34.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff1267fc640 1 -- 172.21.15.103:0/1726360993 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c0fc511400 con 0x55c0fc62eb50 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff1267fc640 10 client.15654 handle_command_reply: tid=0 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 client.15654 shutdown 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5b3800 msgr2=0x55c0fc5b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b7be0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff1280047e0 tx=0x7ff128005f90 comp rx=0 tx=0).stop 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 shutdown_connections 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c0fc62eb50 0x55c0fc630f40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0fc5b3800 0x55c0fc5b7be0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c0fc5a49f0 0x55c0fc5b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 --2- 172.21.15.103:0/1726360993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c0fc5a40d0 0x55c0fc5bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 >> 172.21.15.103:0/1726360993 conn(0x55c0fc5a07a0 msgr2=0x55c0fc59f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.974+0000 7ff1267fc640 0 client.15654 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 shutdown_connections 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff147907740 1 -- 172.21.15.103:0/1726360993 wait complete. 2024-08-19T01:36:34.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff147907740 20 client.15654 trim_cache size 0 max 16384 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: "34aed164-435f-48fd-a97e-65d939a99c2c": { 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "34aed164-435f-48fd-a97e-65d939a99c2c", 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:36:34.982 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff12c0a2b80 msgr2=0x7ff12c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff12c0a2b80 0x7ff12c0a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff138009df0 tx=0x7ff1380871a0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1401067c0 msgr2=0x7ff14019c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1401067c0 0x7ff14019c970 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff138033040 tx=0x7ff138009360 comp rx=0 tx=0).stop 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 shutdown_connections 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff12c0a2b80 0x7ff12c0a5040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1401067c0 0x7ff14019c970 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff140101160 0x7ff1401985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 --2- 172.21.15.103:0/2192258371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1401007c0 0x7ff1401980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:34.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.975+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 >> 172.21.15.103:0/2192258371 conn(0x7ff1400fc500 msgr2=0x7ff14006a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:34.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.976+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 shutdown_connections 2024-08-19T01:36:34.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:36:34.976+0000 7ff145509640 1 -- 172.21.15.103:0/2192258371 wait complete. 2024-08-19T01:36:35.037 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stat64... yes 2024-08-19T01:36:35.120 INFO:tasks.workunit.client.0.smithi103.stdout:checking for fseeko64... yes 2024-08-19T01:36:35.206 INFO:tasks.workunit.client.0.smithi103.stdout:checking for lrand48_r... yes 2024-08-19T01:36:35.301 INFO:tasks.workunit.client.0.smithi103.stdout:checking for srand48_r... yes 2024-08-19T01:36:35.329 INFO:tasks.workunit.client.0.smithi103.stdout:configure: creating ./config.status 2024-08-19T01:36:35.482 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating Makefile 2024-08-19T01:36:35.501 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating config.h 2024-08-19T01:36:35.527 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: executing depfiles commands 2024-08-19T01:36:35.596 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: executing default commands 2024-08-19T01:36:35.661 INFO:tasks.workunit.client.0.smithi103.stderr:+ make 2024-08-19T01:36:35.689 INFO:tasks.workunit.client.0.smithi103.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-08-19T01:36:35.696 INFO:tasks.workunit.client.0.smithi103.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-08-19T01:36:35.697 INFO:tasks.workunit.client.0.smithi103.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-08-19T01:36:35.697 INFO:tasks.workunit.client.0.smithi103.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-08-19T01:36:35.697 INFO:tasks.workunit.client.0.smithi103.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-08-19T01:36:35.697 INFO:tasks.workunit.client.0.smithi103.stderr: any GNU archive site. 2024-08-19T01:36:35.701 INFO:tasks.workunit.client.0.smithi103.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-08-19T01:36:35.706 INFO:tasks.workunit.client.0.smithi103.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-08-19T01:36:35.707 INFO:tasks.workunit.client.0.smithi103.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-08-19T01:36:35.707 INFO:tasks.workunit.client.0.smithi103.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-08-19T01:36:35.707 INFO:tasks.workunit.client.0.smithi103.stderr: You might want to install the `Automake' and `Perl' packages. 2024-08-19T01:36:35.707 INFO:tasks.workunit.client.0.smithi103.stderr: Grab them from any GNU archive site. 2024-08-19T01:36:35.880 INFO:tasks.workunit.client.0.smithi103.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-08-19T01:36:35.971 INFO:tasks.cephfs.filesystem:scrub status for tag:34aed164-435f-48fd-a97e-65d939a99c2c - {'path': '/', 'tag': '34aed164-435f-48fd-a97e-65d939a99c2c', 'options': 'recursive,force'} 2024-08-19T01:36:36.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:35 smithi146 ceph-mon[22098]: pgmap v363: 97 pgs: 97 active+clean; 3.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 235 KiB/s wr, 58 op/s 2024-08-19T01:36:36.236 INFO:tasks.workunit.client.0.smithi103.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-08-19T01:36:36.236 INFO:tasks.workunit.client.0.smithi103.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-08-19T01:36:36.236 INFO:tasks.workunit.client.0.smithi103.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-08-19T01:36:36.236 INFO:tasks.workunit.client.0.smithi103.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-08-19T01:36:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:35 smithi103 ceph-mon[18436]: pgmap v363: 97 pgs: 97 active+clean; 3.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 235 KiB/s wr, 58 op/s 2024-08-19T01:36:36.377 INFO:tasks.workunit.client.0.smithi103.stdout:/bin/sh ./config.status --recheck 2024-08-19T01:36:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:35 smithi190 ceph-mon[22032]: pgmap v363: 97 pgs: 97 active+clean; 3.2 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 235 KiB/s wr, 58 op/s 2024-08-19T01:36:36.436 INFO:tasks.workunit.client.0.smithi103.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-08-19T01:36:36.578 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-19T01:36:37.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:36 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:36:37.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:36 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:36:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:36 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:36:37.587 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether build environment is sane... yes 2024-08-19T01:36:37.596 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-08-19T01:36:37.596 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gawk... gawk 2024-08-19T01:36:37.625 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether make sets $(MAKE)... yes 2024-08-19T01:36:37.668 INFO:tasks.workunit.client.0.smithi103.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-19T01:36:37.668 INFO:tasks.workunit.client.0.smithi103.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-19T01:36:37.689 INFO:tasks.workunit.client.0.smithi103.stdout:checking for style of include used by make... GNU 2024-08-19T01:36:37.694 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc... gcc 2024-08-19T01:36:37.810 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether the C compiler works... yes 2024-08-19T01:36:37.810 INFO:tasks.workunit.client.0.smithi103.stdout:checking for C compiler default output file name... a.out 2024-08-19T01:36:37.861 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of executables... 2024-08-19T01:36:37.914 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are cross compiling... no 2024-08-19T01:36:37.961 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of object files... o 2024-08-19T01:36:37.992 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are using the GNU C compiler... yes 2024-08-19T01:36:38.024 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc accepts -g... yes 2024-08-19T01:36:38.076 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to accept ISO C89... none needed 2024-08-19T01:36:38.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:37 smithi146 ceph-mon[22098]: pgmap v364: 97 pgs: 97 active+clean; 3.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 265 KiB/s wr, 107 op/s 2024-08-19T01:36:38.204 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:37 smithi103 ceph-mon[18436]: pgmap v364: 97 pgs: 97 active+clean; 3.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 265 KiB/s wr, 107 op/s 2024-08-19T01:36:38.233 INFO:tasks.workunit.client.0.smithi103.stdout:checking dependency style of gcc... gcc3 2024-08-19T01:36:38.296 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to run the C preprocessor... gcc -E 2024-08-19T01:36:38.361 INFO:tasks.workunit.client.0.smithi103.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-08-19T01:36:38.363 INFO:tasks.workunit.client.0.smithi103.stdout:checking for egrep... /usr/bin/grep -E 2024-08-19T01:36:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:37 smithi190 ceph-mon[22032]: pgmap v364: 97 pgs: 97 active+clean; 3.2 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 265 KiB/s wr, 107 op/s 2024-08-19T01:36:38.531 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ANSI C header files... yes 2024-08-19T01:36:38.582 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/types.h... yes 2024-08-19T01:36:38.634 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/stat.h... yes 2024-08-19T01:36:38.684 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdlib.h... yes 2024-08-19T01:36:38.741 INFO:tasks.workunit.client.0.smithi103.stdout:checking for string.h... yes 2024-08-19T01:36:38.798 INFO:tasks.workunit.client.0.smithi103.stdout:checking for memory.h... yes 2024-08-19T01:36:38.852 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strings.h... yes 2024-08-19T01:36:38.905 INFO:tasks.workunit.client.0.smithi103.stdout:checking for inttypes.h... yes 2024-08-19T01:36:38.959 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdint.h... yes 2024-08-19T01:36:39.013 INFO:tasks.workunit.client.0.smithi103.stdout:checking for unistd.h... yes 2024-08-19T01:36:39.071 INFO:tasks.workunit.client.0.smithi103.stdout:checking minix/config.h usability... no 2024-08-19T01:36:39.109 INFO:tasks.workunit.client.0.smithi103.stdout:checking minix/config.h presence... no 2024-08-19T01:36:39.109 INFO:tasks.workunit.client.0.smithi103.stdout:checking for minix/config.h... no 2024-08-19T01:36:39.152 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-08-19T01:36:39.163 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc... (cached) gcc 2024-08-19T01:36:39.215 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-08-19T01:36:39.216 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc accepts -g... (cached) yes 2024-08-19T01:36:39.217 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-08-19T01:36:39.217 INFO:tasks.workunit.client.0.smithi103.stdout:checking dependency style of gcc... (cached) gcc3 2024-08-19T01:36:39.278 INFO:tasks.workunit.client.0.smithi103.stdout:checking for main in -lm... yes 2024-08-19T01:36:39.345 INFO:tasks.workunit.client.0.smithi103.stdout:checking for main in -lpthread... yes 2024-08-19T01:36:39.350 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ANSI C header files... (cached) yes 2024-08-19T01:36:39.392 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-08-19T01:36:39.446 INFO:tasks.workunit.client.0.smithi103.stdout:checking pthread.h usability... yes 2024-08-19T01:36:39.471 INFO:tasks.workunit.client.0.smithi103.stdout:checking pthread.h presence... yes 2024-08-19T01:36:39.471 INFO:tasks.workunit.client.0.smithi103.stdout:checking for pthread.h... yes 2024-08-19T01:36:39.530 INFO:tasks.workunit.client.0.smithi103.stdout:checking fcntl.h usability... yes 2024-08-19T01:36:39.552 INFO:tasks.workunit.client.0.smithi103.stdout:checking fcntl.h presence... yes 2024-08-19T01:36:39.552 INFO:tasks.workunit.client.0.smithi103.stdout:checking for fcntl.h... yes 2024-08-19T01:36:39.612 INFO:tasks.workunit.client.0.smithi103.stdout:checking limits.h usability... yes 2024-08-19T01:36:39.635 INFO:tasks.workunit.client.0.smithi103.stdout:checking limits.h presence... yes 2024-08-19T01:36:39.635 INFO:tasks.workunit.client.0.smithi103.stdout:checking for limits.h... yes 2024-08-19T01:36:39.654 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdint.h... (cached) yes 2024-08-19T01:36:39.705 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h usability... yes 2024-08-19T01:36:39.727 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h presence... yes 2024-08-19T01:36:39.727 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/time.h... yes 2024-08-19T01:36:39.747 INFO:tasks.workunit.client.0.smithi103.stdout:checking for unistd.h... (cached) yes 2024-08-19T01:36:39.798 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/vfs.h usability... yes 2024-08-19T01:36:39.820 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/vfs.h presence... yes 2024-08-19T01:36:39.820 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/vfs.h... yes 2024-08-19T01:36:39.886 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/limits.h usability... no 2024-08-19T01:36:39.924 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/limits.h presence... no 2024-08-19T01:36:39.924 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/limits.h... no 2024-08-19T01:36:39.958 INFO:tasks.workunit.client.0.smithi103.stdout:checking for an ANSI C-conforming const... yes 2024-08-19T01:36:39.994 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-19T01:36:40.078 INFO:tasks.workunit.client.0.smithi103.stdout:checking for system... yes 2024-08-19T01:36:40.153 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gettimeofday... yes 2024-08-19T01:36:40.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:39 smithi146 ceph-mon[22098]: pgmap v365: 97 pgs: 97 active+clean; 5.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 529 KiB/s wr, 180 op/s 2024-08-19T01:36:40.229 INFO:tasks.workunit.client.0.smithi103.stdout:checking for mkdir... yes 2024-08-19T01:36:40.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:39 smithi103 ceph-mon[18436]: pgmap v365: 97 pgs: 97 active+clean; 5.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 529 KiB/s wr, 180 op/s 2024-08-19T01:36:40.312 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strerror... yes 2024-08-19T01:36:40.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:39 smithi190 ceph-mon[22032]: pgmap v365: 97 pgs: 97 active+clean; 5.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 529 KiB/s wr, 180 op/s 2024-08-19T01:36:40.393 INFO:tasks.workunit.client.0.smithi103.stdout:checking for open64... yes 2024-08-19T01:36:40.470 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stat64... yes 2024-08-19T01:36:40.545 INFO:tasks.workunit.client.0.smithi103.stdout:checking for fseeko64... yes 2024-08-19T01:36:40.623 INFO:tasks.workunit.client.0.smithi103.stdout:checking for lrand48_r... yes 2024-08-19T01:36:40.701 INFO:tasks.workunit.client.0.smithi103.stdout:checking for srand48_r... yes 2024-08-19T01:36:40.727 INFO:tasks.workunit.client.0.smithi103.stdout:configure: creating ./config.status 2024-08-19T01:36:40.834 INFO:tasks.workunit.client.0.smithi103.stdout: /bin/sh ./config.status 2024-08-19T01:36:40.898 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating Makefile 2024-08-19T01:36:40.915 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating config.h 2024-08-19T01:36:40.946 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: executing depfiles commands 2024-08-19T01:36:40.964 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: executing default commands 2024-08-19T01:36:41.011 INFO:tasks.workunit.client.0.smithi103.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-08-19T01:36:41.174 INFO:tasks.workunit.client.0.smithi103.stdout:rm -f stamp-h1 2024-08-19T01:36:41.176 INFO:tasks.workunit.client.0.smithi103.stdout:touch config.h.in 2024-08-19T01:36:41.178 INFO:tasks.workunit.client.0.smithi103.stdout:cd . && /bin/sh ./config.status config.h 2024-08-19T01:36:41.234 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating config.h 2024-08-19T01:36:41.269 INFO:tasks.workunit.client.0.smithi103.stdout:make all-am 2024-08-19T01:36:41.305 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-08-19T01:36:41.305 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-08-19T01:36:41.509 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-08-19T01:36:41.519 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-08-19T01:36:41.624 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-08-19T01:36:41.632 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-08-19T01:36:41.749 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-08-19T01:36:41.756 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-08-19T01:36:41.868 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-08-19T01:36:41.876 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-08-19T01:36:42.023 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-08-19T01:36:42.032 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-08-19T01:36:42.104 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-08-19T01:36:42.113 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-08-19T01:36:42.168 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-08-19T01:36:42.176 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-08-19T01:36:42.182 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:41 smithi190 ceph-mon[22032]: pgmap v366: 97 pgs: 97 active+clean; 6.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 572 KiB/s wr, 198 op/s 2024-08-19T01:36:42.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:41 smithi146 ceph-mon[22098]: pgmap v366: 97 pgs: 97 active+clean; 6.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 572 KiB/s wr, 198 op/s 2024-08-19T01:36:42.240 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-08-19T01:36:42.247 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-08-19T01:36:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:41 smithi103 ceph-mon[18436]: pgmap v366: 97 pgs: 97 active+clean; 6.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 572 KiB/s wr, 198 op/s 2024-08-19T01:36:42.479 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-08-19T01:36:42.487 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-08-19T01:36:42.633 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-08-19T01:36:42.643 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-08-19T01:36:42.818 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-08-19T01:36:42.827 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-08-19T01:36:42.918 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-08-19T01:36:42.927 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-08-19T01:36:43.020 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-08-19T01:36:43.028 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-08-19T01:36:43.160 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-08-19T01:36:43.168 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-08-19T01:36:43.211 INFO:tasks.workunit.client.0.smithi103.stderr:In file included from parser.h:22, 2024-08-19T01:36:43.211 INFO:tasks.workunit.client.0.smithi103.stderr: from parser.c:25: 2024-08-19T01:36:43.211 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c: In function ‘init_filesys’: 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-19T01:36:43.213 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.214 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.215 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.216 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c: In function ‘init_tg_stats’: 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.217 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: 971 | list_for_each_entry(tmp_list, 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.218 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: 971 | list_for_each_entry(tmp_list, 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.219 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: 980 | list_for_each_entry(tmp_list, 2024-08-19T01:36:43.220 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: | ^ 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~ 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: 36 | container_of(ptr, type, member) 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~ 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-19T01:36:43.221 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~ 2024-08-19T01:36:43.222 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-19T01:36:43.222 INFO:tasks.workunit.client.0.smithi103.stderr: 980 | list_for_each_entry(tmp_list, 2024-08-19T01:36:43.222 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.259 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c: In function ‘get_optstr’: 2024-08-19T01:36:43.259 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-08-19T01:36:43.259 INFO:tasks.workunit.client.0.smithi103.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-08-19T01:36:43.259 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.259 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:200:14: note: source object declared here 2024-08-19T01:36:43.259 INFO:tasks.workunit.client.0.smithi103.stderr: 200 | char temp[BUFSIZE]; 2024-08-19T01:36:43.259 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~ 2024-08-19T01:36:43.266 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c: In function ‘size64_convert’: 2024-08-19T01:36:43.266 INFO:tasks.workunit.client.0.smithi103.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-08-19T01:36:43.266 INFO:tasks.workunit.client.0.smithi103.stderr: 119 | memcpy(unit, "\0", 3); 2024-08-19T01:36:43.266 INFO:tasks.workunit.client.0.smithi103.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-08-19T01:36:43.560 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-08-19T01:36:43.568 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-08-19T01:36:43.650 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-08-19T01:36:43.659 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-08-19T01:36:43.765 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-08-19T01:36:43.773 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-08-19T01:36:43.815 INFO:tasks.workunit.client.0.smithi103.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-08-19T01:36:43.824 INFO:tasks.workunit.client.0.smithi103.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-08-19T01:36:43.922 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-08-19T01:36:43.923 INFO:tasks.workunit.client.0.smithi103.stderr:+ cd .. 2024-08-19T01:36:43.924 INFO:tasks.workunit.client.0.smithi103.stderr:+ mkdir tmp 2024-08-19T01:36:43.926 INFO:tasks.workunit.client.0.smithi103.stderr:+ cd tmp 2024-08-19T01:36:43.927 INFO:tasks.workunit.client.0.smithi103.stderr:+ for f in $mydir/*.ffsb 2024-08-19T01:36:43.928 INFO:tasks.workunit.client.0.smithi103.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-08-19T01:36:43.928 INFO:tasks.workunit.client.0.smithi103.stdout:FFSB version 6.0-RC2 started 2024-08-19T01:36:43.928 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout:benchmark time = 300 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout:ThreadGroup 0 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout:================ 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: num_threads = 32 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: read_random = off 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: read_size = 0 (0B) 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: read_blocksize = 0 (0B) 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: read_skip = off 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: read_skipsize = 0 (0B) 2024-08-19T01:36:43.932 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: write_random = on 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: write_size = 5242880 (5MB) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: fsync_file = 0 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: write_blocksize = 4096 (4KB) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: wait time = 0 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: op weights 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: read = 0 (0.00%) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: readall = 0 (0.00%) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: write = 1 (100.00%) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: create = 0 (0.00%) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: append = 0 (0.00%) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: delete = 0 (0.00%) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: metaop = 0 (0.00%) 2024-08-19T01:36:43.933 INFO:tasks.workunit.client.0.smithi103.stdout: createdir = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: stat = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: writeall = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: writeall_fsync = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: open_close = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: write_fsync = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: create_fsync = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: append_fsync = 0 (0.00%) 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout:FileSystem . 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout:========== 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: num_dirs = 0 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: starting files = 128 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.934 INFO:tasks.workunit.client.0.smithi103.stdout: min file size = 104857600 (100MB) 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: max file size = 104857600 (100MB) 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: directio = off 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: alignedio = on 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: bufferedio = off 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: aging is off 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: current utilization = 0.00% 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout:checking existing fs: . 2024-08-19T01:36:43.935 INFO:tasks.workunit.client.0.smithi103.stdout:recreating new fileset 2024-08-19T01:36:43.936 INFO:tasks.workunit.client.0.smithi103.stderr:opendir: No such file or directory 2024-08-19T01:36:44.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:43 smithi146 ceph-mon[22098]: pgmap v367: 97 pgs: 97 active+clean; 8.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 802 KiB/s wr, 237 op/s 2024-08-19T01:36:44.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:43 smithi103 ceph-mon[18436]: pgmap v367: 97 pgs: 97 active+clean; 8.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 802 KiB/s wr, 237 op/s 2024-08-19T01:36:44.362 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:43 smithi190 ceph-mon[22032]: pgmap v367: 97 pgs: 97 active+clean; 8.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 802 KiB/s wr, 237 op/s 2024-08-19T01:36:46.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:45 smithi146 ceph-mon[22098]: pgmap v368: 97 pgs: 97 active+clean; 8.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 659 KiB/s wr, 230 op/s 2024-08-19T01:36:46.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:45 smithi103 ceph-mon[18436]: pgmap v368: 97 pgs: 97 active+clean; 8.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 659 KiB/s wr, 230 op/s 2024-08-19T01:36:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:45 smithi190 ceph-mon[22032]: pgmap v368: 97 pgs: 97 active+clean; 8.7 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 659 KiB/s wr, 230 op/s 2024-08-19T01:36:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.946+0000 7f25e726e640 1 -- 172.21.15.103:0/3830257560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0101890 msgr2=0x7f25e0101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.946+0000 7f25e726e640 1 --2- 172.21.15.103:0/3830257560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0101890 0x7f25e0101c90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009900 tx=0x7f25dc02a410 comp rx=0 tx=0).stop 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.946+0000 7f25e726e640 1 -- 172.21.15.103:0/3830257560 shutdown_connections 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.946+0000 7f25e726e640 1 --2- 172.21.15.103:0/3830257560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e0103440 0x7f25e0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.946+0000 7f25e726e640 1 --2- 172.21.15.103:0/3830257560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 0x7f25e0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.946+0000 7f25e726e640 1 --2- 172.21.15.103:0/3830257560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0101890 0x7f25e0101c90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.946+0000 7f25e726e640 1 -- 172.21.15.103:0/3830257560 >> 172.21.15.103:0/3830257560 conn(0x7f25e00fd010 msgr2=0x7f25e00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.947+0000 7f25e726e640 1 -- 172.21.15.103:0/3830257560 shutdown_connections 2024-08-19T01:36:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.947+0000 7f25e726e640 1 -- 172.21.15.103:0/3830257560 wait complete. 2024-08-19T01:36:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.947+0000 7f25e726e640 1 Processor -- start 2024-08-19T01:36:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.947+0000 7f25e726e640 1 -- start start 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e726e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e0101890 0x7f25e0199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e726e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 0x7f25e019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e726e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0103440 0x7f25e01a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e726e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25e010d3c0 con 0x7f25e0103440 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e726e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25e010d240 con 0x7f25e0101890 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e726e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f25e010d540 con 0x7f25e0102a80 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e500a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e0101890 0x7f25e0199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e4809640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 0x7f25e019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e500a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e0101890 0x7f25e0199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:47072/0 (socket says 172.21.15.103:47072) 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e500a640 1 -- 172.21.15.103:0/911659548 learned_addr learned my addr 172.21.15.103:0/911659548 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e580b640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0103440 0x7f25e01a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e4809640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 0x7f25e019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:45394/0 (socket says 172.21.15.103:45394) 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e580b640 1 -- 172.21.15.103:0/911659548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 msgr2=0x7f25e019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e580b640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 0x7f25e019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e580b640 1 -- 172.21.15.103:0/911659548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e0101890 msgr2=0x7f25e0199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e580b640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e0101890 0x7f25e0199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e580b640 1 -- 172.21.15.103:0/911659548 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e01a18f0 con 0x7f25e0103440 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.948+0000 7f25e4809640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 0x7f25e019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.949+0000 7f25e580b640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0103440 0x7f25e01a1250 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ec50 tx=0x7f25d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.949+0000 7f25d67fc640 1 -- 172.21.15.103:0/911659548 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8003b70 con 0x7f25e0103440 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.949+0000 7f25d67fc640 1 -- 172.21.15.103:0/911659548 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d8003d10 con 0x7f25e0103440 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.949+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e01a1be0 con 0x7f25e0103440 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.949+0000 7f25d67fc640 1 -- 172.21.15.103:0/911659548 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8010760 con 0x7f25e0103440 2024-08-19T01:36:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.949+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e01a2180 con 0x7f25e0103440 2024-08-19T01:36:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.952+0000 7f25d67fc640 1 -- 172.21.15.103:0/911659548 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f25d8004000 con 0x7f25e0103440 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.952+0000 7f25ea02c740 1 Processor -- start 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.952+0000 7f25ea02c740 1 -- start start 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20ef680 0x5560c20efa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25ea02c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560c20529d0 con 0x5560c20effc0 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25ea02c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5560c1ffa820 con 0x5560c20f5050 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25ea02c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5560c204e990 con 0x5560c20ef680 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25d67fc640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25cc0a2cb0 0x7f25cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e500a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e4809640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e500a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43474/0 (socket says 172.21.15.103:43474) 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e500a640 1 -- 172.21.15.103:0/720299804 learned_addr learned my addr 172.21.15.103:0/720299804 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e4809640 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20ef680 0x5560c20efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25d67fc640 1 -- 172.21.15.103:0/911659548 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f25d80cf530 con 0x7f25e0103440 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e580b640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25cc0a2cb0 0x7f25cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e4809640 1 -- 172.21.15.103:0/720299804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20ef680 msgr2=0x5560c20efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e4809640 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20ef680 0x5560c20efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e4809640 1 -- 172.21.15.103:0/720299804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 msgr2=0x5560c20f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e4809640 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e4809640 1 -- 172.21.15.103:0/720299804 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c1d66210 con 0x5560c20f5050 2024-08-19T01:36:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.953+0000 7f25e500a640 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.954+0000 7f25e4809640 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f7510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25e019b1b0 tx=0x7f25d0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f0ddfe913989e19 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.954+0000 7f25e580b640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25cc0a2cb0 0x7f25cc0a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f25d809bd50 tx=0x7f25d800ce70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.954+0000 7f25cb7fe640 1 -- 172.21.15.103:0/720299804 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0011050 con 0x5560c20f5050 2024-08-19T01:36:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.954+0000 7f25cb7fe640 1 -- 172.21.15.103:0/720299804 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d000fd30 con 0x5560c20f5050 2024-08-19T01:36:46.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.954+0000 7f25cb7fe640 1 -- 172.21.15.103:0/720299804 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00047a0 con 0x5560c20f5050 2024-08-19T01:36:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 -- 172.21.15.103:0/720299804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 msgr2=0x5560c20f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f7510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25e019b1b0 tx=0x7f25d0005850 comp rx=0 tx=0).stop 2024-08-19T01:36:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 -- 172.21.15.103:0/720299804 shutdown_connections 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f7510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 --2- 172.21.15.103:0/720299804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20ef680 0x5560c20efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 -- 172.21.15.103:0/720299804 >> 172.21.15.103:0/720299804 conn(0x5560c20eacf0 msgr2=0x5560c20eb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 -- 172.21.15.103:0/720299804 shutdown_connections 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 -- 172.21.15.103:0/720299804 wait complete. 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 -- start start 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 Processor -- start 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.955+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20f5050 0x5560c2100e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25ea02c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560c20f8020 con 0x5560c20effc0 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25ea02c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5560c200e520 con 0x5560c20ef680 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25ea02c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5560c20f81a0 con 0x5560c20f5050 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25ea02c740 10 client.0 _ll_register_callbacks cb 0x5560c1d65a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e500a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:47084/0 (socket says 172.21.15.103:47084) 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 -- 172.21.15.103:0/3214798959 learned_addr learned my addr 172.21.15.103:0/3214798959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20f5050 0x5560c2100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 -- 172.21.15.103:0/3214798959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20f5050 msgr2=0x5560c2100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20f5050 0x5560c2100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 -- 172.21.15.103:0/3214798959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 msgr2=0x5560c20f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 -- 172.21.15.103:0/3214798959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c1e32620 con 0x5560c20ef680 2024-08-19T01:36:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e500a640 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25e4809640 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20f98f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25d00058c0 tx=0x7f25d00118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6361df3d4f886cd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.956+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3214798959 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0011030 con 0x5560c20ef680 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3214798959 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f25d00111d0 con 0x5560c20ef680 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d000fd30 con 0x5560c20ef680 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3214798959 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f25d00046c0 con 0x5560c20ef680 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 10 client.25211 fetch_fsmap learned FSMap version 28 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560c2103c40 con 0x5560c20ef680 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3214798959 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d000fd30 con 0x5560c20ef680 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3214798959 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f25d001d030 con 0x5560c20ef680 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 10 client.25211 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 10 client.25211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 20 client.25211 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c2104fe0 0x5560c21073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 4 client.25211 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.957+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c2060100 con 0x5560c2104fe0 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.958+0000 7f25e500a640 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c2104fe0 0x5560c21073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.958+0000 7f25e500a640 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c2104fe0 0x5560c21073d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec97589fd61d4312 server_cookie=b05d2bd2a85e2471 in_seq=0 out_seq=0 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.958+0000 7f25c8ff9640 10 client.25211 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3214798959 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560c2060100 con 0x5560c2104fe0 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25c8ff9640 10 client.25211 handle_command_reply: tid=0 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 client.25211 shutdown 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 msgr2=0x5560c20f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20f98f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25d00058c0 tx=0x7f25d00118f0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 shutdown_connections 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c2104fe0 0x5560c21073d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20f5050 0x5560c2100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20effc0 0x5560c20f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25c8ff9640 0 client.25211 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3214798959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20f98f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 >> 172.21.15.103:0/3214798959 conn(0x5560c20eacf0 msgr2=0x5560c20ec500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 shutdown_connections 2024-08-19T01:36:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 1 -- 172.21.15.103:0/3214798959 wait complete. 2024-08-19T01:36:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.959+0000 7f25ea02c740 20 client.25211 trim_cache size 0 max 16384 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 Processor -- start 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 -- start start 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20ef680 0x5560c20f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c2100e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01aae10 con 0x5560c20ef680 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01ac980 con 0x5560c20f5050 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25ea02c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01ab4e0 con 0x5560c20effc0 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e4809640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20ef680 0x5560c20f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e4809640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20ef680 0x5560c20f96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43498/0 (socket says 172.21.15.103:43498) 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e4809640 1 -- 172.21.15.103:0/1971852040 learned_addr learned my addr 172.21.15.103:0/1971852040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e500a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c2100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e4809640 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e500a640 1 -- 172.21.15.103:0/1971852040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 msgr2=0x5560c20f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e500a640 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.975+0000 7f25e500a640 1 -- 172.21.15.103:0/1971852040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20ef680 msgr2=0x5560c20f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.976+0000 7f25e500a640 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20ef680 0x5560c20f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.976+0000 7f25e500a640 1 -- 172.21.15.103:0/1971852040 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c2107740 con 0x5560c20effc0 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.976+0000 7f25e4809640 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20ef680 0x5560c20f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.976+0000 7f25e500a640 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c2100e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5560c20fadd0 tx=0x7f25dc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1968f72b8ed310ca server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.976+0000 7f25caffd640 1 -- 172.21.15.103:0/1971852040 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc049020 con 0x5560c20effc0 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.976+0000 7f25caffd640 1 -- 172.21.15.103:0/1971852040 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25dc004240 con 0x5560c20effc0 2024-08-19T01:36:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.976+0000 7f25caffd640 1 -- 172.21.15.103:0/1971852040 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc035040 con 0x5560c20effc0 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 -- 172.21.15.103:0/1971852040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 msgr2=0x5560c2100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c2100e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5560c20fadd0 tx=0x7f25dc000bc0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 -- 172.21.15.103:0/1971852040 shutdown_connections 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20f5050 0x5560c20f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c2100e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 --2- 172.21.15.103:0/1971852040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20ef680 0x5560c20f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 -- 172.21.15.103:0/1971852040 >> 172.21.15.103:0/1971852040 conn(0x5560c20eacf0 msgr2=0x5560c20f48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 -- 172.21.15.103:0/1971852040 shutdown_connections 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 -- 172.21.15.103:0/1971852040 wait complete. 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 -- start start 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.977+0000 7f25ea02c740 1 Processor -- start 2024-08-19T01:36:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20fda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c20fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25ea02c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20f5050 0x5560c216dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25ea02c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01ac280 con 0x5560c20f5050 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25ea02c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01abbb0 con 0x5560c20ef680 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25ea02c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f25e01a8df0 con 0x5560c20effc0 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25ea02c740 10 client.0 _ll_register_callbacks cb 0x7f25d0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e4809640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20f5050 0x5560c216dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c20fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e4809640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20f5050 0x5560c216dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43502/0 (socket says 172.21.15.103:43502) 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e4809640 1 -- 172.21.15.103:0/3733008585 learned_addr learned my addr 172.21.15.103:0/3733008585 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c20fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:45442/0 (socket says 172.21.15.103:45442) 2024-08-19T01:36:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e4809640 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 -- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 msgr2=0x5560c20fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 -- 172.21.15.103:0/3733008585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20f5050 msgr2=0x5560c216dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20f5050 0x5560c216dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 -- 172.21.15.103:0/3733008585 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c2101090 con 0x5560c20effc0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25e500a640 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c20fdfa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5560c20f91f0 tx=0x7f25dc005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a71c77eddfa047d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.978+0000 7f25c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3733008585 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc049030 con 0x5560c20effc0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25dc004240 con 0x5560c20effc0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3733008585 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f25dc003d40 con 0x5560c20effc0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3733008585 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f25dc035040 con 0x5560c20effc0 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 10 client.25195 fetch_fsmap learned FSMap version 28 2024-08-19T01:36:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560c216f1e0 con 0x5560c20effc0 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3733008585 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25dc004240 con 0x5560c20effc0 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3733008585 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f25dc058850 con 0x5560c20effc0 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 10 client.25195 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 10 client.25195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 20 client.25195 populate_metadata read hostname 'smithi103' 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c216fbf0 0x5560c2172010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 4 client.25195 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560c205a450 con 0x5560c216fbf0 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.979+0000 7f25e4809640 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c216fbf0 0x5560c2172010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.980+0000 7f25c8ff9640 10 client.25195 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:46.980+0000 7f25e4809640 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c216fbf0 0x5560c2172010 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23f7e2663eba331 server_cookie=2406bb105babeae0 in_seq=0 out_seq=0 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25c8ff9640 1 -- 172.21.15.103:0/3733008585 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560c205a450 con 0x5560c216fbf0 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25c8ff9640 10 client.25195 handle_command_reply: tid=0 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 client.25195 shutdown 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 msgr2=0x5560c20fdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c20fdfa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5560c20f91f0 tx=0x7f25dc005db0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 shutdown_connections 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5560c216fbf0 0x5560c2172010 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560c20f5050 0x5560c216dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560c20effc0 0x5560c20fdfa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25c8ff9640 0 client.25195 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:36:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 --2- 172.21.15.103:0/3733008585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560c20ef680 0x5560c20fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.004+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 >> 172.21.15.103:0/3733008585 conn(0x5560c20eacf0 msgr2=0x5560c20f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 shutdown_connections 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25ea02c740 1 -- 172.21.15.103:0/3733008585 wait complete. 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25ea02c740 20 client.25195 trim_cache size 0 max 16384 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:36:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25cc0a2cb0 msgr2=0x7f25cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25cc0a2cb0 0x7f25cc0a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f25d809bd50 tx=0x7f25d800ce70 comp rx=0 tx=0).stop 2024-08-19T01:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0103440 msgr2=0x7f25e01a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0103440 0x7f25e01a1250 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ec50 tx=0x7f25d800c6a0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 shutdown_connections 2024-08-19T01:36:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f25cc0a2cb0 0x7f25cc0a5170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e0103440 0x7f25e01a1250 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e0102a80 0x7f25e019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 --2- 172.21.15.103:0/911659548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e0101890 0x7f25e0199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 >> 172.21.15.103:0/911659548 conn(0x7f25e00fd010 msgr2=0x7f25e0074150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.005+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 shutdown_connections 2024-08-19T01:36:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:36:47.006+0000 7f25e726e640 1 -- 172.21.15.103:0/911659548 wait complete. 2024-08-19T01:36:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:47 smithi146 ceph-mon[22098]: pgmap v369: 97 pgs: 97 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 971 KiB/s wr, 216 op/s 2024-08-19T01:36:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:47 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:36:48.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:47 smithi103 ceph-mon[18436]: pgmap v369: 97 pgs: 97 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 971 KiB/s wr, 216 op/s 2024-08-19T01:36:48.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:47 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:36:48.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:47 smithi190 ceph-mon[22032]: pgmap v369: 97 pgs: 97 active+clean; 13 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 971 KiB/s wr, 216 op/s 2024-08-19T01:36:48.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:47 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:36:50.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:49 smithi146 ceph-mon[22098]: pgmap v370: 97 pgs: 97 active+clean; 70 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 5.7 MiB/s wr, 185 op/s 2024-08-19T01:36:50.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:49 smithi103 ceph-mon[18436]: pgmap v370: 97 pgs: 97 active+clean; 70 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 5.7 MiB/s wr, 185 op/s 2024-08-19T01:36:50.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:49 smithi190 ceph-mon[22032]: pgmap v370: 97 pgs: 97 active+clean; 70 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 5.7 MiB/s wr, 185 op/s 2024-08-19T01:36:51.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:50 smithi103 ceph-mon[18436]: pgmap v371: 97 pgs: 97 active+clean; 134 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 11 MiB/s wr, 119 op/s 2024-08-19T01:36:51.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:50 smithi190 ceph-mon[22032]: pgmap v371: 97 pgs: 97 active+clean; 134 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 11 MiB/s wr, 119 op/s 2024-08-19T01:36:51.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:50 smithi146 ceph-mon[22098]: pgmap v371: 97 pgs: 97 active+clean; 134 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 11 MiB/s wr, 119 op/s 2024-08-19T01:36:54.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:53 smithi190 ceph-mon[22032]: pgmap v372: 97 pgs: 97 active+clean; 202 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 17 MiB/s wr, 108 op/s 2024-08-19T01:36:54.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:53 smithi146 ceph-mon[22098]: pgmap v372: 97 pgs: 97 active+clean; 202 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 17 MiB/s wr, 108 op/s 2024-08-19T01:36:54.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:53 smithi103 ceph-mon[18436]: pgmap v372: 97 pgs: 97 active+clean; 202 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 17 MiB/s wr, 108 op/s 2024-08-19T01:36:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:55 smithi146 ceph-mon[22098]: pgmap v373: 97 pgs: 97 active+clean; 246 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 20 MiB/s wr, 69 op/s 2024-08-19T01:36:56.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:55 smithi103 ceph-mon[18436]: pgmap v373: 97 pgs: 97 active+clean; 246 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 20 MiB/s wr, 69 op/s 2024-08-19T01:36:56.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:55 smithi190 ceph-mon[22032]: pgmap v373: 97 pgs: 97 active+clean; 246 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 20 MiB/s wr, 69 op/s 2024-08-19T01:36:58.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:57 smithi146 ceph-mon[22098]: pgmap v374: 97 pgs: 97 active+clean; 294 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 51 op/s 2024-08-19T01:36:58.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:57 smithi103 ceph-mon[18436]: pgmap v374: 97 pgs: 97 active+clean; 294 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 51 op/s 2024-08-19T01:36:58.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:57 smithi190 ceph-mon[22032]: pgmap v374: 97 pgs: 97 active+clean; 294 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 24 MiB/s wr, 51 op/s 2024-08-19T01:37:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 -- 172.21.15.103:0/1352439946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78101890 msgr2=0x7fea78101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1352439946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78101890 0x7fea78101c90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fea68009900 tx=0x7fea6802a410 comp rx=0 tx=0).stop 2024-08-19T01:37:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 -- 172.21.15.103:0/1352439946 shutdown_connections 2024-08-19T01:37:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1352439946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea78103440 0x7fea78109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1352439946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea78102a80 0x7fea78102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1352439946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78101890 0x7fea78101c90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 -- 172.21.15.103:0/1352439946 >> 172.21.15.103:0/1352439946 conn(0x7fea780fd010 msgr2=0x7fea780ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 -- 172.21.15.103:0/1352439946 shutdown_connections 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.101+0000 7fea7d28d640 1 -- 172.21.15.103:0/1352439946 wait complete. 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 Processor -- start 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 -- start start 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea78101890 0x7fea78197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78102a80 0x7fea78197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea78103440 0x7fea7819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea7810d3c0 con 0x7fea78102a80 2024-08-19T01:37:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fea7810d240 con 0x7fea78103440 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea7d28d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea7810d540 con 0x7fea78101890 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78102a80 0x7fea78197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea76ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea78101890 0x7fea78197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78102a80 0x7fea78197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58130/0 (socket says 172.21.15.103:58130) 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 -- 172.21.15.103:0/1220390976 learned_addr learned my addr 172.21.15.103:0/1220390976 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea777fe640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea78103440 0x7fea7819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 -- 172.21.15.103:0/1220390976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea78101890 msgr2=0x7fea78197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea78101890 0x7fea78197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 -- 172.21.15.103:0/1220390976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea78103440 msgr2=0x7fea7819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea78103440 0x7fea7819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.102+0000 7fea6ffff640 1 -- 172.21.15.103:0/1220390976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7819f670 con 0x7fea78102a80 2024-08-19T01:37:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.103+0000 7fea6ffff640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78102a80 0x7fea78197f90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fea6000d770 tx=0x7fea6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.103+0000 7fea74ff9640 1 -- 172.21.15.103:0/1220390976 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60017070 con 0x7fea78102a80 2024-08-19T01:37:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.103+0000 7fea74ff9640 1 -- 172.21.15.103:0/1220390976 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea600054d0 con 0x7fea78102a80 2024-08-19T01:37:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.103+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7819f960 con 0x7fea78102a80 2024-08-19T01:37:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.103+0000 7fea74ff9640 1 -- 172.21.15.103:0/1220390976 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60004cf0 con 0x7fea78102a80 2024-08-19T01:37:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.103+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7819feb0 con 0x7fea78102a80 2024-08-19T01:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.105+0000 7fea74ff9640 1 -- 172.21.15.103:0/1220390976 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fea60004e90 con 0x7fea78102a80 2024-08-19T01:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.105+0000 7fea80049740 1 Processor -- start 2024-08-19T01:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.105+0000 7fea80049740 1 -- start start 2024-08-19T01:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea80049740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006548a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea80049740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a00654da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea80049740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006550510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea80049740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a0064ab9d0 con 0x55a00654e050 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea80049740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a006453820 con 0x55a006548680 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea80049740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a0064a7990 con 0x55a006548fc0 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006550510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a00654da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a00654da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:60086/0 (socket says 172.21.15.103:60086) 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 -- 172.21.15.103:0/3150384391 learned_addr learned my addr 172.21.15.103:0/3150384391 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006550510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58142/0 (socket says 172.21.15.103:58142) 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea777fe640 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006548a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 -- 172.21.15.103:0/3150384391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 msgr2=0x55a006548a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006548a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 -- 172.21.15.103:0/3150384391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 msgr2=0x55a006550510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006550510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.106+0000 7fea76ffd640 1 -- 172.21.15.103:0/3150384391 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0061bf210 con 0x55a006548fc0 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.107+0000 7fea76ffd640 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a00654da80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea68009900 tx=0x7fea68005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99364818109787a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.107+0000 7fea74ff9640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fea640a2cb0 0x7fea640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.107+0000 7fea6d7fa640 1 -- 172.21.15.103:0/3150384391 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68035030 con 0x55a006548fc0 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.107+0000 7fea777fe640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fea640a2cb0 0x7fea640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.107+0000 7fea74ff9640 1 -- 172.21.15.103:0/1220390976 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fea600cea00 con 0x7fea78102a80 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.107+0000 7fea6d7fa640 1 -- 172.21.15.103:0/3150384391 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea68040070 con 0x55a006548fc0 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.107+0000 7fea6d7fa640 1 -- 172.21.15.103:0/3150384391 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68003b30 con 0x55a006548fc0 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 -- 172.21.15.103:0/3150384391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 msgr2=0x55a00654da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea777fe640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fea640a2cb0 0x7fea640a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fea70005fd0 tx=0x7fea70009620 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a00654da80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea68009900 tx=0x7fea68005c00 comp rx=0 tx=0).stop 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 -- 172.21.15.103:0/3150384391 shutdown_connections 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006550510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a00654da80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 --2- 172.21.15.103:0/3150384391 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006548a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 -- 172.21.15.103:0/3150384391 >> 172.21.15.103:0/3150384391 conn(0x55a006543cf0 msgr2=0x55a0065440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 -- 172.21.15.103:0/3150384391 shutdown_connections 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 -- 172.21.15.103:0/3150384391 wait complete. 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 -- start start 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.108+0000 7fea80049740 1 Processor -- start 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea80049740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a006552770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea80049740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006552cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea80049740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006559cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea80049740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a006550fc0 con 0x55a00654e050 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea80049740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a006467520 con 0x55a006548fc0 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea80049740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a006551140 con 0x55a006548680 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea80049740 10 client.0 _ll_register_callbacks cb 0x55a0061bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006559cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea76ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a006552770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006559cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58156/0 (socket says 172.21.15.103:58156) 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea777fe640 1 -- 172.21.15.103:0/1524870978 learned_addr learned my addr 172.21.15.103:0/1524870978 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea6ffff640 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006552cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea76ffd640 1 -- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 msgr2=0x55a006552cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea76ffd640 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006552cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea76ffd640 1 -- 172.21.15.103:0/1524870978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 msgr2=0x55a006559cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea76ffd640 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006559cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea76ffd640 1 -- 172.21.15.103:0/1524870978 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00628b620 con 0x55a006548680 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea777fe640 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006559cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea76ffd640 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a006552770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea68031a70 tx=0x7fea68003660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ad6d1242a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.109+0000 7fea5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea5e7fc640 1 -- 172.21.15.103:0/1524870978 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68052060 con 0x55a006548680 2024-08-19T01:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea68040070 con 0x55a006548680 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea5e7fc640 1 -- 172.21.15.103:0/1524870978 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fea6803c650 con 0x55a006548680 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea5e7fc640 1 -- 172.21.15.103:0/1524870978 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea68035040 con 0x55a006548680 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 10 client.25207 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a006551010 con 0x55a006548680 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea5e7fc640 1 -- 172.21.15.103:0/1524870978 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea68040070 con 0x55a006548680 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea5e7fc640 1 -- 172.21.15.103:0/1524870978 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7fea6804d5a0 con 0x55a006548680 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 10 client.25207 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 10 client.25207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 20 client.25207 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a00655de20 0x55a006560210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 4 client.25207 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.110+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0064b9100 con 0x55a00655de20 2024-08-19T01:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.111+0000 7fea6ffff640 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a00655de20 0x55a006560210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.111+0000 7fea6ffff640 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a00655de20 0x55a006560210 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2bd0e90491da056 server_cookie=83c3bbf73fc60d19 in_seq=0 out_seq=0 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.111+0000 7fea5e7fc640 10 client.25207 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea5e7fc640 1 -- 172.21.15.103:0/1524870978 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0064b9100 con 0x55a00655de20 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea5e7fc640 10 client.25207 handle_command_reply: tid=0 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 client.25207 shutdown 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 msgr2=0x55a006552770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a006552770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea68031a70 tx=0x7fea68003660 comp rx=0 tx=0).stop 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 shutdown_connections 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a00655de20 0x55a006560210 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006559cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006552cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 --2- 172.21.15.103:0/1524870978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a006552770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea5e7fc640 0 client.25207 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 >> 172.21.15.103:0/1524870978 conn(0x55a006543cf0 msgr2=0x55a006545500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 shutdown_connections 2024-08-19T01:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 1 -- 172.21.15.103:0/1524870978 wait complete. 2024-08-19T01:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.112+0000 7fea80049740 20 client.25207 trim_cache size 0 max 16384 2024-08-19T01:37:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.126+0000 7fea80049740 1 Processor -- start 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea80049740 1 -- start start 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea80049740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a0065440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea80049740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006559cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea80049740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006544610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea80049740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea781a8a30 con 0x55a00654e050 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea80049740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fea781aa5a0 con 0x55a006548fc0 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea80049740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea781a9100 con 0x55a006548680 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006544610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006544610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58166/0 (socket says 172.21.15.103:58166) 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea777fe640 1 -- 172.21.15.103:0/1633700244 learned_addr learned my addr 172.21.15.103:0/1633700244 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea76ffd640 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a0065440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea6ffff640 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006559cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea76ffd640 1 -- 172.21.15.103:0/1633700244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 msgr2=0x55a006559cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea76ffd640 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006559cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea76ffd640 1 -- 172.21.15.103:0/1633700244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 msgr2=0x55a006544610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea76ffd640 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006544610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea76ffd640 1 -- 172.21.15.103:0/1633700244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a006542220 con 0x55a006548680 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.127+0000 7fea777fe640 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006544610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.128+0000 7fea76ffd640 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a0065440d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea6802a930 tx=0x7fea68005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd315182ed21e60 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.128+0000 7fea6cff9640 1 -- 172.21.15.103:0/1633700244 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6805d030 con 0x55a006548680 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.128+0000 7fea6cff9640 1 -- 172.21.15.103:0/1633700244 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea68040070 con 0x55a006548680 2024-08-19T01:37:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.128+0000 7fea6cff9640 1 -- 172.21.15.103:0/1633700244 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68052040 con 0x55a006548680 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 -- 172.21.15.103:0/1633700244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 msgr2=0x55a0065440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a0065440d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea6802a930 tx=0x7fea68005fd0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 -- 172.21.15.103:0/1633700244 shutdown_connections 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006544610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548fc0 0x55a006559cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 --2- 172.21.15.103:0/1633700244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548680 0x55a0065440d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 -- 172.21.15.103:0/1633700244 >> 172.21.15.103:0/1633700244 conn(0x55a006543cf0 msgr2=0x55a006552520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 -- 172.21.15.103:0/1633700244 shutdown_connections 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 -- 172.21.15.103:0/1633700244 wait complete. 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 -- start start 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.129+0000 7fea80049740 1 Processor -- start 2024-08-19T01:37:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea80049740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006556dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea80049740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a006557310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea80049740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006558320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea80049740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea781a9ea0 con 0x55a00654e050 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea80049740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fea781a97d0 con 0x55a006548680 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea80049740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea781a6a10 con 0x55a006548fc0 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea80049740 10 client.0 _ll_register_callbacks cb 0x7fea600d6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006558320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006558320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58172/0 (socket says 172.21.15.103:58172) 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 -- 172.21.15.103:0/576981280 learned_addr learned my addr 172.21.15.103:0/576981280 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea6ffff640 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a006557310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea76ffd640 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006556dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 -- 172.21.15.103:0/576981280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 msgr2=0x55a006557310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a006557310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 -- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 msgr2=0x55a006556dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006556dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 -- 172.21.15.103:0/576981280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00654ddf0 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea777fe640 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006558320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fea70012ce0 tx=0x7fea70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2aa4acd4998939 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.130+0000 7fea5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea5e7fc640 1 -- 172.21.15.103:0/576981280 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70004b20 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea5e7fc640 1 -- 172.21.15.103:0/576981280 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fea70004cc0 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea68040070 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea5e7fc640 1 -- 172.21.15.103:0/576981280 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea70029410 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 10 client.15690 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a006559ae0 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea5e7fc640 1 -- 172.21.15.103:0/576981280 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea68040070 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea5e7fc640 1 -- 172.21.15.103:0/576981280 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7fea70035740 con 0x55a00654e050 2024-08-19T01:37:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 20 client.15690 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a0065c8d90 0x55a0065cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 4 client.15690 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.131+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0064b3450 con 0x55a0065c8d90 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.132+0000 7fea76ffd640 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a0065c8d90 0x55a0065cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.132+0000 7fea76ffd640 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a0065c8d90 0x55a0065cb180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f4945e8ea09874 server_cookie=26d80617459032ca in_seq=0 out_seq=0 2024-08-19T01:37:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.132+0000 7fea5e7fc640 10 client.15690 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea5e7fc640 1 -- 172.21.15.103:0/576981280 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0064b3450 con 0x55a0065c8d90 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea5e7fc640 10 client.15690 handle_command_reply: tid=0 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 client.15690 shutdown 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 msgr2=0x55a006558320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006558320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fea70012ce0 tx=0x7fea70004a40 comp rx=0 tx=0).stop 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 shutdown_connections 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a0065c8d90 0x55a0065cb180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a00654e050 0x55a006558320 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea5e7fc640 0 client.15690 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a006548fc0 0x55a006557310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 --2- 172.21.15.103:0/576981280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a006548680 0x55a006556dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 >> 172.21.15.103:0/576981280 conn(0x55a006543cf0 msgr2=0x55a006542680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 shutdown_connections 2024-08-19T01:37:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.144+0000 7fea80049740 1 -- 172.21.15.103:0/576981280 wait complete. 2024-08-19T01:37:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea80049740 20 client.15690 trim_cache size 0 max 16384 2024-08-19T01:37:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:37:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:37:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:37:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:37:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fea640a2cb0 msgr2=0x7fea640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fea640a2cb0 0x7fea640a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fea70005fd0 tx=0x7fea70009620 comp rx=0 tx=0).stop 2024-08-19T01:37:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78102a80 msgr2=0x7fea78197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78102a80 0x7fea78197f90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fea6000d770 tx=0x7fea6000dc30 comp rx=0 tx=0).stop 2024-08-19T01:37:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 shutdown_connections 2024-08-19T01:37:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fea640a2cb0 0x7fea640a5170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea78103440 0x7fea7819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea78102a80 0x7fea78197f90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 --2- 172.21.15.103:0/1220390976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea78101890 0x7fea78197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 >> 172.21.15.103:0/1220390976 conn(0x7fea780fd010 msgr2=0x7fea78105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 shutdown_connections 2024-08-19T01:37:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:00.145+0000 7fea7d28d640 1 -- 172.21.15.103:0/1220390976 wait complete. 2024-08-19T01:37:00.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:36:59 smithi146 ceph-mon[22098]: pgmap v375: 97 pgs: 97 active+clean; 366 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 30 MiB/s wr, 47 op/s 2024-08-19T01:37:00.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:36:59 smithi103 ceph-mon[18436]: pgmap v375: 97 pgs: 97 active+clean; 366 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 30 MiB/s wr, 47 op/s 2024-08-19T01:37:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:36:59 smithi190 ceph-mon[22032]: pgmap v375: 97 pgs: 97 active+clean; 366 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 30 MiB/s wr, 47 op/s 2024-08-19T01:37:02.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:01 smithi146 ceph-mon[22098]: pgmap v376: 97 pgs: 97 active+clean; 410 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 31 op/s 2024-08-19T01:37:02.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:01 smithi103 ceph-mon[18436]: pgmap v376: 97 pgs: 97 active+clean; 410 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 31 op/s 2024-08-19T01:37:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:01 smithi190 ceph-mon[22032]: pgmap v376: 97 pgs: 97 active+clean; 410 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 MiB/s wr, 31 op/s 2024-08-19T01:37:04.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:03 smithi146 ceph-mon[22098]: pgmap v377: 97 pgs: 97 active+clean; 506 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 MiB/s wr, 32 op/s 2024-08-19T01:37:04.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:03 smithi103 ceph-mon[18436]: pgmap v377: 97 pgs: 97 active+clean; 506 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 MiB/s wr, 32 op/s 2024-08-19T01:37:04.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:03 smithi190 ceph-mon[22032]: pgmap v377: 97 pgs: 97 active+clean; 506 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 31 MiB/s wr, 32 op/s 2024-08-19T01:37:05.973 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:37:06.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:05 smithi146 ceph-mon[22098]: pgmap v378: 97 pgs: 97 active+clean; 538 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:37:06.249 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:06.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:05 smithi103 ceph-mon[18436]: pgmap v378: 97 pgs: 97 active+clean; 538 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:37:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:05 smithi190 ceph-mon[22032]: pgmap v378: 97 pgs: 97 active+clean; 538 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 -- 172.21.15.103:0/519131885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102c20 msgr2=0x7f8af010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 --2- 172.21.15.103:0/519131885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102c20 0x7f8af010d770 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ca0 tx=0x7f8ae802a950 comp rx=0 tx=0).stop 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 -- 172.21.15.103:0/519131885 shutdown_connections 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 --2- 172.21.15.103:0/519131885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102c20 0x7f8af010d770 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 --2- 172.21.15.103:0/519131885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8af01022a0 0x7f8af01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 --2- 172.21.15.103:0/519131885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af0100b50 0x7f8af0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 -- 172.21.15.103:0/519131885 >> 172.21.15.103:0/519131885 conn(0x7f8af00fc500 msgr2=0x7f8af00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.788+0000 7f8af52b9640 1 -- 172.21.15.103:0/519131885 shutdown_connections 2024-08-19T01:37:06.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.789+0000 7f8af52b9640 1 -- 172.21.15.103:0/519131885 wait complete. 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.789+0000 7f8af52b9640 1 Processor -- start 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.789+0000 7f8af52b9640 1 -- start start 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8af52b9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af0100b50 0x7f8af0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8af52b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01022a0 0x7f8af01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8af52b9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8af0102c20 0x7f8af019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8af52b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0110bb0 con 0x7f8af01022a0 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8af52b9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0110a30 con 0x7f8af0102c20 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8af52b9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0110d30 con 0x7f8af0100b50 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01022a0 0x7f8af01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01022a0 0x7f8af01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42662/0 (socket says 172.21.15.103:42662) 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 -- 172.21.15.103:0/964463537 learned_addr learned my addr 172.21.15.103:0/964463537 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8af0102c20 0x7f8af019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af0100b50 0x7f8af0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 -- 172.21.15.103:0/964463537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af0100b50 msgr2=0x7f8af0197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af0100b50 0x7f8af0197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 -- 172.21.15.103:0/964463537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8af0102c20 msgr2=0x7f8af019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8af0102c20 0x7f8af019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 -- 172.21.15.103:0/964463537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af019cf50 con 0x7f8af01022a0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01022a0 0x7f8af01984e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c920 tx=0x7f8ae400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.790+0000 7f8ad7fff640 1 -- 172.21.15.103:0/964463537 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4007c70 con 0x7f8af01022a0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.791+0000 7f8ad7fff640 1 -- 172.21.15.103:0/964463537 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ae4007e10 con 0x7f8af01022a0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.791+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af019d240 con 0x7f8af01022a0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.791+0000 7f8ad7fff640 1 -- 172.21.15.103:0/964463537 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4018820 con 0x7f8af01022a0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.791+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af01a4b20 con 0x7f8af01022a0 2024-08-19T01:37:06.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.793+0000 7f8ad7fff640 1 -- 172.21.15.103:0/964463537 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8ae40189c0 con 0x7f8af01022a0 2024-08-19T01:37:06.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 Processor -- start 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 -- start start 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8ad7fff640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ad80a2b80 0x7f8ad80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5210 0x562498ce55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5b30 0x562498cefeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ad80a2b80 0x7f8ad80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8ad7fff640 1 -- 172.21.15.103:0/964463537 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8ae40ceff0 con 0x7f8af01022a0 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8aeeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5210 0x562498ce55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498cf03f0 0x562498cf27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8aeeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5210 0x562498ce55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42674/0 (socket says 172.21.15.103:42674) 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8aeeffd640 1 -- 172.21.15.103:0/307682715 learned_addr learned my addr 172.21.15.103:0/307682715 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:06.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562498c4a980 con 0x562498ce5210 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562498be2a70 con 0x562498cf03f0 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562498c46940 con 0x562498ce5b30 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.794+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5b30 0x562498cefeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aeeffd640 1 -- 172.21.15.103:0/307682715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5b30 msgr2=0x562498cefeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498cf03f0 0x562498cf27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5b30 0x562498cefeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aeeffd640 1 -- 172.21.15.103:0/307682715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498cf03f0 msgr2=0x562498cf27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498cf03f0 0x562498cf27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aeeffd640 1 -- 172.21.15.103:0/307682715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562498c456e0 con 0x562498ce5210 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498cf03f0 0x562498cf27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ad80a2b80 0x7f8ad80a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009430 tx=0x7f8ae8009800 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5210 0x562498ce55f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000cc60 tx=0x7f8ae000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fd2e1253c53fae server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:06.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8ad57fa640 1 -- 172.21.15.103:0/307682715 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0014070 con 0x562498ce5210 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8ad57fa640 1 -- 172.21.15.103:0/307682715 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ae00047a0 con 0x562498ce5210 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.795+0000 7f8ad57fa640 1 -- 172.21.15.103:0/307682715 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0010670 con 0x562498ce5210 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5210 msgr2=0x562498ce55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5210 0x562498ce55f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000cc60 tx=0x7f8ae000e720 comp rx=0 tx=0).stop 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 shutdown_connections 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498cf03f0 0x562498cf27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5b30 0x562498cefeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 --2- 172.21.15.103:0/307682715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5210 0x562498ce55f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 >> 172.21.15.103:0/307682715 conn(0x562498ce17a0 msgr2=0x562498ce1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 shutdown_connections 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.796+0000 7f8af76b7740 1 -- 172.21.15.103:0/307682715 wait complete. 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 -- start start 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 Processor -- start 2024-08-19T01:37:06.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498ced230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ced770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cedcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562498cf2fc0 con 0x562498ce5b30 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562498c05aa0 con 0x562498ce5210 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562498cf3140 con 0x562498cf03f0 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8af76b7740 10 client.0 _ll_register_callbacks cb 0x562498bf4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8aee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ced770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8aee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ced770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42690/0 (socket says 172.21.15.103:42690) 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8aee7fc640 1 -- 172.21.15.103:0/590817138 learned_addr learned my addr 172.21.15.103:0/590817138 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498ced230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.797+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cedcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8aee7fc640 1 -- 172.21.15.103:0/590817138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 msgr2=0x562498cedcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cedcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8aee7fc640 1 -- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 msgr2=0x562498ced230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498ced230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8aee7fc640 1 -- 172.21.15.103:0/590817138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562498be9650 con 0x562498ce5b30 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ced770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4097ce0 tx=0x7f8ae409c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6767cd460239b47 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/590817138 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40860b0 con 0x562498ce5b30 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/590817138 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8ae4086250 con 0x562498ce5b30 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae00047a0 con 0x562498ce5b30 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/590817138 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ae40ba170 con 0x562498ce5b30 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8af76b7740 10 client.15702 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624989bc3c0 con 0x562498ce5b30 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/590817138 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ae40a4060 con 0x562498ce5b30 2024-08-19T01:37:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.798+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/590817138 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f8ae40975d0 con 0x562498ce5b30 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8af76b7740 10 client.15702 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8af76b7740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8af76b7740 20 client.15702 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8af76b7740 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498cffda0 0x562498d02220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8af76b7740 4 client.15702 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8ae00047a0 con 0x562498cffda0 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498cffda0 0x562498d02220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498cffda0 0x562498d02220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301f108dce7427b5 server_cookie=89b70c327a29b6e7 in_seq=0 out_seq=0 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.799+0000 7f8ad2ff5640 10 client.15702 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/590817138 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8ae00047a0 con 0x562498cffda0 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8ad2ff5640 10 client.15702 handle_command_reply: tid=0 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 client.15702 shutdown 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 msgr2=0x562498ced770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ced770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4097ce0 tx=0x7f8ae409c0c0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 shutdown_connections 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498cffda0 0x562498d02220 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cedcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ced770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8ad2ff5640 0 client.15702 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 --2- 172.21.15.103:0/590817138 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498ced230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.800+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 >> 172.21.15.103:0/590817138 conn(0x562498ce17a0 msgr2=0x562498ce7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.801+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 shutdown_connections 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.801+0000 7f8af76b7740 1 -- 172.21.15.103:0/590817138 wait complete. 2024-08-19T01:37:06.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.801+0000 7f8af76b7740 20 client.15702 trim_cache size 0 max 16384 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 Processor -- start 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 -- start start 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5210 0x562498ce7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5b30 0x562498cef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498cf03f0 0x562498d07af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc00e380 con 0x562498cf03f0 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc00fef0 con 0x562498ce5b30 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.822+0000 7f8af76b7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc00ea50 con 0x562498ce5210 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498cf03f0 0x562498d07af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498cf03f0 0x562498d07af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42698/0 (socket says 172.21.15.103:42698) 2024-08-19T01:37:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 -- 172.21.15.103:0/2349278875 learned_addr learned my addr 172.21.15.103:0/2349278875 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5210 0x562498ce7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5b30 0x562498cef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 -- 172.21.15.103:0/2349278875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5210 msgr2=0x562498ce7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5210 0x562498ce7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 -- 172.21.15.103:0/2349278875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5b30 msgr2=0x562498cef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5b30 0x562498cef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 -- 172.21.15.103:0/2349278875 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562498ce5960 con 0x562498cf03f0 2024-08-19T01:37:06.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498cf03f0 0x562498d07af0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ca0 tx=0x7f8ae803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d257d891c4ebf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:06.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.823+0000 7f8ad4ff9640 1 -- 172.21.15.103:0/2349278875 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8034040 con 0x562498cf03f0 2024-08-19T01:37:06.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8ad4ff9640 1 -- 172.21.15.103:0/2349278875 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ae8003e40 con 0x562498cf03f0 2024-08-19T01:37:06.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8ad4ff9640 1 -- 172.21.15.103:0/2349278875 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80046f0 con 0x562498cf03f0 2024-08-19T01:37:06.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 -- 172.21.15.103:0/2349278875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498cf03f0 msgr2=0x562498d07af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498cf03f0 0x562498d07af0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ca0 tx=0x7f8ae803f000 comp rx=0 tx=0).stop 2024-08-19T01:37:06.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 -- 172.21.15.103:0/2349278875 shutdown_connections 2024-08-19T01:37:06.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498cf03f0 0x562498d07af0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5b30 0x562498cef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 --2- 172.21.15.103:0/2349278875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498ce5210 0x562498ce7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 -- 172.21.15.103:0/2349278875 >> 172.21.15.103:0/2349278875 conn(0x562498ce17a0 msgr2=0x562498cee980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 -- 172.21.15.103:0/2349278875 shutdown_connections 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.824+0000 7f8af76b7740 1 -- 172.21.15.103:0/2349278875 wait complete. 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 -- start start 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 Processor -- start 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498cf89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ce9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc00f7f0 con 0x562498ce5b30 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc00f120 con 0x562498ce5210 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc00c360 con 0x562498cf03f0 2024-08-19T01:37:06.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8af76b7740 10 client.0 _ll_register_callbacks cb 0x7f8ae4018d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8aee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ce9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8aee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ce9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42704/0 (socket says 172.21.15.103:42704) 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8aee7fc640 1 -- 172.21.15.103:0/1154667497 learned_addr learned my addr 172.21.15.103:0/1154667497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.825+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498cf89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aee7fc640 1 -- 172.21.15.103:0/1154667497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 msgr2=0x562498cea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aee7fc640 1 -- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 msgr2=0x562498cf89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498cf89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aee7fc640 1 -- 172.21.15.103:0/1154667497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562498ceab30 con 0x562498ce5b30 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498cf89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ce9ee0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8ae409c130 tx=0x7f8ae40974d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e896db1a707416 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:06.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/1154667497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae409cdb0 con 0x562498ce5b30 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/1154667497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8ae40a4060 con 0x562498ce5b30 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562498c57ef0 con 0x562498ce5b30 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/1154667497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ae40862c0 con 0x562498ce5b30 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8af76b7740 10 client.15708 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.826+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562498d6ee20 con 0x562498ce5b30 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/1154667497 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562498c57ef0 con 0x562498ce5b30 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/1154667497 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f8ae409a8d0 con 0x562498ce5b30 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8af76b7740 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8af76b7740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8af76b7740 20 client.15708 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8af76b7740 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498d6f5d0 0x562498d719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:06.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8af76b7740 4 client.15708 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562498c52400 con 0x562498d6f5d0 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498d6f5d0 0x562498d719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498d6f5d0 0x562498d719f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4215070fdf588c server_cookie=1b06889927ca663f in_seq=0 out_seq=0 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.827+0000 7f8ad2ff5640 10 client.15708 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8ad2ff5640 1 -- 172.21.15.103:0/1154667497 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562498c52400 con 0x562498d6f5d0 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8ad2ff5640 10 client.15708 handle_command_reply: tid=0 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 client.15708 shutdown 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 msgr2=0x562498ce9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ce9ee0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8ae409c130 tx=0x7f8ae40974d0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 shutdown_connections 2024-08-19T01:37:06.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x562498d6f5d0 0x562498d719f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562498cf03f0 0x562498cea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562498ce5b30 0x562498ce9ee0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 --2- 172.21.15.103:0/1154667497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562498ce5210 0x562498cf89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 >> 172.21.15.103:0/1154667497 conn(0x562498ce17a0 msgr2=0x562498ce0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8ad2ff5640 0 client.15708 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 shutdown_connections 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 1 -- 172.21.15.103:0/1154667497 wait complete. 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.828+0000 7f8af76b7740 20 client.15708 trim_cache size 0 max 16384 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:37:06.835 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:37:06.836 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:37:06.836 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:37:06.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ad80a2b80 msgr2=0x7f8ad80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ad80a2b80 0x7f8ad80a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009430 tx=0x7f8ae8009800 comp rx=0 tx=0).stop 2024-08-19T01:37:06.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01022a0 msgr2=0x7f8af01984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:06.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01022a0 0x7f8af01984e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c920 tx=0x7f8ae400cde0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 shutdown_connections 2024-08-19T01:37:06.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ad80a2b80 0x7f8ad80a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8af0102c20 0x7f8af019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01022a0 0x7f8af01984e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 --2- 172.21.15.103:0/964463537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af0100b50 0x7f8af0197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:06.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 >> 172.21.15.103:0/964463537 conn(0x7f8af00fc500 msgr2=0x7f8af0069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:06.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 shutdown_connections 2024-08-19T01:37:06.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:06.829+0000 7f8af52b9640 1 -- 172.21.15.103:0/964463537 wait complete. 2024-08-19T01:37:07.792 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:37:07.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:37:08.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:07 smithi103 ceph-mon[18436]: pgmap v379: 97 pgs: 97 active+clean; 574 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:37:08.099 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:07 smithi146 ceph-mon[22098]: pgmap v379: 97 pgs: 97 active+clean; 574 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:37:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:07 smithi190 ceph-mon[22032]: pgmap v379: 97 pgs: 97 active+clean; 574 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:37:08.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 -- 172.21.15.103:0/3615646419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b81022a0 msgr2=0x7fe9b81026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:08.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/3615646419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b81022a0 0x7fe9b81026e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009900 tx=0x7fe9a802a480 comp rx=0 tx=0).stop 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 -- 172.21.15.103:0/3615646419 shutdown_connections 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/3615646419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8102c20 0x7fe9b810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/3615646419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b81022a0 0x7fe9b81026e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/3615646419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9b8100b50 0x7fe9b8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 -- 172.21.15.103:0/3615646419 >> 172.21.15.103:0/3615646419 conn(0x7fe9b80fc500 msgr2=0x7fe9b80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 -- 172.21.15.103:0/3615646419 shutdown_connections 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.618+0000 7fe9bfd15640 1 -- 172.21.15.103:0/3615646419 wait complete. 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.619+0000 7fe9bfd15640 1 Processor -- start 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.619+0000 7fe9bfd15640 1 -- start start 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.619+0000 7fe9bfd15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8100b50 0x7fe9b80fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.619+0000 7fe9bfd15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9b81022a0 0x7fe9b80fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9bfd15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b8102c20 0x7fe9b80fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9bfd15640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8110bb0 con 0x7fe9b8102c20 2024-08-19T01:37:08.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9bfd15640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8110a30 con 0x7fe9b8100b50 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9bfd15640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8110d30 con 0x7fe9b81022a0 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b8102c20 0x7fe9b80fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b8102c20 0x7fe9b80fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42722/0 (socket says 172.21.15.103:42722) 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 -- 172.21.15.103:0/2964519169 learned_addr learned my addr 172.21.15.103:0/2964519169 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9bd2b0640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9b81022a0 0x7fe9b80fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9bdab1640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8100b50 0x7fe9b80fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 -- 172.21.15.103:0/2964519169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9b81022a0 msgr2=0x7fe9b80fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9b81022a0 0x7fe9b80fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 -- 172.21.15.103:0/2964519169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8100b50 msgr2=0x7fe9b80fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8100b50 0x7fe9b80fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 -- 172.21.15.103:0/2964519169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b81a65f0 con 0x7fe9b8102c20 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.620+0000 7fe9be2b2640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b8102c20 0x7fe9b80fd430 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000eaa0 tx=0x7fe9b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.621+0000 7fe9aeffd640 1 -- 172.21.15.103:0/2964519169 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000cdb0 con 0x7fe9b8102c20 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.621+0000 7fe9aeffd640 1 -- 172.21.15.103:0/2964519169 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9b0005020 con 0x7fe9b8102c20 2024-08-19T01:37:08.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.621+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b81a68e0 con 0x7fe9b8102c20 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.621+0000 7fe9aeffd640 1 -- 172.21.15.103:0/2964519169 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00116d0 con 0x7fe9b8102c20 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.621+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b81a6da0 con 0x7fe9b8102c20 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.622+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b8100f30 con 0x7fe9b8102c20 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.623+0000 7fe9aeffd640 1 -- 172.21.15.103:0/2964519169 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe9b00051d0 con 0x7fe9b8102c20 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.624+0000 7fe9aeffd640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe9a40a2b80 0x7fe9a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.624+0000 7fe9bdab1640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe9a40a2b80 0x7fe9a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.624+0000 7fe9aeffd640 1 -- 172.21.15.103:0/2964519169 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe9b00cf110 con 0x7fe9b8102c20 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.625+0000 7fe9bdab1640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe9a40a2b80 0x7fe9a40a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005fd0 tx=0x7fe9b400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:08.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.631+0000 7fe9aeffd640 1 -- 172.21.15.103:0/2964519169 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe9b0016030 con 0x7fe9b8102c20 2024-08-19T01:37:08.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.868+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9b8109dd0 con 0x7fe9b8102c20 2024-08-19T01:37:08.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.869+0000 7fe9aeffd640 1 -- 172.21.15.103:0/2964519169 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11990 (secure 0 0 0) 0x7fe9b009ba50 con 0x7fe9b8102c20 2024-08-19T01:37:08.871 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:37:08.871 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-08-19T01:36:46:917569+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:36:46.342690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:37:08.871 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-08-19T01:37:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe9a40a2b80 msgr2=0x7fe9a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe9a40a2b80 0x7fe9a40a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005fd0 tx=0x7fe9b400b390 comp rx=0 tx=0).stop 2024-08-19T01:37:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b8102c20 msgr2=0x7fe9b80fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b8102c20 0x7fe9b80fd430 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000eaa0 tx=0x7fe9b000ef60 comp rx=0 tx=0).stop 2024-08-19T01:37:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 shutdown_connections 2024-08-19T01:37:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe9a40a2b80 0x7fe9a40a5040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9b8102c20 0x7fe9b80fd430 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9b81022a0 0x7fe9b80fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 --2- 172.21.15.103:0/2964519169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9b8100b50 0x7fe9b80fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 >> 172.21.15.103:0/2964519169 conn(0x7fe9b80fc500 msgr2=0x7fe9b8069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 shutdown_connections 2024-08-19T01:37:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:08.871+0000 7fe9bfd15640 1 -- 172.21.15.103:0/2964519169 wait complete. 2024-08-19T01:37:09.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:08 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2964519169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:09.261 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:08 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2964519169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:08 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2964519169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:09.615 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:37:09.889 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:10.174 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:09 smithi103 ceph-mon[18436]: pgmap v380: 97 pgs: 97 active+clean; 666 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 28 op/s 2024-08-19T01:37:10.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:09 smithi146 ceph-mon[22098]: pgmap v380: 97 pgs: 97 active+clean; 666 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 28 op/s 2024-08-19T01:37:10.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:09 smithi190 ceph-mon[22032]: pgmap v380: 97 pgs: 97 active+clean; 666 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 28 op/s 2024-08-19T01:37:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 -- 172.21.15.103:0/840395232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101150 msgr2=0x7fe2d010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/840395232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101150 0x7fe2d010d680 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009cd0 tx=0x7fe2c802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 -- 172.21.15.103:0/840395232 shutdown_connections 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/840395232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101150 0x7fe2d010d680 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/840395232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d01007b0 0x7fe2d0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/840395232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d01067b0 0x7fe2d0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 -- 172.21.15.103:0/840395232 >> 172.21.15.103:0/840395232 conn(0x7fe2d00fc4b0 msgr2=0x7fe2d00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 -- 172.21.15.103:0/840395232 shutdown_connections 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.395+0000 7fe2d73a9640 1 -- 172.21.15.103:0/840395232 wait complete. 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 Processor -- start 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 -- start start 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01007b0 0x7fe2d00fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d0101150 0x7fe2d00fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d01067b0 0x7fe2d00fd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0110a80 con 0x7fe2d01007b0 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0110900 con 0x7fe2d0101150 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d73a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0110c00 con 0x7fe2d01067b0 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01007b0 0x7fe2d00fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01007b0 0x7fe2d00fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42740/0 (socket says 172.21.15.103:42740) 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 -- 172.21.15.103:0/576605998 learned_addr learned my addr 172.21.15.103:0/576605998 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d4944640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d0101150 0x7fe2d00fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 -- 172.21.15.103:0/576605998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d01067b0 msgr2=0x7fe2d00fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d01067b0 0x7fe2d00fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 -- 172.21.15.103:0/576605998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d0101150 msgr2=0x7fe2d00fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d0101150 0x7fe2d00fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.396+0000 7fe2d5145640 1 -- 172.21.15.103:0/576605998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d01a65d0 con 0x7fe2d01007b0 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.397+0000 7fe2d5145640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01007b0 0x7fe2d00fcbd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00d950 tx=0x7fe2cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.397+0000 7fe2c67fc640 1 -- 172.21.15.103:0/576605998 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc017070 con 0x7fe2d01007b0 2024-08-19T01:37:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.397+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d01a68c0 con 0x7fe2d01007b0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.397+0000 7fe2c67fc640 1 -- 172.21.15.103:0/576605998 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2cc009c30 con 0x7fe2d01007b0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.397+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d01a6e50 con 0x7fe2d01007b0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.397+0000 7fe2c67fc640 1 -- 172.21.15.103:0/576605998 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00f620 con 0x7fe2d01007b0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.399+0000 7fe2c67fc640 1 -- 172.21.15.103:0/576605998 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe2cc00f7c0 con 0x7fe2d01007b0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 Processor -- start 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 -- start start 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cacff0 0x559670cad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cb7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2c67fc640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2c00 0x7fe2bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559670c12a40 con 0x559670cacff0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559670baa9d0 con 0x559670cad910 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559670c0ea00 con 0x559670cb81d0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d5946640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cacff0 0x559670cad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d4944640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2c00 0x7fe2bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2c67fc640 1 -- 172.21.15.103:0/576605998 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe2cc0cebb0 con 0x7fe2d01007b0 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d5946640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cacff0 0x559670cad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42752/0 (socket says 172.21.15.103:42752) 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.400+0000 7fe2d5946640 1 -- 172.21.15.103:0/2511320224 learned_addr learned my addr 172.21.15.103:0/2511320224 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5145640 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cb7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5946640 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5946640 1 -- 172.21.15.103:0/2511320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 msgr2=0x559670cba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5946640 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5946640 1 -- 172.21.15.103:0/2511320224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 msgr2=0x559670cb7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5946640 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cb7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5946640 1 -- 172.21.15.103:0/2511320224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559670bb1650 con 0x559670cacff0 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5145640 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cb7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d4944640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2c00 0x7fe2bc0a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00fe660 tx=0x7fe2c0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2d5946640 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cacff0 0x559670cad3d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8004af0 tx=0x7fe2c8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b8e8ef52a74035 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2bb7fe640 1 -- 172.21.15.103:0/2511320224 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8031ad0 con 0x559670cacff0 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2bb7fe640 1 -- 172.21.15.103:0/2511320224 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c8036040 con 0x559670cacff0 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.401+0000 7fe2bb7fe640 1 -- 172.21.15.103:0/2511320224 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80044a0 con 0x559670cacff0 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2511320224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cacff0 msgr2=0x559670cad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cacff0 0x559670cad3d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8004af0 tx=0x7fe2c8005d80 comp rx=0 tx=0).stop 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2511320224 shutdown_connections 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cb7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2511320224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cacff0 0x559670cad3d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2511320224 >> 172.21.15.103:0/2511320224 conn(0x559670ca9580 msgr2=0x559670ca9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2511320224 shutdown_connections 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2511320224 wait complete. 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.402+0000 7fe2d97a7740 1 -- start start 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 1 Processor -- start 2024-08-19T01:37:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cb5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cb81d0 0x559670cb5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559670cbada0 con 0x559670cad910 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559670bcdb20 con 0x559670cb81d0 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559670cbaf20 con 0x559670cacff0 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d97a7740 10 client.0 _ll_register_callbacks cb 0x559670bcbbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d5946640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cb81d0 0x559670cb5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d4944640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d5946640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cb81d0 0x559670cb5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:46932/0 (socket says 172.21.15.103:46932) 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d5946640 1 -- 172.21.15.103:0/2224080305 learned_addr learned my addr 172.21.15.103:0/2224080305 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.403+0000 7fe2d5145640 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cb5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d5946640 1 -- 172.21.15.103:0/2224080305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 msgr2=0x559670cb5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d5946640 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cb5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d5946640 1 -- 172.21.15.103:0/2224080305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 msgr2=0x559670cb5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d5946640 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d5946640 1 -- 172.21.15.103:0/2224080305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596709843c0 con 0x559670cb81d0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d5946640 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cb81d0 0x559670cb5a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2c802b060 tx=0x7fe2c8005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d192611a7e5321 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2224080305 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8003d80 con 0x559670cb81d0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c8036040 con 0x559670cb81d0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2224080305 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe2c8004b40 con 0x559670cb81d0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2224080305 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2c8057020 con 0x559670cb81d0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d97a7740 10 client.25259 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.404+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559670cc6970 con 0x559670cb81d0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2224080305 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c8036040 con 0x559670cb81d0 2024-08-19T01:37:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2224080305 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7fe2c8030070 con 0x559670cb81d0 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2d97a7740 10 client.25259 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2d97a7740 10 client.25259 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2d97a7740 20 client.25259 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670cc7cc0 0x559670cca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2d97a7740 4 client.25259 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559670c1ffb0 con 0x559670cc7cc0 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.405+0000 7fe2d5145640 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670cc7cc0 0x559670cca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2b8ff9640 10 client.25259 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d5145640 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670cc7cc0 0x559670cca0b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d58c718498b30956 server_cookie=eeabb84edcf9466 in_seq=0 out_seq=0 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2224080305 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559670c1ffb0 con 0x559670cc7cc0 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2b8ff9640 10 client.25259 handle_command_reply: tid=0 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 client.25259 shutdown 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cb81d0 msgr2=0x559670cb5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cb81d0 0x559670cb5a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2c802b060 tx=0x7fe2c8005c50 comp rx=0 tx=0).stop 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 shutdown_connections 2024-08-19T01:37:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670cc7cc0 0x559670cca0b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cb81d0 0x559670cb5a90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/2224080305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cb5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2b8ff9640 0 client.25259 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.406+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 >> 172.21.15.103:0/2224080305 conn(0x559670ca9580 msgr2=0x559670caf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.407+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 shutdown_connections 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.407+0000 7fe2d97a7740 1 -- 172.21.15.103:0/2224080305 wait complete. 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.407+0000 7fe2d97a7740 20 client.25259 trim_cache size 0 max 16384 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.423+0000 7fe2d97a7740 1 Processor -- start 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.423+0000 7fe2d97a7740 1 -- start start 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cacff0 0x559670cb6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cb27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b400e360 con 0x559670cad910 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b400fed0 con 0x559670cacff0 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b400ea30 con 0x559670cb81d0 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42772/0 (socket says 172.21.15.103:42772) 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 -- 172.21.15.103:0/4157404062 learned_addr learned my addr 172.21.15.103:0/4157404062 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d5145640 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cacff0 0x559670cb6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d5946640 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cb27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 -- 172.21.15.103:0/4157404062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 msgr2=0x559670cb27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cb27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 -- 172.21.15.103:0/4157404062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cacff0 msgr2=0x559670cb6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cacff0 0x559670cb6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 -- 172.21.15.103:0/4157404062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559670cad740 con 0x559670cad910 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.424+0000 7fe2d4944640 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb2270 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d8c0 tx=0x7fe2c0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9096b073e7dcbd63 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2baffd640 1 -- 172.21.15.103:0/4157404062 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000de40 con 0x559670cad910 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2baffd640 1 -- 172.21.15.103:0/4157404062 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c0025070 con 0x559670cad910 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2baffd640 1 -- 172.21.15.103:0/4157404062 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0020d60 con 0x559670cad910 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 -- 172.21.15.103:0/4157404062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 msgr2=0x559670cb2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb2270 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d8c0 tx=0x7fe2c0010000 comp rx=0 tx=0).stop 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 -- 172.21.15.103:0/4157404062 shutdown_connections 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cb81d0 0x559670cb27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cad910 0x559670cb2270 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/4157404062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cacff0 0x559670cb6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 -- 172.21.15.103:0/4157404062 >> 172.21.15.103:0/4157404062 conn(0x559670ca9580 msgr2=0x559670cb6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.425+0000 7fe2d97a7740 1 -- 172.21.15.103:0/4157404062 shutdown_connections 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.426+0000 7fe2d97a7740 1 -- 172.21.15.103:0/4157404062 wait complete. 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.426+0000 7fe2d97a7740 1 -- start start 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.426+0000 7fe2d97a7740 1 Processor -- start 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.426+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cc9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d97a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cb81d0 0x559670cca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b400f7d0 con 0x559670cb81d0 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b400f100 con 0x559670cad910 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d97a7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b400c340 con 0x559670cacff0 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d97a7740 10 client.0 _ll_register_callbacks cb 0x7fe2c80316e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d5946640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cb81d0 0x559670cca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d5946640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cb81d0 0x559670cca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42788/0 (socket says 172.21.15.103:42788) 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d5946640 1 -- 172.21.15.103:0/3163700066 learned_addr learned my addr 172.21.15.103:0/3163700066 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d4944640 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d5145640 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cc9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d4944640 1 -- 172.21.15.103:0/3163700066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 msgr2=0x559670cc9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d4944640 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cc9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d4944640 1 -- 172.21.15.103:0/3163700066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cb81d0 msgr2=0x559670cca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d4944640 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cb81d0 0x559670cca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d4944640 1 -- 172.21.15.103:0/3163700066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559670d306c0 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2d4944640 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cca4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x559670cb4830 tx=0x7fe2c0020690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c4c17e05e61d99 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.427+0000 7fe2b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/3163700066 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0029410 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/3163700066 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe2c00295b0 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c0025070 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/3163700066 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2c0038050 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2d97a7740 10 client.25265 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559670d319a0 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/3163700066 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c0029c80 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/3163700066 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7fe2c0020ce0 con 0x559670cad910 2024-08-19T01:37:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2d97a7740 10 client.25265 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2d97a7740 10 client.25265 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:37:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2d97a7740 20 client.25265 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.428+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670d320e0 0x559670d34500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2d97a7740 4 client.25265 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:10.455 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:37:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe2c0025070 con 0x559670d320e0 2024-08-19T01:37:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2d5145640 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670d320e0 0x559670d34500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2d5145640 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670d320e0 0x559670d34500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=634e76225b2339ef server_cookie=42aa9a8812658e4b in_seq=0 out_seq=0 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2b8ff9640 10 client.25265 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/3163700066 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe2c0025070 con 0x559670d320e0 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2b8ff9640 10 client.25265 handle_command_reply: tid=0 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2d97a7740 1 client.25265 shutdown 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.429+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 msgr2=0x559670cca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cca4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x559670cb4830 tx=0x7fe2c0020690 comp rx=0 tx=0).stop 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 shutdown_connections 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x559670d320e0 0x559670d34500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559670cb81d0 0x559670cca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559670cad910 0x559670cca4b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 --2- 172.21.15.103:0/3163700066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559670cacff0 0x559670cc9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 >> 172.21.15.103:0/3163700066 conn(0x559670ca9580 msgr2=0x559670cc39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2b8ff9640 0 client.25265 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 shutdown_connections 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 1 -- 172.21.15.103:0/3163700066 wait complete. 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d97a7740 20 client.25265 trim_cache size 0 max 16384 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2c00 msgr2=0x7fe2bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2c00 0x7fe2bc0a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00fe660 tx=0x7fe2c0008040 comp rx=0 tx=0).stop 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01007b0 msgr2=0x7fe2d00fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.430+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01007b0 0x7fe2d00fcbd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00d950 tx=0x7fe2cc00de10 comp rx=0 tx=0).stop 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 shutdown_connections 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2c00 0x7fe2bc0a50c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d01067b0 0x7fe2d00fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d0101150 0x7fe2d00fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 --2- 172.21.15.103:0/576605998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01007b0 0x7fe2d00fcbd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 >> 172.21.15.103:0/576605998 conn(0x7fe2d00fc4b0 msgr2=0x7fe2d006a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 shutdown_connections 2024-08-19T01:37:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:10.431+0000 7fe2d73a9640 1 -- 172.21.15.103:0/576605998 wait complete. 2024-08-19T01:37:11.287 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:37:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:12.026 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:11 smithi103 ceph-mon[18436]: pgmap v381: 97 pgs: 97 active+clean; 698 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 -- 172.21.15.103:0/3136347298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41018c0 msgr2=0x7f41c4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 --2- 172.21.15.103:0/3136347298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41018c0 0x7f41c4101d20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009930 tx=0x7f41b402a420 comp rx=0 tx=0).stop 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 -- 172.21.15.103:0/3136347298 shutdown_connections 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 --2- 172.21.15.103:0/3136347298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c4102260 0x7f41c410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 --2- 172.21.15.103:0/3136347298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41018c0 0x7f41c4101d20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 --2- 172.21.15.103:0/3136347298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c41078c0 0x7f41c4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 -- 172.21.15.103:0/3136347298 >> 172.21.15.103:0/3136347298 conn(0x7f41c40fd5e0 msgr2=0x7f41c40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 -- 172.21.15.103:0/3136347298 shutdown_connections 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 -- 172.21.15.103:0/3136347298 wait complete. 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.120+0000 7f41cba8e640 1 Processor -- start 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41cba8e640 1 -- start start 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41cba8e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c41018c0 0x7f41c41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41cba8e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4102260 0x7f41c4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41cba8e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41078c0 0x7f41c419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41cba8e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4111bc0 con 0x7f41c41078c0 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41cba8e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4111a40 con 0x7f41c41018c0 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41cba8e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4111d40 con 0x7f41c4102260 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41078c0 0x7f41c419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41078c0 0x7f41c419c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52688/0 (socket says 172.21.15.103:52688) 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 -- 172.21.15.103:0/2238402490 learned_addr learned my addr 172.21.15.103:0/2238402490 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41c982a640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c41018c0 0x7f41c41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41c9029640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4102260 0x7f41c4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 -- 172.21.15.103:0/2238402490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4102260 msgr2=0x7f41c4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4102260 0x7f41c4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 -- 172.21.15.103:0/2238402490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c41018c0 msgr2=0x7f41c41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c41018c0 0x7f41c41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41ca02b640 1 -- 172.21.15.103:0/2238402490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c419d0a0 con 0x7f41c41078c0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.121+0000 7f41c9029640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4102260 0x7f41c4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.122+0000 7f41ca02b640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41078c0 0x7f41c419c9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00eaa0 tx=0x7f41bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.122+0000 7f41baffd640 1 -- 172.21.15.103:0/2238402490 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00cdb0 con 0x7f41c41078c0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.122+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c419d390 con 0x7f41c41078c0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.122+0000 7f41baffd640 1 -- 172.21.15.103:0/2238402490 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41bc005020 con 0x7f41c41078c0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.122+0000 7f41baffd640 1 -- 172.21.15.103:0/2238402490 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc0116d0 con 0x7f41c41078c0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.122+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41a4c70 con 0x7f41c41078c0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.124+0000 7f41cde8c740 1 Processor -- start 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.124+0000 7f41baffd640 1 -- 172.21.15.103:0/2238402490 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f41bc0051d0 con 0x7f41c41078c0 2024-08-19T01:37:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41cde8c740 1 -- start start 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57b9830 0x5577b57c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57c85e0 0x5577b57ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41cde8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5577b571ea40 con 0x5577b57c85e0 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41cde8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5577b56b69d0 con 0x5577b57b9830 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41cde8c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5577b571aa00 con 0x5577b57b8f10 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41c9029640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57c85e0 0x5577b57ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41c982a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57b9830 0x5577b57c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41c9029640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57c85e0 0x5577b57ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52698/0 (socket says 172.21.15.103:52698) 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41c9029640 1 -- 172.21.15.103:0/2031482395 learned_addr learned my addr 172.21.15.103:0/2031482395 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41c982a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57b9830 0x5577b57c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:49508/0 (socket says 172.21.15.103:49508) 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.125+0000 7f41c9029640 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41c982a640 1 -- 172.21.15.103:0/2031482395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 msgr2=0x5577b57b92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41c982a640 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57b92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41c982a640 1 -- 172.21.15.103:0/2031482395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57c85e0 msgr2=0x5577b57ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41c982a640 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57c85e0 0x5577b57ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41c982a640 1 -- 172.21.15.103:0/2031482395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b56bd650 con 0x5577b57b9830 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41baffd640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f41b00a2b80 0x7f41b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41c9029640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f41b00a2b80 0x7f41b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41baffd640 1 -- 172.21.15.103:0/2238402490 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f41bc0cf210 con 0x7f41c41078c0 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41c982a640 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57b9830 0x5577b57c80a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f41c000aac0 tx=0x7f41c000af80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f064c3f30d7f0ed server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41affff640 1 -- 172.21.15.103:0/2031482395 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0004020 con 0x5577b57b9830 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41affff640 1 -- 172.21.15.103:0/2031482395 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41c00041c0 con 0x5577b57b9830 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.126+0000 7f41affff640 1 -- 172.21.15.103:0/2031482395 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c00186a0 con 0x5577b57b9830 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 -- 172.21.15.103:0/2031482395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57b9830 msgr2=0x5577b57c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41c9029640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f41b00a2b80 0x7f41b00a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41b402aa10 tx=0x7f41b4040040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57b9830 0x5577b57c80a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f41c000aac0 tx=0x7f41c000af80 comp rx=0 tx=0).stop 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 -- 172.21.15.103:0/2031482395 shutdown_connections 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57c85e0 0x5577b57ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57b9830 0x5577b57c80a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 --2- 172.21.15.103:0/2031482395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57b92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 -- 172.21.15.103:0/2031482395 >> 172.21.15.103:0/2031482395 conn(0x5577b57b5580 msgr2=0x5577b57b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 -- 172.21.15.103:0/2031482395 shutdown_connections 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.127+0000 7f41cde8c740 1 -- 172.21.15.103:0/2031482395 wait complete. 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 -- start start 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 Processor -- start 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b8f10 0x5577b57c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b9830 0x5577b57c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5577b57cb150 con 0x5577b57b8f10 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5577b56d9b20 con 0x5577b57c85e0 2024-08-19T01:37:12.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5577b57cb2d0 con 0x5577b57b9830 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41cde8c740 10 client.0 _ll_register_callbacks cb 0x5577b56d7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41c982a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b8f10 0x5577b57c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41c982a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b8f10 0x5577b57c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52708/0 (socket says 172.21.15.103:52708) 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41c982a640 1 -- 172.21.15.103:0/635342095 learned_addr learned my addr 172.21.15.103:0/635342095 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.128+0000 7f41ca02b640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41c9029640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b9830 0x5577b57c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41c982a640 1 -- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b9830 msgr2=0x5577b57c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41c982a640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b9830 0x5577b57c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41c982a640 1 -- 172.21.15.103:0/635342095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 msgr2=0x5577b57c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41c982a640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41c982a640 1 -- 172.21.15.103:0/635342095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b54903c0 con 0x5577b57b8f10 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41ca02b640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41c982a640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b8f10 0x5577b57c5510 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005e10 tx=0x7f41c000a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e232d18a63edea0c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:12.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41ad7fa640 1 -- 172.21.15.103:0/635342095 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0027020 con 0x5577b57b8f10 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41ad7fa640 1 -- 172.21.15.103:0/635342095 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f41c00035f0 con 0x5577b57b8f10 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c00041c0 con 0x5577b57b8f10 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41ad7fa640 1 -- 172.21.15.103:0/635342095 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41c00037f0 con 0x5577b57b8f10 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41cde8c740 10 client.15744 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577b57d6dd0 con 0x5577b57b8f10 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.129+0000 7f41ad7fa640 1 -- 172.21.15.103:0/635342095 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41c00041c0 con 0x5577b57b8f10 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41ad7fa640 1 -- 172.21.15.103:0/635342095 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f41c0004ad0 con 0x5577b57b8f10 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41cde8c740 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41cde8c740 10 client.15744 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41cde8c740 20 client.15744 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41cde8c740 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b57da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41cde8c740 4 client.15744 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b572bfb0 con 0x5577b57d8180 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.130+0000 7f41c9029640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b57da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.131+0000 7f41c9029640 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b57da570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0ecb946673fb69d server_cookie=a3537267e9acbd23 in_seq=0 out_seq=0 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.131+0000 7f41ad7fa640 10 client.15744 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.131+0000 7f41ad7fa640 1 -- 172.21.15.103:0/635342095 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577b572bfb0 con 0x5577b57d8180 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.131+0000 7f41ad7fa640 10 client.15744 handle_command_reply: tid=0 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.131+0000 7f41cde8c740 1 client.15744 shutdown 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b8f10 msgr2=0x5577b57c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b8f10 0x5577b57c5510 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f41c0005e10 tx=0x7f41c000a700 comp rx=0 tx=0).stop 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 shutdown_connections 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b57da570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41ad7fa640 0 client.15744 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:12.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b9830 0x5577b57c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 --2- 172.21.15.103:0/635342095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b8f10 0x5577b57c5510 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 >> 172.21.15.103:0/635342095 conn(0x5577b57b5580 msgr2=0x5577b57bb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:12.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 shutdown_connections 2024-08-19T01:37:12.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 1 -- 172.21.15.103:0/635342095 wait complete. 2024-08-19T01:37:12.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.132+0000 7f41cde8c740 20 client.15744 trim_cache size 0 max 16384 2024-08-19T01:37:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.146+0000 7f41cde8c740 1 Processor -- start 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41cde8c740 1 -- start start 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57dfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41cde8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41aaff0 con 0x5577b57b9830 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41cde8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41acb60 con 0x5577b57c85e0 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41cde8c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41ab6c0 con 0x5577b57b8f10 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57e0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52710/0 (socket says 172.21.15.103:52710) 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 -- 172.21.15.103:0/1804936795 learned_addr learned my addr 172.21.15.103:0/1804936795 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c982a640 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57dfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41ca02b640 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 -- 172.21.15.103:0/1804936795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 msgr2=0x5577b57dfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57dfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 -- 172.21.15.103:0/1804936795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 msgr2=0x5577b57b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c9029640 1 -- 172.21.15.103:0/1804936795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b57c3a60 con 0x5577b57b9830 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.147+0000 7f41c982a640 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57dfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.148+0000 7f41c9029640 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57e0050 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009900 tx=0x7f41b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3272be7acca4b79a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.148+0000 7f41af7fe640 1 -- 172.21.15.103:0/1804936795 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4035040 con 0x5577b57b9830 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.148+0000 7f41af7fe640 1 -- 172.21.15.103:0/1804936795 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41b4058070 con 0x5577b57b9830 2024-08-19T01:37:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.148+0000 7f41af7fe640 1 -- 172.21.15.103:0/1804936795 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4053380 con 0x5577b57b9830 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.148+0000 7f41cde8c740 1 -- 172.21.15.103:0/1804936795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 msgr2=0x5577b57e0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.148+0000 7f41cde8c740 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57e0050 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009900 tx=0x7f41b4009260 comp rx=0 tx=0).stop 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 -- 172.21.15.103:0/1804936795 shutdown_connections 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b57b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57e0050 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 --2- 172.21.15.103:0/1804936795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57dfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 -- 172.21.15.103:0/1804936795 >> 172.21.15.103:0/1804936795 conn(0x5577b57b5580 msgr2=0x5577b57c3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 -- 172.21.15.103:0/1804936795 shutdown_connections 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 -- 172.21.15.103:0/1804936795 wait complete. 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 -- start start 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 Processor -- start 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b5840ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.149+0000 7f41cde8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41ac460 con 0x5577b57b9830 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41cde8c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41abd90 con 0x5577b57c85e0 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41cde8c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41a8fd0 con 0x5577b57b8f10 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41cde8c740 10 client.0 _ll_register_callbacks cb 0x7f41b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52714/0 (socket says 172.21.15.103:52714) 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 -- 172.21.15.103:0/3539165653 learned_addr learned my addr 172.21.15.103:0/3539165653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c982a640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41ca02b640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b5840ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 -- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 msgr2=0x5577b57cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 -- 172.21.15.103:0/3539165653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 msgr2=0x5577b5840ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b5840ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 -- 172.21.15.103:0/3539165653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b58411c0 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c982a640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41c9029640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57cc1e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f41b4006010 tx=0x7f41b402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b6b83a27183cdf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.150+0000 7f41ad7fa640 1 -- 172.21.15.103:0/3539165653 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4060070 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41ad7fa640 1 -- 172.21.15.103:0/3539165653 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f41b4053650 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b4058070 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41ad7fa640 1 -- 172.21.15.103:0/3539165653 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41b406c050 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577b58424a0 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41ad7fa640 1 -- 172.21.15.103:0/3539165653 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41b4058070 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41ad7fa640 1 -- 172.21.15.103:0/3539165653 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f41b4048040 con 0x5577b57b9830 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 10 client.15750 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 20 client.15750 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b5844ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 4 client.15750 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.151+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577b57264c0 con 0x5577b57d8180 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41c982a640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b5844ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41c982a640 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b5844ba0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64965a6f83bee227 server_cookie=98ddaac69e799adf in_seq=0 out_seq=0 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41ad7fa640 10 client.15750 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41ad7fa640 1 -- 172.21.15.103:0/3539165653 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577b57264c0 con 0x5577b57d8180 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41ad7fa640 10 client.15750 handle_command_reply: tid=0 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 client.15750 shutdown 2024-08-19T01:37:12.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 msgr2=0x5577b57cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57cc1e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f41b4006010 tx=0x7f41b402a420 comp rx=0 tx=0).stop 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 shutdown_connections 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5577b57d8180 0x5577b5844ba0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5577b57c85e0 0x5577b5840ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5577b57b9830 0x5577b57cc1e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41ad7fa640 0 client.15750 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 --2- 172.21.15.103:0/3539165653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5577b57b8f10 0x5577b57cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.152+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 >> 172.21.15.103:0/3539165653 conn(0x5577b57b5580 msgr2=0x5577b57b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 shutdown_connections 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cde8c740 1 -- 172.21.15.103:0/3539165653 wait complete. 2024-08-19T01:37:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cde8c740 20 client.15750 trim_cache size 0 max 16384 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f41b00a2b80 msgr2=0x7f41b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f41b00a2b80 0x7f41b00a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f41b402aa10 tx=0x7f41b4040040 comp rx=0 tx=0).stop 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41078c0 msgr2=0x7f41c419c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41078c0 0x7f41c419c9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00eaa0 tx=0x7f41bc00ef60 comp rx=0 tx=0).stop 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 shutdown_connections 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f41b00a2b80 0x7f41b00a5040 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c41078c0 0x7f41c419c9c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c4102260 0x7f41c4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 --2- 172.21.15.103:0/2238402490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c41018c0 0x7f41c41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:12.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 >> 172.21.15.103:0/2238402490 conn(0x7f41c40fd5e0 msgr2=0x7f41c40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:12.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 shutdown_connections 2024-08-19T01:37:12.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:12.153+0000 7f41cba8e640 1 -- 172.21.15.103:0/2238402490 wait complete. 2024-08-19T01:37:12.182 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:11 smithi190 ceph-mon[22032]: pgmap v381: 97 pgs: 97 active+clean; 698 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:37:12.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:11 smithi146 ceph-mon[22098]: pgmap v381: 97 pgs: 97 active+clean; 698 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:37:12.904 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:37:13.187 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 -- 172.21.15.103:0/3770105080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90101960 msgr2=0x7f0e90101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 --2- 172.21.15.103:0/3770105080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90101960 0x7f0e90101de0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009900 tx=0x7f0e7c02a410 comp rx=0 tx=0).stop 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 -- 172.21.15.103:0/3770105080 shutdown_connections 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 --2- 172.21.15.103:0/3770105080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e90102320 0x7f0e90108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 --2- 172.21.15.103:0/3770105080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90101960 0x7f0e90101de0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 --2- 172.21.15.103:0/3770105080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e90100770 0x7f0e90100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 -- 172.21.15.103:0/3770105080 >> 172.21.15.103:0/3770105080 conn(0x7f0e900fbf10 msgr2=0x7f0e900fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 -- 172.21.15.103:0/3770105080 shutdown_connections 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 -- 172.21.15.103:0/3770105080 wait complete. 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 Processor -- start 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.251+0000 7f0e96c79640 1 -- start start 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e96c79640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90100770 0x7f0e9006a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e96c79640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e90101960 0x7f0e9019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e96c79640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e90102320 0x7f0e9019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e96c79640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9010c280 con 0x7f0e90100770 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e96c79640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9010c100 con 0x7f0e90102320 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e96c79640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9010c400 con 0x7f0e90101960 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90100770 0x7f0e9006a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90100770 0x7f0e9006a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52722/0 (socket says 172.21.15.103:52722) 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 -- 172.21.15.103:0/2895777959 learned_addr learned my addr 172.21.15.103:0/2895777959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e90101960 0x7f0e9019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e95216640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e90102320 0x7f0e9019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 -- 172.21.15.103:0/2895777959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e90101960 msgr2=0x7f0e9019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e90101960 0x7f0e9019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 -- 172.21.15.103:0/2895777959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e90102320 msgr2=0x7f0e9019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e90102320 0x7f0e9019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 -- 172.21.15.103:0/2895777959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9019f6c0 con 0x7f0e90100770 2024-08-19T01:37:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e94a15640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90100770 0x7f0e9006a140 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400d770 tx=0x7f0e8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.252+0000 7f0e8dffb640 1 -- 172.21.15.103:0/2895777959 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84017070 con 0x7f0e90100770 2024-08-19T01:37:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.253+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9019f950 con 0x7f0e90100770 2024-08-19T01:37:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.253+0000 7f0e8dffb640 1 -- 172.21.15.103:0/2895777959 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e84009c30 con 0x7f0e90100770 2024-08-19T01:37:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.253+0000 7f0e8dffb640 1 -- 172.21.15.103:0/2895777959 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84003d00 con 0x7f0e90100770 2024-08-19T01:37:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.253+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9019fef0 con 0x7f0e90100770 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.255+0000 7f0e8dffb640 1 -- 172.21.15.103:0/2895777959 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0e84003ea0 con 0x7f0e90100770 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.255+0000 7f0e99a36740 1 Processor -- start 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.255+0000 7f0e99a36740 1 -- start start 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e99a36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fde772e9d0 con 0x55fde77d1050 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e99a36740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fde76d6820 con 0x55fde77cbfc0 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e99a36740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fde772a990 con 0x55fde77cb680 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77d3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52726/0 (socket says 172.21.15.103:52726) 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 -- 172.21.15.103:0/1557048388 learned_addr learned my addr 172.21.15.103:0/1557048388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e94a15640 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e95216640 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 -- 172.21.15.103:0/1557048388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 msgr2=0x55fde77cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 -- 172.21.15.103:0/1557048388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 msgr2=0x55fde77d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8ffff640 1 -- 172.21.15.103:0/1557048388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde7442210 con 0x55fde77d1050 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e8dffb640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0e800a2cb0 0x7f0e800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.256+0000 7f0e95216640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0e800a2cb0 0x7f0e800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.257+0000 7f0e8dffb640 1 -- 172.21.15.103:0/2895777959 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0e840ced60 con 0x7f0e90100770 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.257+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77d3510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02a930 tx=0x7f0e7c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6955ebd7ea8768bb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.257+0000 7f0e7affd640 1 -- 172.21.15.103:0/1557048388 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c03f070 con 0x55fde77d1050 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.257+0000 7f0e7affd640 1 -- 172.21.15.103:0/1557048388 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e7c031780 con 0x55fde77d1050 2024-08-19T01:37:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.257+0000 7f0e7affd640 1 -- 172.21.15.103:0/1557048388 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c035070 con 0x55fde77d1050 2024-08-19T01:37:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 -- 172.21.15.103:0/1557048388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 msgr2=0x55fde77d3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e95216640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0e800a2cb0 0x7f0e800a5170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e88008e40 tx=0x7f0e880085f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77d3510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02a930 tx=0x7f0e7c032cc0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 -- 172.21.15.103:0/1557048388 shutdown_connections 2024-08-19T01:37:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77d3510 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 --2- 172.21.15.103:0/1557048388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 -- 172.21.15.103:0/1557048388 >> 172.21.15.103:0/1557048388 conn(0x55fde77c6cf0 msgr2=0x55fde77c70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 -- 172.21.15.103:0/1557048388 shutdown_connections 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 -- 172.21.15.103:0/1557048388 wait complete. 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 -- start start 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.258+0000 7f0e99a36740 1 Processor -- start 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77cbfc0 0x55fde77d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77d1050 0x55fde77dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fde77d3fc0 con 0x55fde77cbfc0 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fde76ea520 con 0x55fde77d1050 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fde77d4140 con 0x55fde77cb680 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 10 client.0 _ll_register_callbacks cb 0x55fde7441a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77cbfc0 0x55fde77d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77cbfc0 0x55fde77d5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52742/0 (socket says 172.21.15.103:52742) 2024-08-19T01:37:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 -- 172.21.15.103:0/188010504 learned_addr learned my addr 172.21.15.103:0/188010504 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e94a15640 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e95216640 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77d1050 0x55fde77dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 -- 172.21.15.103:0/188010504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 msgr2=0x55fde77d57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77d57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 -- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77d1050 msgr2=0x55fde77dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77d1050 0x55fde77dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 -- 172.21.15.103:0/188010504 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde750e620 con 0x55fde77cbfc0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77cbfc0 0x55fde77d5ce0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0092a0 tx=0x7f0e7c032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318167b20778c588 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e6ffff640 1 -- 172.21.15.103:0/188010504 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c04a020 con 0x55fde77cbfc0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e6ffff640 1 -- 172.21.15.103:0/188010504 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0e7c003800 con 0x55fde77cbfc0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.259+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c031780 con 0x55fde77cbfc0 2024-08-19T01:37:13.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e6ffff640 1 -- 172.21.15.103:0/188010504 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e7c03f070 con 0x55fde77cbfc0 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fde77dfb10 con 0x55fde77cbfc0 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e6ffff640 1 -- 172.21.15.103:0/188010504 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e7c031780 con 0x55fde77cbfc0 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e6ffff640 1 -- 172.21.15.103:0/188010504 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f0e7c004bb0 con 0x55fde77cbfc0 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 20 client.15762 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde77e0e60 0x55fde77e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 4 client.15762 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fde773c100 con 0x55fde77e0e60 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.260+0000 7f0e94a15640 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde77e0e60 0x55fde77e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.261+0000 7f0e6ffff640 10 client.15762 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.261+0000 7f0e94a15640 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde77e0e60 0x55fde77e3250 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae01051fd1ed9aed server_cookie=6d2f0d2ea35dea5 in_seq=0 out_seq=0 2024-08-19T01:37:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.261+0000 7f0e6ffff640 1 -- 172.21.15.103:0/188010504 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fde773c100 con 0x55fde77e0e60 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.261+0000 7f0e6ffff640 10 client.15762 handle_command_reply: tid=0 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.261+0000 7f0e99a36740 1 client.15762 shutdown 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77cbfc0 msgr2=0x55fde77d5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77cbfc0 0x55fde77d5ce0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0092a0 tx=0x7f0e7c032ed0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 shutdown_connections 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde77e0e60 0x55fde77e3250 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77d1050 0x55fde77dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77cbfc0 0x55fde77d5ce0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 --2- 172.21.15.103:0/188010504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77d57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e6ffff640 0 client.15762 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 >> 172.21.15.103:0/188010504 conn(0x55fde77c6cf0 msgr2=0x55fde77c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 shutdown_connections 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 1 -- 172.21.15.103:0/188010504 wait complete. 2024-08-19T01:37:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.262+0000 7f0e99a36740 20 client.15762 trim_cache size 0 max 16384 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 Processor -- start 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 -- start start 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901a8ab0 con 0x55fde77d1050 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901aa620 con 0x55fde77cbfc0 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e99a36740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901a9180 con 0x55fde77cb680 2024-08-19T01:37:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e95216640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e95216640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77e7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52748/0 (socket says 172.21.15.103:52748) 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e95216640 1 -- 172.21.15.103:0/2446653983 learned_addr learned my addr 172.21.15.103:0/2446653983 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e94a15640 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e95216640 1 -- 172.21.15.103:0/2446653983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 msgr2=0x55fde77c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e95216640 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e95216640 1 -- 172.21.15.103:0/2446653983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 msgr2=0x55fde77dcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.279+0000 7f0e95216640 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77dcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.280+0000 7f0e95216640 1 -- 172.21.15.103:0/2446653983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde77e3880 con 0x55fde77d1050 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.280+0000 7f0e95216640 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77e7aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0e8801d4c0 tx=0x7f0e88014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a98ceefe2130b7f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.280+0000 7f0e7a7fc640 1 -- 172.21.15.103:0/2446653983 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8802b070 con 0x55fde77d1050 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.280+0000 7f0e7a7fc640 1 -- 172.21.15.103:0/2446653983 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e8801cac0 con 0x55fde77d1050 2024-08-19T01:37:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.280+0000 7f0e7a7fc640 1 -- 172.21.15.103:0/2446653983 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88026bb0 con 0x55fde77d1050 2024-08-19T01:37:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 -- 172.21.15.103:0/2446653983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 msgr2=0x55fde77e7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77e7aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0e8801d4c0 tx=0x7f0e88014000 comp rx=0 tx=0).stop 2024-08-19T01:37:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 -- 172.21.15.103:0/2446653983 shutdown_connections 2024-08-19T01:37:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77e7aa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cbfc0 0x55fde77dcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 --2- 172.21.15.103:0/2446653983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cb680 0x55fde77c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 -- 172.21.15.103:0/2446653983 >> 172.21.15.103:0/2446653983 conn(0x55fde77c6cf0 msgr2=0x55fde77c5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 -- 172.21.15.103:0/2446653983 shutdown_connections 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 -- 172.21.15.103:0/2446653983 wait complete. 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 -- start start 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.281+0000 7f0e99a36740 1 Processor -- start 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cb680 0x55fde77dac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cbfc0 0x55fde77db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e99a36740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e99a36740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901a9f20 con 0x55fde77d1050 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e99a36740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901a9850 con 0x55fde77cb680 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e99a36740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901a6a90 con 0x55fde77cbfc0 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e99a36740 10 client.0 _ll_register_callbacks cb 0x7f0e840d2710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e94a15640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cb680 0x55fde77dac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77dc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52756/0 (socket says 172.21.15.103:52756) 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 -- 172.21.15.103:0/3368163022 learned_addr learned my addr 172.21.15.103:0/3368163022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cbfc0 0x55fde77db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 -- 172.21.15.103:0/3368163022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cbfc0 msgr2=0x55fde77db160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cbfc0 0x55fde77db160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 -- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cb680 msgr2=0x55fde77dac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cb680 0x55fde77dac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 -- 172.21.15.103:0/3368163022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde77dc710 con 0x55fde77d1050 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e8ffff640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cbfc0 0x55fde77db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e94a15640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cb680 0x55fde77dac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e95216640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77dc100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0e88029040 tx=0x7f0e8801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f830369869e3a24c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.282+0000 7f0e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e6ffff640 1 -- 172.21.15.103:0/3368163022 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88036060 con 0x55fde77d1050 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e6ffff640 1 -- 172.21.15.103:0/3368163022 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0e88004c50 con 0x55fde77d1050 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e8801cac0 con 0x55fde77d1050 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e6ffff640 1 -- 172.21.15.103:0/3368163022 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e8802b070 con 0x55fde77d1050 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fde784c1d0 con 0x55fde77d1050 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e6ffff640 1 -- 172.21.15.103:0/3368163022 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e8801cac0 con 0x55fde77d1050 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e6ffff640 1 -- 172.21.15.103:0/3368163022 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f0e880054c0 con 0x55fde77d1050 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 20 client.15768 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde784cb60 0x55fde784ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 4 client.15768 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fde7736450 con 0x55fde784cb60 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.283+0000 7f0e94a15640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde784cb60 0x55fde784ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.284+0000 7f0e6ffff640 10 client.15768 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.284+0000 7f0e94a15640 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde784cb60 0x55fde784ef80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95856890fd14d1d4 server_cookie=3ef4bcafef4d5ec0 in_seq=0 out_seq=0 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.292+0000 7f0e6ffff640 1 -- 172.21.15.103:0/3368163022 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fde7736450 con 0x55fde784cb60 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e6ffff640 10 client.15768 handle_command_reply: tid=0 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 client.15768 shutdown 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 msgr2=0x55fde77dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77dc100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0e88029040 tx=0x7f0e8801c8c0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 shutdown_connections 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fde784cb60 0x55fde784ef80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde77d1050 0x55fde77dc100 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fde77cbfc0 0x55fde77db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e6ffff640 0 client.15768 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 --2- 172.21.15.103:0/3368163022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fde77cb680 0x55fde77dac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 >> 172.21.15.103:0/3368163022 conn(0x55fde77c6cf0 msgr2=0x55fde77c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 shutdown_connections 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 1 -- 172.21.15.103:0/3368163022 wait complete. 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.293+0000 7f0e99a36740 20 client.15768 trim_cache size 0 max 16384 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:37:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:37:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0e800a2cb0 msgr2=0x7f0e800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0e800a2cb0 0x7f0e800a5170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e88008e40 tx=0x7f0e880085f0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90100770 msgr2=0x7f0e9006a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90100770 0x7f0e9006a140 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400d770 tx=0x7f0e8400dc30 comp rx=0 tx=0).stop 2024-08-19T01:37:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 shutdown_connections 2024-08-19T01:37:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0e800a2cb0 0x7f0e800a5170 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e90102320 0x7f0e9019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e90101960 0x7f0e9019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 --2- 172.21.15.103:0/2895777959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e90100770 0x7f0e9006a140 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 >> 172.21.15.103:0/2895777959 conn(0x7f0e900fbf10 msgr2=0x7f0e90104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 shutdown_connections 2024-08-19T01:37:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:13.294+0000 7f0e96c79640 1 -- 172.21.15.103:0/2895777959 wait complete. 2024-08-19T01:37:13.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 -- 172.21.15.103:0/4055810444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4102ac0 msgr2=0x7f97f4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/4055810444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4102ac0 0x7f97f4102ea0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009930 tx=0x7f97e402a420 comp rx=0 tx=0).stop 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 -- 172.21.15.103:0/4055810444 shutdown_connections 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/4055810444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97f4107fc0 0x7f97f410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/4055810444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97f4103470 0x7f97f41078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/4055810444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4102ac0 0x7f97f4102ea0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 -- 172.21.15.103:0/4055810444 >> 172.21.15.103:0/4055810444 conn(0x7f97f40fc500 msgr2=0x7f97f40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 -- 172.21.15.103:0/4055810444 shutdown_connections 2024-08-19T01:37:13.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.736+0000 7f97f9c0a640 1 -- 172.21.15.103:0/4055810444 wait complete. 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.737+0000 7f97f9c0a640 1 Processor -- start 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.737+0000 7f97f9c0a640 1 -- start start 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f9c0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97f4102ac0 0x7f97f4100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f9c0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97f4103470 0x7f97f4100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f9c0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4107fc0 0x7f97f41011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f9c0a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4111cb0 con 0x7f97f4107fc0 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f9c0a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4111b30 con 0x7f97f4103470 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f9c0a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4111e30 con 0x7f97f4102ac0 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4107fc0 0x7f97f41011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97f4103470 0x7f97f4100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4107fc0 0x7f97f41011a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52778/0 (socket says 172.21.15.103:52778) 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97f4102ac0 0x7f97f4100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 -- 172.21.15.103:0/597385887 learned_addr learned my addr 172.21.15.103:0/597385887 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 -- 172.21.15.103:0/597385887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97f4102ac0 msgr2=0x7f97f4100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97f4102ac0 0x7f97f4100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 -- 172.21.15.103:0/597385887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97f4103470 msgr2=0x7f97f4100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97f4103470 0x7f97f4100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.738+0000 7f97f3fff640 1 -- 172.21.15.103:0/597385887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f41a6750 con 0x7f97f4107fc0 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.739+0000 7f97f3fff640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4107fc0 0x7f97f41011a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00eaa0 tx=0x7f97ec00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.739+0000 7f97f0ff9640 1 -- 172.21.15.103:0/597385887 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec004d20 con 0x7f97f4107fc0 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.739+0000 7f97f0ff9640 1 -- 172.21.15.103:0/597385887 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97ec005020 con 0x7f97f4107fc0 2024-08-19T01:37:13.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.739+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f41a6a40 con 0x7f97f4107fc0 2024-08-19T01:37:13.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.739+0000 7f97f0ff9640 1 -- 172.21.15.103:0/597385887 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0116b0 con 0x7f97f4107fc0 2024-08-19T01:37:13.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.739+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f41a6f30 con 0x7f97f4107fc0 2024-08-19T01:37:13.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 Processor -- start 2024-08-19T01:37:13.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97f0ff9640 1 -- 172.21.15.103:0/597385887 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f97ec0051d0 con 0x7f97f4107fc0 2024-08-19T01:37:13.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 -- start start 2024-08-19T01:37:13.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec3521f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec35229c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3522a1d0 0x55ec3522c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ec35184a40 con 0x55ec3522a1d0 2024-08-19T01:37:13.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ec3511c9d0 con 0x55ec3521f910 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97fc008740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ec35180a00 con 0x55ec3521eff0 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3522a1d0 0x55ec3522c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3522a1d0 0x55ec3522c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52782/0 (socket says 172.21.15.103:52782) 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.742+0000 7f97f2ffd640 1 -- 172.21.15.103:0/142391748 learned_addr learned my addr 172.21.15.103:0/142391748 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f37fe640 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec35229c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f0ff9640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97dc0a2b80 0x7f97dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec3521f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f3fff640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97dc0a2b80 0x7f97dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f0ff9640 1 -- 172.21.15.103:0/597385887 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f97ec0cf1f0 con 0x7f97f4107fc0 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f2ffd640 1 -- 172.21.15.103:0/142391748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 msgr2=0x55ec3521f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec3521f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f2ffd640 1 -- 172.21.15.103:0/142391748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 msgr2=0x55ec35229c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec35229c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f2ffd640 1 -- 172.21.15.103:0/142391748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec35123650 con 0x55ec3522a1d0 2024-08-19T01:37:13.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3522a1d0 0x55ec3522c5c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f97e8017700 tx=0x7f97e80094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3663a8e8b879415 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97d9ffb640 1 -- 172.21.15.103:0/142391748 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8015760 con 0x55ec3522a1d0 2024-08-19T01:37:13.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97d9ffb640 1 -- 172.21.15.103:0/142391748 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e801d070 con 0x55ec3522a1d0 2024-08-19T01:37:13.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97f3fff640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97dc0a2b80 0x7f97dc0a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f97ec09b780 tx=0x7f97ec01d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.743+0000 7f97d9ffb640 1 -- 172.21.15.103:0/142391748 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8018970 con 0x55ec3522a1d0 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 -- 172.21.15.103:0/142391748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3522a1d0 msgr2=0x55ec3522c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3522a1d0 0x55ec3522c5c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f97e8017700 tx=0x7f97e80094d0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 -- 172.21.15.103:0/142391748 shutdown_connections 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3522a1d0 0x55ec3522c5c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec35229c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 --2- 172.21.15.103:0/142391748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec3521f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 -- 172.21.15.103:0/142391748 >> 172.21.15.103:0/142391748 conn(0x55ec3521b580 msgr2=0x55ec3521b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 -- 172.21.15.103:0/142391748 shutdown_connections 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.744+0000 7f97fc008740 1 -- 172.21.15.103:0/142391748 wait complete. 2024-08-19T01:37:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 -- start start 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 Processor -- start 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec35227070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec352275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 0x55ec35227af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ec3522cda0 con 0x55ec3521eff0 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ec3513fb20 con 0x55ec3521f910 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ec3522cf20 con 0x55ec3522a1d0 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97fc008740 10 client.0 _ll_register_callbacks cb 0x55ec3513dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec35227070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec35227070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52798/0 (socket says 172.21.15.103:52798) 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.745+0000 7f97f2ffd640 1 -- 172.21.15.103:0/100887648 learned_addr learned my addr 172.21.15.103:0/100887648 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 0x55ec35227af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f37fe640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec352275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f2ffd640 1 -- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 msgr2=0x55ec35227af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 0x55ec35227af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f2ffd640 1 -- 172.21.15.103:0/100887648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 msgr2=0x55ec352275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec352275b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f2ffd640 1 -- 172.21.15.103:0/100887648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec34ef63c0 con 0x55ec3521eff0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f37fe640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec352275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec35227070 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009ad0 tx=0x7f97e8015fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770255e598a30728 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97d2ffd640 1 -- 172.21.15.103:0/100887648 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8017e90 con 0x55ec3521eff0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97d2ffd640 1 -- 172.21.15.103:0/100887648 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f97e8015ba0 con 0x55ec3521eff0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e801d070 con 0x55ec3521eff0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97d2ffd640 1 -- 172.21.15.103:0/100887648 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97e8015da0 con 0x55ec3521eff0 2024-08-19T01:37:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97fc008740 10 client.15780 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.746+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec352389a0 con 0x55ec3521eff0 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97d2ffd640 1 -- 172.21.15.103:0/100887648 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e801d070 con 0x55ec3521eff0 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97d2ffd640 1 -- 172.21.15.103:0/100887648 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f97e80316a0 con 0x55ec3521eff0 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97fc008740 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97fc008740 10 client.15780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97fc008740 20 client.15780 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97fc008740 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec35239cf0 0x55ec3523c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97fc008740 4 client.15780 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec35191fb0 con 0x55ec35239cf0 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.747+0000 7f97f37fe640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec35239cf0 0x55ec3523c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.748+0000 7f97d2ffd640 10 client.15780 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:37:13.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.748+0000 7f97f37fe640 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec35239cf0 0x55ec3523c0e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f805e954b81fc21e server_cookie=b9de9d7824f0fa2e in_seq=0 out_seq=0 2024-08-19T01:37:13.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.748+0000 7f97d2ffd640 1 -- 172.21.15.103:0/100887648 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec35191fb0 con 0x55ec35239cf0 2024-08-19T01:37:13.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97d2ffd640 10 client.15780 handle_command_reply: tid=0 2024-08-19T01:37:13.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 client.15780 shutdown 2024-08-19T01:37:13.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 msgr2=0x55ec35227070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec35227070 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009ad0 tx=0x7f97e8015fa0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 shutdown_connections 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec35239cf0 0x55ec3523c0e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 0x55ec35227af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3521f910 0x55ec352275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 --2- 172.21.15.103:0/100887648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec35227070 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 >> 172.21.15.103:0/100887648 conn(0x55ec3521b580 msgr2=0x55ec35221260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97d2ffd640 0 client.15780 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 shutdown_connections 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 1 -- 172.21.15.103:0/100887648 wait complete. 2024-08-19T01:37:13.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.749+0000 7f97fc008740 20 client.15780 trim_cache size 0 max 16384 2024-08-19T01:37:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.766+0000 7f97fc008740 1 Processor -- start 2024-08-19T01:37:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.766+0000 7f97fc008740 1 -- start start 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97fc008740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec352262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97fc008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521f910 0x55ec352287c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97fc008740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3522a1d0 0x55ec35228d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97fc008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41af3c0 con 0x55ec3521f910 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97fc008740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41b0f30 con 0x55ec3522a1d0 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97fc008740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41afa90 con 0x55ec3521eff0 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521f910 0x55ec352287c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521f910 0x55ec352287c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52806/0 (socket says 172.21.15.103:52806) 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 -- 172.21.15.103:0/464899884 learned_addr learned my addr 172.21.15.103:0/464899884 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec352262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3522a1d0 0x55ec35228d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 -- 172.21.15.103:0/464899884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 msgr2=0x55ec352262f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec352262f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 -- 172.21.15.103:0/464899884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3522a1d0 msgr2=0x55ec35228d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3522a1d0 0x55ec35228d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f37fe640 1 -- 172.21.15.103:0/464899884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3521b3b0 con 0x55ec3521f910 2024-08-19T01:37:13.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.767+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec352262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:13.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.768+0000 7f97f37fe640 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521f910 0x55ec352287c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009900 tx=0x7f97e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c2f2ef84ad7ad9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.768+0000 7f97d97fa640 1 -- 172.21.15.103:0/464899884 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4049070 con 0x55ec3521f910 2024-08-19T01:37:13.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.768+0000 7f97d97fa640 1 -- 172.21.15.103:0/464899884 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e4031860 con 0x55ec3521f910 2024-08-19T01:37:13.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.768+0000 7f97d97fa640 1 -- 172.21.15.103:0/464899884 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4035040 con 0x55ec3521f910 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.769+0000 7f97fc008740 1 -- 172.21.15.103:0/464899884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521f910 msgr2=0x55ec352287c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.769+0000 7f97fc008740 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521f910 0x55ec352287c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009900 tx=0x7f97e4000bc0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 -- 172.21.15.103:0/464899884 shutdown_connections 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3522a1d0 0x55ec35228d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521f910 0x55ec352287c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 --2- 172.21.15.103:0/464899884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3521eff0 0x55ec352262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 -- 172.21.15.103:0/464899884 >> 172.21.15.103:0/464899884 conn(0x55ec3521b580 msgr2=0x55ec3521b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 -- 172.21.15.103:0/464899884 shutdown_connections 2024-08-19T01:37:13.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 -- 172.21.15.103:0/464899884 wait complete. 2024-08-19T01:37:13.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.770+0000 7f97fc008740 1 -- start start 2024-08-19T01:37:13.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 1 Processor -- start 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec3523f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 0x55ec3523bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3523c240 0x55ec3523c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41b0830 con 0x55ec3521eff0 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41b0160 con 0x55ec3523c240 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f97f41ad3a0 con 0x55ec3522a1d0 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.771+0000 7f97fc008740 10 client.0 _ll_register_callbacks cb 0x7f97e4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec3523f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec3523f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52812/0 (socket says 172.21.15.103:52812) 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 -- 172.21.15.103:0/272822617 learned_addr learned my addr 172.21.15.103:0/272822617 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3523c240 0x55ec3523c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 -- 172.21.15.103:0/272822617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 msgr2=0x55ec3523bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 0x55ec3523bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 -- 172.21.15.103:0/272822617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3523c240 msgr2=0x55ec3523c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3523c240 0x55ec3523c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 -- 172.21.15.103:0/272822617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec352a27d0 con 0x55ec3521eff0 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97f2ffd640 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec3523f510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f97e8015730 tx=0x7f97e8027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4798f895f9aa727 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.772+0000 7f97d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97d2ffd640 1 -- 172.21.15.103:0/272822617 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8009800 con 0x55ec3521eff0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97d2ffd640 1 -- 172.21.15.103:0/272822617 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f97e8015ba0 con 0x55ec3521eff0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e4031860 con 0x55ec3521eff0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97d2ffd640 1 -- 172.21.15.103:0/272822617 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97e80184d0 con 0x55ec3521eff0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97fc008740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec352a3a10 con 0x55ec3521eff0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97d2ffd640 1 -- 172.21.15.103:0/272822617 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e4031860 con 0x55ec3521eff0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.773+0000 7f97d2ffd640 1 -- 172.21.15.103:0/272822617 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f97e8023690 con 0x55ec3521eff0 2024-08-19T01:37:13.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97fc008740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97fc008740 10 client.15786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97fc008740 20 client.15786 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97fc008740 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec352a4010 0x55ec352a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97fc008740 4 client.15786 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec3518c4c0 con 0x55ec352a4010 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97f37fe640 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec352a4010 0x55ec352a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97f37fe640 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec352a4010 0x55ec352a6430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fc65fc4125a5315 server_cookie=eff451e2e28a9ce3 in_seq=0 out_seq=0 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.774+0000 7f97d2ffd640 10 client.15786 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97d2ffd640 1 -- 172.21.15.103:0/272822617 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec3518c4c0 con 0x55ec352a4010 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97d2ffd640 10 client.15786 handle_command_reply: tid=0 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 client.15786 shutdown 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 msgr2=0x55ec3523f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec3523f510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f97e8015730 tx=0x7f97e8027040 comp rx=0 tx=0).stop 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 shutdown_connections 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55ec352a4010 0x55ec352a6430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ec3523c240 0x55ec3523c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ec3522a1d0 0x55ec3523bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 --2- 172.21.15.103:0/272822617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec3521eff0 0x55ec3523f510 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 >> 172.21.15.103:0/272822617 conn(0x55ec3521b580 msgr2=0x55ec352351a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97d2ffd640 0 client.15786 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 shutdown_connections 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 1 -- 172.21.15.103:0/272822617 wait complete. 2024-08-19T01:37:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.775+0000 7f97fc008740 20 client.15786 trim_cache size 0 max 16384 2024-08-19T01:37:13.780 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:37:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97dc0a2b80 msgr2=0x7f97dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97dc0a2b80 0x7f97dc0a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f97ec09b780 tx=0x7f97ec01d000 comp rx=0 tx=0).stop 2024-08-19T01:37:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4107fc0 msgr2=0x7f97f41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4107fc0 0x7f97f41011a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00eaa0 tx=0x7f97ec00eea0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 shutdown_connections 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97dc0a2b80 0x7f97dc0a5040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97f4107fc0 0x7f97f41011a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97f4103470 0x7f97f4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 --2- 172.21.15.103:0/597385887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97f4102ac0 0x7f97f4100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 >> 172.21.15.103:0/597385887 conn(0x7f97f40fc500 msgr2=0x7f97f4105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 shutdown_connections 2024-08-19T01:37:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:13.776+0000 7f97f9c0a640 1 -- 172.21.15.103:0/597385887 wait complete. 2024-08-19T01:37:14.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:13 smithi146 ceph-mon[22098]: pgmap v382: 97 pgs: 97 active+clean; 798 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-08-19T01:37:14.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:13 smithi103 ceph-mon[18436]: pgmap v382: 97 pgs: 97 active+clean; 798 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-08-19T01:37:14.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:13 smithi190 ceph-mon[22032]: pgmap v382: 97 pgs: 97 active+clean; 798 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-08-19T01:37:14.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:37:14.976 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:15.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.488+0000 7f5b517c6640 1 -- 172.21.15.103:0/1796231476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 msgr2=0x7f5b4c10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.488+0000 7f5b517c6640 1 --2- 172.21.15.103:0/1796231476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c10e970 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009cd0 tx=0x7f5b4402a910 comp rx=0 tx=0).stop 2024-08-19T01:37:15.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.488+0000 7f5b517c6640 1 -- 172.21.15.103:0/1796231476 shutdown_connections 2024-08-19T01:37:15.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.488+0000 7f5b517c6640 1 --2- 172.21.15.103:0/1796231476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c10e970 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.488+0000 7f5b517c6640 1 --2- 172.21.15.103:0/1796231476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c103410 0x7f5b4c1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.488+0000 7f5b517c6640 1 --2- 172.21.15.103:0/1796231476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b4c102a60 0x7f5b4c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.488+0000 7f5b517c6640 1 -- 172.21.15.103:0/1796231476 >> 172.21.15.103:0/1796231476 conn(0x7f5b4c0fc500 msgr2=0x7f5b4c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.489+0000 7f5b517c6640 1 -- 172.21.15.103:0/1796231476 shutdown_connections 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.489+0000 7f5b517c6640 1 -- 172.21.15.103:0/1796231476 wait complete. 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.489+0000 7f5b517c6640 1 Processor -- start 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.489+0000 7f5b517c6640 1 -- start start 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b517c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c102a60 0x7f5b4c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b517c6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b4c103410 0x7f5b4c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b517c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b517c6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111cc0 con 0x7f5b4c107fd0 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b517c6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111b40 con 0x7f5b4c103410 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b517c6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c111e40 con 0x7f5b4c102a60 2024-08-19T01:37:15.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52824/0 (socket says 172.21.15.103:52824) 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3015571223 learned_addr learned my addr 172.21.15.103:0/3015571223 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b4c103410 0x7f5b4c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c102a60 0x7f5b4c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3015571223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c102a60 msgr2=0x7f5b4c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c102a60 0x7f5b4c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3015571223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b4c103410 msgr2=0x7f5b4c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b4c103410 0x7f5b4c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3015571223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c19d1b0 con 0x7f5b4c107fd0 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.490+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c19cad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009df0 tx=0x7f5b44003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.491+0000 7f5b33fff640 1 -- 172.21.15.103:0/3015571223 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44004040 con 0x7f5b4c107fd0 2024-08-19T01:37:15.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.491+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c19d440 con 0x7f5b4c107fd0 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.491+0000 7f5b33fff640 1 -- 172.21.15.103:0/3015571223 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b44004690 con 0x7f5b4c107fd0 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.491+0000 7f5b33fff640 1 -- 172.21.15.103:0/3015571223 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44030dc0 con 0x7f5b4c107fd0 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.491+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1005f0 con 0x7f5b4c107fd0 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.493+0000 7f5b33fff640 1 -- 172.21.15.103:0/3015571223 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5b440305d0 con 0x7f5b4c107fd0 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 Processor -- start 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 -- start start 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21ea9e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b33fff640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b340a2b80 0x7f5b340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eaa8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eaab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b21ea03a40 con 0x55b21ea9e910 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b21e99b9d0 con 0x55b21eaa91d0 2024-08-19T01:37:15.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b21e9ffa00 con 0x55b21ea9dff0 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b33fff640 1 -- 172.21.15.103:0/3015571223 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5b440f05d0 con 0x7f5b4c107fd0 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b4affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21ea9e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b4a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eaa8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eaab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b4a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eaa8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52826/0 (socket says 172.21.15.103:52826) 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/302458942 learned_addr learned my addr 172.21.15.103:0/302458942 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b4affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21ea9e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:37076/0 (socket says 172.21.15.103:37076) 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.494+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b340a2b80 0x7f5b340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/302458942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 msgr2=0x55b21ea9e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21ea9e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/302458942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 msgr2=0x55b21eaab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eaab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/302458942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21e9a2650 con 0x55b21ea9e910 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eaab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4affd640 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21ea9e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eaa8c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5b40006aa0 tx=0x7f5b40017e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40f3a9f3769046f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b340a2b80 0x7f5b340a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5b40015790 tx=0x7f5b4000b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b317fa640 1 -- 172.21.15.103:0/302458942 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b400097a0 con 0x55b21ea9e910 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b317fa640 1 -- 172.21.15.103:0/302458942 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b40025070 con 0x55b21ea9e910 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.495+0000 7f5b317fa640 1 -- 172.21.15.103:0/302458942 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40020b90 con 0x55b21ea9e910 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 -- 172.21.15.103:0/302458942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 msgr2=0x55b21eaa8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eaa8c90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5b40006aa0 tx=0x7f5b40017e10 comp rx=0 tx=0).stop 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 -- 172.21.15.103:0/302458942 shutdown_connections 2024-08-19T01:37:15.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eaab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eaa8c90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/302458942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21ea9e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 -- 172.21.15.103:0/302458942 >> 172.21.15.103:0/302458942 conn(0x55b21ea9a580 msgr2=0x55b21ea9a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 -- 172.21.15.103:0/302458942 shutdown_connections 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 -- 172.21.15.103:0/302458942 wait complete. 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 -- start start 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.496+0000 7f5b53bc4740 1 Processor -- start 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaa6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eaa6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b21eaabda0 con 0x55b21eaa91d0 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b21e9beb20 con 0x55b21ea9e910 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b21eaabf20 con 0x55b21ea9dff0 2024-08-19T01:37:15.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b53bc4740 10 client.0 _ll_register_callbacks cb 0x55b21e9bcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eaa6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eaa6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52830/0 (socket says 172.21.15.103:52830) 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3360969855 learned_addr learned my addr 172.21.15.103:0/3360969855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaa6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 msgr2=0x55b21eaa60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3360969855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 msgr2=0x55b21eaa6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaa6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3360969855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21e7753c0 con 0x55b21eaa91d0 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.497+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eaa6b50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5b440dd0d0 tx=0x7f5b440d6300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cc53e225fd8003 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:15.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3360969855 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440bd560 con 0x55b21eaa91d0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3360969855 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5b440b94d0 con 0x55b21eaa91d0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b40025070 con 0x55b21eaa91d0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3360969855 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b440bce60 con 0x55b21eaa91d0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 10 client.15798 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b21eab79d0 con 0x55b21eaa91d0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3360969855 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b40025070 con 0x55b21eaa91d0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3360969855 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f5b440d6420 con 0x55b21eaa91d0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 20 client.15798 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eabb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 4 client.15798 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.498+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b21ea10fb0 con 0x55b21eab8fa0 2024-08-19T01:37:15.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.499+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eabb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.499+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eabb390 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bddb6f4c2b65218d server_cookie=f2b7d4a1baf627ef in_seq=0 out_seq=0 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.499+0000 7f5b2eff5640 10 client.15798 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3360969855 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b21ea10fb0 con 0x55b21eab8fa0 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b2eff5640 10 client.15798 handle_command_reply: tid=0 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 client.15798 shutdown 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 msgr2=0x55b21eaa6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eaa6b50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5b440dd0d0 tx=0x7f5b440d6300 comp rx=0 tx=0).stop 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 shutdown_connections 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eabb390 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eaa6b50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b2eff5640 0 client.15798 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaa6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3360969855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 >> 172.21.15.103:0/3360969855 conn(0x55b21ea9a580 msgr2=0x55b21eaa0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 shutdown_connections 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3360969855 wait complete. 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.500+0000 7f5b53bc4740 20 client.15798 trim_cache size 0 max 16384 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 Processor -- start 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 -- start start 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eac0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eac1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1aae50 con 0x55b21ea9e910 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1ac9c0 con 0x55b21eaa91d0 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1ab520 con 0x55b21ea9dff0 2024-08-19T01:37:15.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b4a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eac0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.550 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:37:15.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.513+0000 7f5b4a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eac0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52836/0 (socket says 172.21.15.103:52836) 2024-08-19T01:37:15.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/1001770893 learned_addr learned my addr 172.21.15.103:0/1001770893 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:15.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eac1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4affd640 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/1001770893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 msgr2=0x55b21eaa0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/1001770893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 msgr2=0x55b21eac1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eac1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4a7fc640 1 -- 172.21.15.103:0/1001770893 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21ea98a90 con 0x55b21ea9e910 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4affd640 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eac0b40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55b21ea9b3e0 tx=0x7f5b40009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc8636c5acdf72d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b30ff9640 1 -- 172.21.15.103:0/1001770893 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40018a60 con 0x55b21ea9e910 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b30ff9640 1 -- 172.21.15.103:0/1001770893 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b40025050 con 0x55b21ea9e910 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.514+0000 7f5b30ff9640 1 -- 172.21.15.103:0/1001770893 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40006390 con 0x55b21ea9e910 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 -- 172.21.15.103:0/1001770893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 msgr2=0x55b21eac0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eac0b40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55b21ea9b3e0 tx=0x7f5b40009de0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 -- 172.21.15.103:0/1001770893 shutdown_connections 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21eaa91d0 0x55b21eac1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21ea9e910 0x55b21eac0b40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/1001770893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaa0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 -- 172.21.15.103:0/1001770893 >> 172.21.15.103:0/1001770893 conn(0x55b21ea9a580 msgr2=0x55b21eaa13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 -- 172.21.15.103:0/1001770893 shutdown_connections 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 -- 172.21.15.103:0/1001770893 wait complete. 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.515+0000 7f5b53bc4740 1 -- start start 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 1 Processor -- start 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaacdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eb1bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1ac2c0 con 0x55b21eaa91d0 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1abbf0 con 0x55b21ea9e910 2024-08-19T01:37:15.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1a8e50 con 0x55b21ea9dff0 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b53bc4740 10 client.0 _ll_register_callbacks cb 0x7f5b44009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eb1bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eb1bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52842/0 (socket says 172.21.15.103:52842) 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4b7fe640 1 -- 172.21.15.103:0/3420607617 learned_addr learned my addr 172.21.15.103:0/3420607617 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaacdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4affd640 1 -- 172.21.15.103:0/3420607617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 msgr2=0x55b21eaacdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaacdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4affd640 1 -- 172.21.15.103:0/3420607617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 msgr2=0x55b21eb1bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eb1bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4affd640 1 -- 172.21.15.103:0/3420607617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21eb1c170 con 0x55b21ea9dff0 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaacdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:15.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.516+0000 7f5b4b7fe640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eb1bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b4affd640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaac8b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00e9d0 tx=0x7f5b3c004eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83aeaf04f4107910 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3420607617 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c005700 con 0x55b21ea9dff0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b40025050 con 0x55b21ea9dff0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3420607617 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5b3c00c640 con 0x55b21ea9dff0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3420607617 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b3c00c840 con 0x55b21ea9dff0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b53bc4740 10 client.25297 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b21eb23540 con 0x55b21ea9dff0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.517+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3420607617 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b40025050 con 0x55b21ea9dff0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3420607617 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f5b3c016b80 con 0x55b21ea9dff0 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b53bc4740 10 client.25297 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b53bc4740 10 client.25297 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:37:15.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b53bc4740 20 client.25297 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eb25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b53bc4740 4 client.25297 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b21ea0b4c0 con 0x55b21eab8fa0 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eb25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b4a7fc640 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eb25d00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab0b35632d91d441 server_cookie=bd6760eaa67d97c in_seq=0 out_seq=0 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.518+0000 7f5b2eff5640 10 client.25297 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b2eff5640 1 -- 172.21.15.103:0/3420607617 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b21ea0b4c0 con 0x55b21eab8fa0 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b2eff5640 10 client.25297 handle_command_reply: tid=0 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 client.25297 shutdown 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 msgr2=0x55b21eaac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaac8b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00e9d0 tx=0x7f5b3c004eb0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 shutdown_connections 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55b21eab8fa0 0x55b21eb25d00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b21eaa91d0 0x55b21eb1bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b21ea9e910 0x55b21eaacdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 --2- 172.21.15.103:0/3420607617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b21ea9dff0 0x55b21eaac8b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 >> 172.21.15.103:0/3420607617 conn(0x55b21ea9a580 msgr2=0x55b21eaa6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b2eff5640 0 client.25297 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 shutdown_connections 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 1 -- 172.21.15.103:0/3420607617 wait complete. 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.519+0000 7f5b53bc4740 20 client.25297 trim_cache size 0 max 16384 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b340a2b80 msgr2=0x7f5b340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b340a2b80 0x7f5b340a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5b40015790 tx=0x7f5b4000b000 comp rx=0 tx=0).stop 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 msgr2=0x7f5b4c19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c19cad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009df0 tx=0x7f5b44003ba0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 shutdown_connections 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b340a2b80 0x7f5b340a5040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b4c107fd0 0x7f5b4c19cad0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b4c103410 0x7f5b4c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 --2- 172.21.15.103:0/3015571223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b4c102a60 0x7f5b4c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:15.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 >> 172.21.15.103:0/3015571223 conn(0x7f5b4c0fc500 msgr2=0x7f5b4c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:15.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 shutdown_connections 2024-08-19T01:37:15.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:15.520+0000 7f5b517c6640 1 -- 172.21.15.103:0/3015571223 wait complete. 2024-08-19T01:37:16.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:15 smithi146 ceph-mon[22098]: pgmap v383: 97 pgs: 97 active+clean; 826 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:37:16.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:15 smithi103 ceph-mon[18436]: pgmap v383: 97 pgs: 97 active+clean; 826 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:37:16.316 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:37:16.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:15 smithi190 ceph-mon[22032]: pgmap v383: 97 pgs: 97 active+clean; 826 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:37:16.589 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:17.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 -- 172.21.15.103:0/4273635449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d01018a0 msgr2=0x7f96d0101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 --2- 172.21.15.103:0/4273635449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d01018a0 0x7f96d0101d00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009930 tx=0x7f96c402a420 comp rx=0 tx=0).stop 2024-08-19T01:37:17.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 -- 172.21.15.103:0/4273635449 shutdown_connections 2024-08-19T01:37:17.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 --2- 172.21.15.103:0/4273635449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d0102240 0x7f96d010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 --2- 172.21.15.103:0/4273635449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d01018a0 0x7f96d0101d00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 --2- 172.21.15.103:0/4273635449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d01078a0 0x7f96d0107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 -- 172.21.15.103:0/4273635449 >> 172.21.15.103:0/4273635449 conn(0x7f96d00fd5c0 msgr2=0x7f96d00ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 -- 172.21.15.103:0/4273635449 shutdown_connections 2024-08-19T01:37:17.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.137+0000 7f96d4d05640 1 -- 172.21.15.103:0/4273635449 wait complete. 2024-08-19T01:37:17.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.138+0000 7f96d4d05640 1 Processor -- start 2024-08-19T01:37:17.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.138+0000 7f96d4d05640 1 -- start start 2024-08-19T01:37:17.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.138+0000 7f96d4d05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d01018a0 0x7f96d01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96d4d05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d0102240 0x7f96d01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96d4d05640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d01078a0 0x7f96d019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96d4d05640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0111b90 con 0x7f96d0102240 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96d4d05640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0111a10 con 0x7f96d01078a0 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96d4d05640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0111d10 con 0x7f96d01018a0 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d0102240 0x7f96d01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d0102240 0x7f96d01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52866/0 (socket says 172.21.15.103:52866) 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 -- 172.21.15.103:0/2402915072 learned_addr learned my addr 172.21.15.103:0/2402915072 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96ce59c640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d01018a0 0x7f96d01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96ced9d640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d01078a0 0x7f96d019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 -- 172.21.15.103:0/2402915072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d01018a0 msgr2=0x7f96d01981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d01018a0 0x7f96d01981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 -- 172.21.15.103:0/2402915072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d01078a0 msgr2=0x7f96d019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d01078a0 0x7f96d019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 -- 172.21.15.103:0/2402915072 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d019d160 con 0x7f96d0102240 2024-08-19T01:37:17.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96ce59c640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d01018a0 0x7f96d01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:17.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d0102240 0x7f96d01986f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f96c4005940 tx=0x7f96c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.139+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2402915072 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c403f030 con 0x7f96d0102240 2024-08-19T01:37:17.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.140+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d019d3f0 con 0x7f96d0102240 2024-08-19T01:37:17.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.140+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2402915072 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c4003cb0 con 0x7f96d0102240 2024-08-19T01:37:17.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.140+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2402915072 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4035070 con 0x7f96d0102240 2024-08-19T01:37:17.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.140+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d01a4d30 con 0x7f96d0102240 2024-08-19T01:37:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.142+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2402915072 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f96c4042030 con 0x7f96d0102240 2024-08-19T01:37:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.142+0000 7f96d7103740 1 Processor -- start 2024-08-19T01:37:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96d7103740 1 -- start start 2024-08-19T01:37:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96d7103740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96d7103740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235215c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96d7103740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c2352185c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96d7103740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c235170a40 con 0x55c23520aff0 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96d7103740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c2351089d0 con 0x55c23520b910 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96d7103740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c23516ca00 con 0x55c2352161d0 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52882/0 (socket says 172.21.15.103:52882) 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96ced9d640 1 -- 172.21.15.103:0/980203352 learned_addr learned my addr 172.21.15.103:0/980203352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c2352185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.143+0000 7f96bf7fe640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96b80a2c00 0x7f96b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ced9d640 1 -- 172.21.15.103:0/980203352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 msgr2=0x55c2352185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ced9d640 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c2352185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ce59c640 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235215c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ced9d640 1 -- 172.21.15.103:0/980203352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 msgr2=0x55c235215c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ced9d640 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235215c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ced9d640 1 -- 172.21.15.103:0/980203352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23510f650 con 0x55c23520aff0 2024-08-19T01:37:17.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ce59c640 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235215c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96bf7fe640 1 -- 172.21.15.103:0/2402915072 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f96c40ef2a0 con 0x7f96d0102240 2024-08-19T01:37:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ced9d640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96b80a2c00 0x7f96b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96ced9d640 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520b3d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c4a0 tx=0x7f96c800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11ef1b748a7f82 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96bcff9640 1 -- 172.21.15.103:0/980203352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8018640 con 0x55c23520aff0 2024-08-19T01:37:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96bcff9640 1 -- 172.21.15.103:0/980203352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c801e070 con 0x55c23520aff0 2024-08-19T01:37:17.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.144+0000 7f96bcff9640 1 -- 172.21.15.103:0/980203352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c800a8a0 con 0x55c23520aff0 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 -- 172.21.15.103:0/980203352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 msgr2=0x55c23520b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96ced9d640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96b80a2c00 0x7f96b80a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96c801d000 tx=0x7f96c8017000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520b3d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c4a0 tx=0x7f96c800c960 comp rx=0 tx=0).stop 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 -- 172.21.15.103:0/980203352 shutdown_connections 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c2352185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235215c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 --2- 172.21.15.103:0/980203352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520b3d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 -- 172.21.15.103:0/980203352 >> 172.21.15.103:0/980203352 conn(0x55c235207580 msgr2=0x55c235207960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 -- 172.21.15.103:0/980203352 shutdown_connections 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.145+0000 7f96d7103740 1 -- 172.21.15.103:0/980203352 wait complete. 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 -- start start 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 Processor -- start 2024-08-19T01:37:17.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c23520aff0 0x55c235213080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c2352135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2352161d0 0x55c235213b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c235218da0 con 0x55c2352161d0 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c23512bb20 con 0x55c23520b910 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c235218f20 con 0x55c23520aff0 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96d7103740 10 client.0 _ll_register_callbacks cb 0x55c235129bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96ce59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c23520aff0 0x55c235213080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96ce59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c23520aff0 0x55c235213080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:37144/0 (socket says 172.21.15.103:37144) 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96ce59c640 1 -- 172.21.15.103:0/3922561326 learned_addr learned my addr 172.21.15.103:0/3922561326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96ced9d640 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2352161d0 0x55c235213b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.146+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c2352135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96ce59c640 1 -- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 msgr2=0x55c2352135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96ce59c640 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c2352135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96ce59c640 1 -- 172.21.15.103:0/3922561326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2352161d0 msgr2=0x55c235213b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96ce59c640 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2352161d0 0x55c235213b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96ce59c640 1 -- 172.21.15.103:0/3922561326 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c234ee23c0 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96ce59c640 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c23520aff0 0x55c235213080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96c000f690 tx=0x7f96c000fb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b58bf6d119c512d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3922561326 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0004220 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3922561326 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f96c00043c0 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96c801e070 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3922561326 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f96c000fe90 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96d7103740 10 client.25306 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.147+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c235224940 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3922561326 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c801e070 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3922561326 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f96c0003b30 con 0x55c23520aff0 2024-08-19T01:37:17.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96d7103740 10 client.25306 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96d7103740 10 client.25306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96d7103740 20 client.25306 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96d7103740 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235225d50 0x55c2352281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96d7103740 4 client.25306 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c23517dfb0 con 0x55c235225d50 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.148+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235225d50 0x55c2352281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.149+0000 7f96b5ffb640 10 client.25306 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:17.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.149+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235225d50 0x55c2352281a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819b758f041a78d3 server_cookie=3e53c275e0b95b57 in_seq=0 out_seq=0 2024-08-19T01:37:17.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.149+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3922561326 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c23517dfb0 con 0x55c235225d50 2024-08-19T01:37:17.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.149+0000 7f96b5ffb640 10 client.25306 handle_command_reply: tid=0 2024-08-19T01:37:17.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.149+0000 7f96d7103740 1 client.25306 shutdown 2024-08-19T01:37:17.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c23520aff0 msgr2=0x55c235213080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c23520aff0 0x55c235213080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f96c000f690 tx=0x7f96c000fb50 comp rx=0 tx=0).stop 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 shutdown_connections 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235225d50 0x55c2352281a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2352161d0 0x55c235213b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c2352135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 --2- 172.21.15.103:0/3922561326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c23520aff0 0x55c235213080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 >> 172.21.15.103:0/3922561326 conn(0x55c235207580 msgr2=0x55c23520d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96b5ffb640 0 client.25306 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 shutdown_connections 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 1 -- 172.21.15.103:0/3922561326 wait complete. 2024-08-19T01:37:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.150+0000 7f96d7103740 20 client.25306 trim_cache size 0 max 16384 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.169+0000 7f96d7103740 1 Processor -- start 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.169+0000 7f96d7103740 1 -- start start 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96d7103740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96d7103740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c23522da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96d7103740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c23522dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96d7103740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01aaf90 con 0x55c23520aff0 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96d7103740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01acb00 con 0x55c23520b910 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96d7103740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01ab660 con 0x55c2352161d0 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52896/0 (socket says 172.21.15.103:52896) 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 -- 172.21.15.103:0/2781401068 learned_addr learned my addr 172.21.15.103:0/2781401068 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:17.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ced9d640 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c23522dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c23522da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 -- 172.21.15.103:0/2781401068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 msgr2=0x55c23522dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c23522dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 -- 172.21.15.103:0/2781401068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 msgr2=0x55c23522da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c23522da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 -- 172.21.15.103:0/2781401068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c235222000 con 0x55c23520aff0 2024-08-19T01:37:17.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.170+0000 7f96ce59c640 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520daa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f96c000ad70 tx=0x7f96c000f570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2488f3e4a4979b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96b7fff640 1 -- 172.21.15.103:0/2781401068 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000a730 con 0x55c23520aff0 2024-08-19T01:37:17.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96b7fff640 1 -- 172.21.15.103:0/2781401068 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c00235e0 con 0x55c23520aff0 2024-08-19T01:37:17.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96b7fff640 1 -- 172.21.15.103:0/2781401068 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000db20 con 0x55c23520aff0 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 -- 172.21.15.103:0/2781401068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 msgr2=0x55c23520daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520daa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f96c000ad70 tx=0x7f96c000f570 comp rx=0 tx=0).stop 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 -- 172.21.15.103:0/2781401068 shutdown_connections 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c23522dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c23522da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 --2- 172.21.15.103:0/2781401068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23520aff0 0x55c23520daa0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 -- 172.21.15.103:0/2781401068 >> 172.21.15.103:0/2781401068 conn(0x55c235207580 msgr2=0x55c235212170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.171+0000 7f96d7103740 1 -- 172.21.15.103:0/2781401068 shutdown_connections 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 -- 172.21.15.103:0/2781401068 wait complete. 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 -- start start 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 Processor -- start 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235219870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c235219db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23521df90 0x55c23521e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01ac400 con 0x55c23521df90 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01abd30 con 0x55c23520b910 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01a8f70 con 0x55c2352161d0 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96d7103740 10 client.0 _ll_register_callbacks cb 0x7f96c000a450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23521df90 0x55c23521e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23521df90 0x55c23521e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52912/0 (socket says 172.21.15.103:52912) 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.172+0000 7f96ced9d640 1 -- 172.21.15.103:0/3221985970 learned_addr learned my addr 172.21.15.103:0/3221985970 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ce59c640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235219870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96cdd9b640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c235219db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ced9d640 1 -- 172.21.15.103:0/3221985970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 msgr2=0x55c235219db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ced9d640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c235219db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ced9d640 1 -- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 msgr2=0x55c235219870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ced9d640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235219870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ced9d640 1 -- 172.21.15.103:0/3221985970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c23521ea50 con 0x55c23521df90 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ce59c640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235219870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96ced9d640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23521df90 0x55c23521e410 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f96c8018fd0 tx=0x7f96c8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b58e31c63eaa009 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.173+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3221985970 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8030020 con 0x55c23521df90 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3221985970 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f96c800a400 con 0x55c23521df90 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96c00235e0 con 0x55c23521df90 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3221985970 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f96c8009390 con 0x55c23521df90 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 10 client.15822 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:17.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2352909d0 con 0x55c23521df90 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3221985970 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c801e040 con 0x55c23521df90 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3221985970 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f96c8022430 con 0x55c23521df90 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 20 client.15822 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235290ea0 0x55c235293290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 4 client.15822 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.174+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f96c00235e0 con 0x55c235290ea0 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96ce59c640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235290ea0 0x55c235293290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96ce59c640 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235290ea0 0x55c235293290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eccb41217518bd36 server_cookie=edd10c9bb7e4fd86 in_seq=0 out_seq=0 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96b5ffb640 10 client.15822 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96b5ffb640 1 -- 172.21.15.103:0/3221985970 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f96c00235e0 con 0x55c235290ea0 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96b5ffb640 10 client.15822 handle_command_reply: tid=0 2024-08-19T01:37:17.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96d7103740 1 client.15822 shutdown 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23521df90 msgr2=0x55c23521e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96d7103740 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23521df90 0x55c23521e410 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f96c8018fd0 tx=0x7f96c8009c20 comp rx=0 tx=0).stop 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 shutdown_connections 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.175+0000 7f96d7103740 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55c235290ea0 0x55c235293290 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d7103740 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c23521df90 0x55c23521e410 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96b5ffb640 0 client.15822 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d7103740 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c2352161d0 0x55c235219db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d7103740 1 --2- 172.21.15.103:0/3221985970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c23520b910 0x55c235219870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 >> 172.21.15.103:0/3221985970 conn(0x55c235207580 msgr2=0x55c235205d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 shutdown_connections 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d7103740 1 -- 172.21.15.103:0/3221985970 wait complete. 2024-08-19T01:37:17.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d7103740 20 client.15822 trim_cache size 0 max 16384 2024-08-19T01:37:17.182 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96b80a2c00 msgr2=0x7f96b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d4d05640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96b80a2c00 0x7f96b80a50c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96c801d000 tx=0x7f96c8017000 comp rx=0 tx=0).stop 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d0102240 msgr2=0x7f96d01986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.176+0000 7f96d4d05640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d0102240 0x7f96d01986f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f96c4005940 tx=0x7f96c40042f0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 shutdown_connections 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f96b80a2c00 0x7f96b80a50c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d01078a0 0x7f96d019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96d0102240 0x7f96d01986f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 --2- 172.21.15.103:0/2402915072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f96d01018a0 0x7f96d01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 >> 172.21.15.103:0/2402915072 conn(0x7f96d00fd5c0 msgr2=0x7f96d00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 shutdown_connections 2024-08-19T01:37:17.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:17.177+0000 7f96d4d05640 1 -- 172.21.15.103:0/2402915072 wait complete. 2024-08-19T01:37:17.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:16 smithi103 ceph-mon[18436]: pgmap v384: 97 pgs: 97 active+clean; 874 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:37:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:16 smithi190 ceph-mon[22032]: pgmap v384: 97 pgs: 97 active+clean; 874 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:37:17.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:16 smithi146 ceph-mon[22098]: pgmap v384: 97 pgs: 97 active+clean; 874 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:37:19.017 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:37:19.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:37:19.306 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:19.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 -- 172.21.15.103:0/3357239345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c102250 msgr2=0x7f4e2c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 --2- 172.21.15.103:0/3357239345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c102250 0x7f4e2c10e780 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009cd0 tx=0x7f4e2402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 -- 172.21.15.103:0/3357239345 shutdown_connections 2024-08-19T01:37:19.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 --2- 172.21.15.103:0/3357239345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c102250 0x7f4e2c10e780 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 --2- 172.21.15.103:0/3357239345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c1018b0 0x7f4e2c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 --2- 172.21.15.103:0/3357239345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1078b0 0x7f4e2c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 -- 172.21.15.103:0/3357239345 >> 172.21.15.103:0/3357239345 conn(0x7f4e2c0fd5b0 msgr2=0x7f4e2c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 -- 172.21.15.103:0/3357239345 shutdown_connections 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.805+0000 7f4e310df640 1 -- 172.21.15.103:0/3357239345 wait complete. 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.806+0000 7f4e310df640 1 Processor -- start 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.806+0000 7f4e310df640 1 -- start start 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e310df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1018b0 0x7f4e2c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e310df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c102250 0x7f4e2c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e310df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c1078b0 0x7f4e2c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e310df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c111b60 con 0x7f4e2c1078b0 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e310df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1119e0 con 0x7f4e2c1018b0 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e310df640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c111ce0 con 0x7f4e2c102250 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c1078b0 0x7f4e2c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1018b0 0x7f4e2c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1018b0 0x7f4e2c074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:49722/0 (socket says 172.21.15.103:49722) 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c1078b0 0x7f4e2c074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52928/0 (socket says 172.21.15.103:52928) 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/2976611639 learned_addr learned my addr 172.21.15.103:0/2976611639 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c102250 0x7f4e2c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/2976611639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c102250 msgr2=0x7f4e2c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c102250 0x7f4e2c074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c1078b0 msgr2=0x7f4e2c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c1078b0 0x7f4e2c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/2976611639 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1a6500 con 0x7f4e2c1018b0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c1078b0 0x7f4e2c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.807+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1018b0 0x7f4e2c074220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e9f0 tx=0x7f4e1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.808+0000 7f4e13fff640 1 -- 172.21.15.103:0/2976611639 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00cd80 con 0x7f4e2c1018b0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.808+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1a67f0 con 0x7f4e2c1018b0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.808+0000 7f4e13fff640 1 -- 172.21.15.103:0/2976611639 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e1c0108e0 con 0x7f4e2c1018b0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.808+0000 7f4e13fff640 1 -- 172.21.15.103:0/2976611639 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c018920 con 0x7f4e2c1018b0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.808+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1a6d80 con 0x7f4e2c1018b0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.810+0000 7f4e13fff640 1 -- 172.21.15.103:0/2976611639 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4e1c010a90 con 0x7f4e2c1018b0 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 Processor -- start 2024-08-19T01:37:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 -- start start 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e13fff640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4e140a2b80 0x7f4e140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e68aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e68d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56337e643e40 con 0x56337e680200 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56337e5e57f0 con 0x56337e68b3e0 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e334dd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56337e58cfb0 con 0x56337e680b20 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e13fff640 1 -- 172.21.15.103:0/2976611639 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4e1c0cf2a0 con 0x7f4e2c1018b0 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4e140a2b80 0x7f4e140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6805e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52936/0 (socket says 172.21.15.103:52936) 2024-08-19T01:37:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/1098863342 learned_addr learned my addr 172.21.15.103:0/1098863342 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e68d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e68aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/1098863342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 msgr2=0x56337e68aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e68aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/1098863342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 msgr2=0x56337e68d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e68d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.811+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/1098863342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337e3576a0 con 0x56337e680200 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.812+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4e140a2b80 0x7f4e140a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e24004af0 tx=0x7f4e24009880 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.812+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6805e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c09bb70 tx=0x7f4e1c010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9394f1bae0c2f3fb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.812+0000 7f4e117fa640 1 -- 172.21.15.103:0/1098863342 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0a47a0 con 0x56337e680200 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.812+0000 7f4e117fa640 1 -- 172.21.15.103:0/1098863342 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e1c099c80 con 0x56337e680200 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.812+0000 7f4e117fa640 1 -- 172.21.15.103:0/1098863342 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c09b030 con 0x56337e680200 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 -- 172.21.15.103:0/1098863342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 msgr2=0x56337e6805e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6805e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c09bb70 tx=0x7f4e1c010de0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 -- 172.21.15.103:0/1098863342 shutdown_connections 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e68d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e68aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 --2- 172.21.15.103:0/1098863342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6805e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 -- 172.21.15.103:0/1098863342 >> 172.21.15.103:0/1098863342 conn(0x56337e67c790 msgr2=0x56337e67cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 -- 172.21.15.103:0/1098863342 shutdown_connections 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 -- 172.21.15.103:0/1098863342 wait complete. 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.813+0000 7f4e334dd740 1 -- start start 2024-08-19T01:37:19.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 1 Processor -- start 2024-08-19T01:37:19.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e688810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e688d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56337e5a0cb0 con 0x56337e680200 2024-08-19T01:37:19.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56337e5e17b0 con 0x56337e68b3e0 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56337e68dfb0 con 0x56337e680b20 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e334dd740 10 client.0 _ll_register_callbacks cb 0x56337e2f4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6882d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52938/0 (socket says 172.21.15.103:52938) 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/3750545 learned_addr learned my addr 172.21.15.103:0/3750545 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e688810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e688d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/3750545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 msgr2=0x56337e688810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e688810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 msgr2=0x56337e688d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e688d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2ad9d640 1 -- 172.21.15.103:0/3750545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337e59fbc0 con 0x56337e680200 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.814+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e688d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:19.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6882d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c08eb80 tx=0x7f4e1c0a4820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce033e8f3619866 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e0eff5640 1 -- 172.21.15.103:0/3750545 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c083070 con 0x56337e680200 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e0eff5640 1 -- 172.21.15.103:0/3750545 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4e1c010630 con 0x56337e680200 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1c099c80 con 0x56337e680200 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e0eff5640 1 -- 172.21.15.103:0/3750545 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c09ce60 con 0x56337e680200 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e334dd740 10 client.15834 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56337e699e70 con 0x56337e680200 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e0eff5640 1 -- 172.21.15.103:0/3750545 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e1c0b3080 con 0x56337e680200 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e0eff5640 1 -- 172.21.15.103:0/3750545 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f4e1c09aaa0 con 0x56337e680200 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e334dd740 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e334dd740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.815+0000 7f4e334dd740 20 client.15834 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.816+0000 7f4e334dd740 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e69b1c0 0x56337e69d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.816+0000 7f4e334dd740 4 client.15834 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.816+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56337e5f2d60 con 0x56337e69b1c0 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.816+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e69b1c0 0x56337e69d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.816+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e69b1c0 0x56337e69d5b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c8513da57b4ec3 server_cookie=a1b3ee6cd10723eb in_seq=0 out_seq=0 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.816+0000 7f4e0eff5640 10 client.15834 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e0eff5640 1 -- 172.21.15.103:0/3750545 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56337e5f2d60 con 0x56337e69b1c0 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e0eff5640 10 client.15834 handle_command_reply: tid=0 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 client.15834 shutdown 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 msgr2=0x56337e6882d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6882d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c08eb80 tx=0x7f4e1c0a4820 comp rx=0 tx=0).stop 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 shutdown_connections 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e69b1c0 0x56337e69d5b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e688d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680b20 0x56337e688810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 --2- 172.21.15.103:0/3750545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680200 0x56337e6882d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e0eff5640 0 client.15834 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 >> 172.21.15.103:0/3750545 conn(0x56337e67c790 msgr2=0x56337e6824c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.817+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 shutdown_connections 2024-08-19T01:37:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.818+0000 7f4e334dd740 1 -- 172.21.15.103:0/3750545 wait complete. 2024-08-19T01:37:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.818+0000 7f4e334dd740 20 client.15834 trim_cache size 0 max 16384 2024-08-19T01:37:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 Processor -- start 2024-08-19T01:37:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 -- start start 2024-08-19T01:37:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680200 0x56337e682c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e688ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e6a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1af2f0 con 0x56337e680b20 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1b0e60 con 0x56337e68b3e0 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e334dd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1af9c0 con 0x56337e680200 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e2a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e688ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e2a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e688ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52948/0 (socket says 172.21.15.103:52948) 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e2a59c640 1 -- 172.21.15.103:0/85252320 learned_addr learned my addr 172.21.15.103:0/85252320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.838+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e6a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680200 0x56337e682c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2a59c640 1 -- 172.21.15.103:0/85252320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680200 msgr2=0x56337e682c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680200 0x56337e682c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2a59c640 1 -- 172.21.15.103:0/85252320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 msgr2=0x56337e6a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e6a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2a59c640 1 -- 172.21.15.103:0/85252320 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337e680950 con 0x56337e680b20 2024-08-19T01:37:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e6a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e688ba0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4e20000c00 tx=0x7f4e2000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6514bf8bd60570c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e10ff9640 1 -- 172.21.15.103:0/85252320 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20003d60 con 0x56337e680b20 2024-08-19T01:37:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e10ff9640 1 -- 172.21.15.103:0/85252320 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e20003f00 con 0x56337e680b20 2024-08-19T01:37:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.839+0000 7f4e10ff9640 1 -- 172.21.15.103:0/85252320 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20011440 con 0x56337e680b20 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 -- 172.21.15.103:0/85252320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 msgr2=0x56337e688ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e688ba0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4e20000c00 tx=0x7f4e2000bdc0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 -- 172.21.15.103:0/85252320 shutdown_connections 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e68b3e0 0x56337e6a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e688ba0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 --2- 172.21.15.103:0/85252320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e680200 0x56337e682c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 -- 172.21.15.103:0/85252320 >> 172.21.15.103:0/85252320 conn(0x56337e67c790 msgr2=0x56337e68a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 -- 172.21.15.103:0/85252320 shutdown_connections 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.840+0000 7f4e334dd740 1 -- 172.21.15.103:0/85252320 wait complete. 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 -- start start 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 Processor -- start 2024-08-19T01:37:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e680200 0x56337e693760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e684e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e68b3e0 0x56337e685360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1b0760 con 0x56337e680b20 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1b0090 con 0x56337e680200 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1ad2d0 con 0x56337e68b3e0 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e334dd740 10 client.0 _ll_register_callbacks cb 0x7f4e24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e2a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e684e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e2a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e684e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52960/0 (socket says 172.21.15.103:52960) 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e2a59c640 1 -- 172.21.15.103:0/355238579 learned_addr learned my addr 172.21.15.103:0/355238579 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e680200 0x56337e693760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.841+0000 7f4e2b59e640 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e68b3e0 0x56337e685360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e2a59c640 1 -- 172.21.15.103:0/355238579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e68b3e0 msgr2=0x56337e685360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e68b3e0 0x56337e685360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e2a59c640 1 -- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e680200 msgr2=0x56337e693760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e680200 0x56337e693760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e2a59c640 1 -- 172.21.15.103:0/355238579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337e6859a0 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e2a59c640 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e684e20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e20004db0 tx=0x7f4e20004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eca8693c969667d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e0eff5640 1 -- 172.21.15.103:0/355238579 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20004900 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e0eff5640 1 -- 172.21.15.103:0/355238579 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4e20005020 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e20003f00 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e0eff5640 1 -- 172.21.15.103:0/355238579 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e20005220 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e334dd740 10 client.15840 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56337e705a20 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.842+0000 7f4e0eff5640 1 -- 172.21.15.103:0/355238579 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e2002a030 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e0eff5640 1 -- 172.21.15.103:0/355238579 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f4e20011aa0 con 0x56337e680b20 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e334dd740 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e334dd740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e334dd740 20 client.15840 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e334dd740 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e7060c0 0x56337e7084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e334dd740 4 client.15840 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4e20003f00 con 0x56337e7060c0 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e7060c0 0x56337e7084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e2ad9d640 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e7060c0 0x56337e7084e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e465cae89c455d1f server_cookie=d2532fc5dd818977 in_seq=0 out_seq=0 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.843+0000 7f4e0eff5640 10 client.15840 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e0eff5640 1 -- 172.21.15.103:0/355238579 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4e20003f00 con 0x56337e7060c0 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e0eff5640 10 client.15840 handle_command_reply: tid=0 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 client.15840 shutdown 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 msgr2=0x56337e684e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e684e20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4e20004db0 tx=0x7f4e20004de0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 shutdown_connections 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56337e7060c0 0x56337e7084e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337e68b3e0 0x56337e685360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56337e680b20 0x56337e684e20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 --2- 172.21.15.103:0/355238579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56337e680200 0x56337e693760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 >> 172.21.15.103:0/355238579 conn(0x56337e67c790 msgr2=0x56337e67b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.844+0000 7f4e0eff5640 0 client.15840 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 shutdown_connections 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e334dd740 1 -- 172.21.15.103:0/355238579 wait complete. 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e334dd740 20 client.15840 trim_cache size 0 max 16384 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "1a3e886e-7c59-4ad4-a60d-b75a04d39e91", 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:37:19.851 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:37:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4e140a2b80 msgr2=0x7f4e140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4e140a2b80 0x7f4e140a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4e24004af0 tx=0x7f4e24009880 comp rx=0 tx=0).stop 2024-08-19T01:37:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1018b0 msgr2=0x7f4e2c074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1018b0 0x7f4e2c074220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e9f0 tx=0x7f4e1c00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 shutdown_connections 2024-08-19T01:37:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4e140a2b80 0x7f4e140a5040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e2c1078b0 0x7f4e2c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c102250 0x7f4e2c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 --2- 172.21.15.103:0/2976611639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4e2c1018b0 0x7f4e2c074220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.845+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 >> 172.21.15.103:0/2976611639 conn(0x7f4e2c0fd5b0 msgr2=0x7f4e2c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.846+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 shutdown_connections 2024-08-19T01:37:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:19.846+0000 7f4e310df640 1 -- 172.21.15.103:0/2976611639 wait complete. 2024-08-19T01:37:20.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:19 smithi103 ceph-mon[18436]: pgmap v385: 97 pgs: 97 active+clean; 954 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:37:20.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:19 smithi146 ceph-mon[22098]: pgmap v385: 97 pgs: 97 active+clean; 954 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:37:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:19 smithi190 ceph-mon[22032]: pgmap v385: 97 pgs: 97 active+clean; 954 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:37:20.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:37:20.863 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:21.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:37:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:37:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:37:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:37:21.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:37:21.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:20 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.226 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:37:21.227 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.227 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:20 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:37:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:37:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:37:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:37:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:37:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:20 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:37:21.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/1857309036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8107fd0 msgr2=0x7f4cc810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/1857309036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8107fd0 0x7f4cc810e970 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009cd0 tx=0x7f4cc002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/1857309036 shutdown_connections 2024-08-19T01:37:21.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/1857309036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8107fd0 0x7f4cc810e970 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/1857309036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cc8103410 0x7f4cc81078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/1857309036 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cc8102a60 0x7f4cc8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/1857309036 >> 172.21.15.103:0/1857309036 conn(0x7f4cc80fc500 msgr2=0x7f4cc80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/1857309036 shutdown_connections 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.455+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/1857309036 wait complete. 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.456+0000 7f4ccf5f7640 1 Processor -- start 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.456+0000 7f4ccf5f7640 1 -- start start 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.456+0000 7f4ccf5f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8102a60 0x7f4cc8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.456+0000 7f4ccf5f7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cc8103410 0x7f4cc8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccf5f7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cc8107fd0 0x7f4cc819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccf5f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8111c80 con 0x7f4cc8102a60 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccf5f7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8111b00 con 0x7f4cc8107fd0 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccf5f7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8111e00 con 0x7f4cc8103410 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8102a60 0x7f4cc8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8102a60 0x7f4cc8198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52982/0 (socket says 172.21.15.103:52982) 2024-08-19T01:37:21.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 -- 172.21.15.103:0/2734367986 learned_addr learned my addr 172.21.15.103:0/2734367986 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cc8107fd0 0x7f4cc819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4cccb92640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cc8103410 0x7f4cc8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 -- 172.21.15.103:0/2734367986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cc8103410 msgr2=0x7f4cc8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cc8103410 0x7f4cc8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 -- 172.21.15.103:0/2734367986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cc8107fd0 msgr2=0x7f4cc819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cc8107fd0 0x7f4cc819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 -- 172.21.15.103:0/2734367986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc819d0e0 con 0x7f4cc8102a60 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4cccb92640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cc8103410 0x7f4cc8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccd393640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8102a60 0x7f4cc8198130 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c920 tx=0x7f4cc400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4cbe7fc640 1 -- 172.21.15.103:0/2734367986 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4007c70 con 0x7f4cc8102a60 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc819d3d0 con 0x7f4cc8102a60 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4cbe7fc640 1 -- 172.21.15.103:0/2734367986 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc4007e10 con 0x7f4cc8102a60 2024-08-19T01:37:21.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.457+0000 7f4cbe7fc640 1 -- 172.21.15.103:0/2734367986 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4018820 con 0x7f4cc8102a60 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.458+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc81a4cb0 con 0x7f4cc8102a60 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.460+0000 7f4cbe7fc640 1 -- 172.21.15.103:0/2734367986 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4cc40189c0 con 0x7f4cc8102a60 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 Processor -- start 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 -- start start 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1ed620 0x55d8ee1eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d8ee14e980 con 0x55d8ee1ed620 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d8ee0e6a70 con 0x55d8ee1f8800 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d8ee14a940 con 0x55d8ee1edf40 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4ccdb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1ed620 0x55d8ee1eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4ccdb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1ed620 0x55d8ee1eda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52988/0 (socket says 172.21.15.103:52988) 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4ccdb94640 1 -- 172.21.15.103:0/3304829175 learned_addr learned my addr 172.21.15.103:0/3304829175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:21.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.461+0000 7f4ccd393640 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4cccb92640 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4cbe7fc640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4cb40a2b80 0x7f4cb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccdb94640 1 -- 172.21.15.103:0/3304829175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 msgr2=0x55d8ee1f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccdb94640 1 -- 172.21.15.103:0/3304829175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 msgr2=0x55d8ee1fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccdb94640 1 -- 172.21.15.103:0/3304829175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ee1496e0 con 0x55d8ee1ed620 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccd393640 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4cbe7fc640 1 -- 172.21.15.103:0/2734367986 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4cc40cf230 con 0x7f4cc8102a60 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1ed620 0x55d8ee1eda00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0035000 tx=0x7f4cc0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1051843e6e3d0b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4cb40a2b80 0x7f4cb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4cb37fe640 1 -- 172.21.15.103:0/3304829175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc004c070 con 0x55d8ee1ed620 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4cb37fe640 1 -- 172.21.15.103:0/3304829175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc0036040 con 0x55d8ee1ed620 2024-08-19T01:37:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.462+0000 7f4cb37fe640 1 -- 172.21.15.103:0/3304829175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0030800 con 0x55d8ee1ed620 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 -- 172.21.15.103:0/3304829175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1ed620 msgr2=0x55d8ee1eda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1ed620 0x55d8ee1eda00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0035000 tx=0x7f4cc0009a20 comp rx=0 tx=0).stop 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4cb40a2b80 0x7f4cb40a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0049000 tx=0x7f4cc0030eb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 -- 172.21.15.103:0/3304829175 shutdown_connections 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/3304829175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1ed620 0x55d8ee1eda00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 -- 172.21.15.103:0/3304829175 >> 172.21.15.103:0/3304829175 conn(0x55d8ee1e9bb0 msgr2=0x55d8ee1e9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 -- 172.21.15.103:0/3304829175 shutdown_connections 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.463+0000 7f4cd19f5740 1 -- 172.21.15.103:0/3304829175 wait complete. 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 -- start start 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 Processor -- start 2024-08-19T01:37:21.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee1f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1f6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d8ee1fb3d0 con 0x55d8ee1f8800 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d8ee109aa0 con 0x55d8ee1edf40 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d8ee1fb550 con 0x55d8ee1ed620 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cd19f5740 10 client.0 _ll_register_callbacks cb 0x55d8ee0f8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4ccdb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4ccdb94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53000/0 (socket says 172.21.15.103:53000) 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4ccdb94640 1 -- 172.21.15.103:0/614574602 learned_addr learned my addr 172.21.15.103:0/614574602 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4cccb92640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.464+0000 7f4ccd393640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee1f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4ccdb94640 1 -- 172.21.15.103:0/614574602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 msgr2=0x55d8ee1f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee1f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4ccdb94640 1 -- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1edf40 msgr2=0x55d8ee1f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4ccdb94640 1 -- 172.21.15.103:0/614574602 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ee0ed650 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4ccd393640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee1f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1f6160 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4cc002a920 tx=0x7f4cc0048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4760864f06cd215 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/614574602 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0058070 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/614574602 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4cc0046d30 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc0036040 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/614574602 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cc004c040 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cd19f5740 10 client.15852 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8edec03c0 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.465+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/614574602 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc0036040 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/614574602 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f4cc0030070 con 0x55d8ee1f8800 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4cd19f5740 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4cd19f5740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:21.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4cd19f5740 20 client.15852 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee208150 0x55d8ee20a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4cd19f5740 4 client.15852 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8ee15bef0 con 0x55d8ee208150 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.466+0000 7f4ccd393640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee208150 0x55d8ee20a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.467+0000 7f4ccd393640 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee208150 0x55d8ee20a540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7193e6ad8409b87 server_cookie=55a919dc4b1eadb4 in_seq=0 out_seq=0 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.467+0000 7f4cb0ff9640 10 client.15852 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.467+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/614574602 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8ee15bef0 con 0x55d8ee208150 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.467+0000 7f4cb0ff9640 10 client.15852 handle_command_reply: tid=0 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.467+0000 7f4cd19f5740 1 client.15852 shutdown 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 msgr2=0x55d8ee1f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1f6160 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4cc002a920 tx=0x7f4cc0048040 comp rx=0 tx=0).stop 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 shutdown_connections 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee208150 0x55d8ee20a540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cb0ff9640 0 client.15852 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:21.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1f6160 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1edf40 0x55d8ee1f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/614574602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee1f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 >> 172.21.15.103:0/614574602 conn(0x55d8ee1e9bb0 msgr2=0x55d8ee1e4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:21.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 shutdown_connections 2024-08-19T01:37:21.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 1 -- 172.21.15.103:0/614574602 wait complete. 2024-08-19T01:37:21.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.468+0000 7f4cd19f5740 20 client.15852 trim_cache size 0 max 16384 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 Processor -- start 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 -- start start 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee20fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1edf40 0x55d8ee2100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81aaf30 con 0x55d8ee1edf40 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81acaa0 con 0x55d8ee1f8800 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.491+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81ab600 con 0x55d8ee1ed620 2024-08-19T01:37:21.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1edf40 0x55d8ee2100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1edf40 0x55d8ee2100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53008/0 (socket says 172.21.15.103:53008) 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4ccdb94640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 -- 172.21.15.103:0/1242576567 learned_addr learned my addr 172.21.15.103:0/1242576567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4ccd393640 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee20fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 -- 172.21.15.103:0/1242576567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 msgr2=0x55d8ee20fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee20fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 -- 172.21.15.103:0/1242576567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 msgr2=0x55d8ee1e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 -- 172.21.15.103:0/1242576567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ee1f04d0 con 0x55d8ee1edf40 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cccb92640 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1edf40 0x55d8ee2100b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4cc806a0a0 tx=0x7f4cb8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856c6ed80e12e6d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cb2ffd640 1 -- 172.21.15.103:0/1242576567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8009cb0 con 0x55d8ee1edf40 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cb2ffd640 1 -- 172.21.15.103:0/1242576567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cb8009e50 con 0x55d8ee1edf40 2024-08-19T01:37:21.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.492+0000 7f4cb2ffd640 1 -- 172.21.15.103:0/1242576567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb801c6d0 con 0x55d8ee1edf40 2024-08-19T01:37:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1242576567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1edf40 msgr2=0x55d8ee2100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1edf40 0x55d8ee2100b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4cc806a0a0 tx=0x7f4cb8012820 comp rx=0 tx=0).stop 2024-08-19T01:37:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1242576567 shutdown_connections 2024-08-19T01:37:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1f8800 0x55d8ee1e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1edf40 0x55d8ee2100b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1242576567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1ed620 0x55d8ee20fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1242576567 >> 172.21.15.103:0/1242576567 conn(0x55d8ee1e9bb0 msgr2=0x55d8ee1f47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1242576567 shutdown_connections 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.493+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1242576567 wait complete. 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 -- start start 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 Processor -- start 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1ed620 0x55d8ee200d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1fc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81ac3a0 con 0x55d8ee1f8800 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81abcd0 con 0x55d8ee1ed620 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81a8f10 con 0x55d8ee1edf40 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.494+0000 7f4cd19f5740 10 client.0 _ll_register_callbacks cb 0x7f4cc004a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1fc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1fc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:37282/0 (socket says 172.21.15.103:37282) 2024-08-19T01:37:21.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 -- 172.21.15.103:0/1454026333 learned_addr learned my addr 172.21.15.103:0/1454026333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4ccd393640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1ed620 0x55d8ee200d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 -- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1ed620 msgr2=0x55d8ee200d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1ed620 0x55d8ee200d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 -- 172.21.15.103:0/1454026333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 msgr2=0x55d8ee1fc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 -- 172.21.15.103:0/1454026333 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ee1fcbd0 con 0x55d8ee1edf40 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4ccdb94640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cccb92640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1fc050 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4cb801b000 tx=0x7f4cb8005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b7963dbef31c59c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/1454026333 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8009cb0 con 0x55d8ee1edf40 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.495+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/1454026333 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4cb8004000 con 0x55d8ee1edf40 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb8009e50 con 0x55d8ee1edf40 2024-08-19T01:37:21.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/1454026333 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cb802f070 con 0x55d8ee1edf40 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 10 client.25342 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8ee272980 con 0x55d8ee1edf40 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/1454026333 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cb8009e50 con 0x55d8ee1edf40 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/1454026333 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f4cb800f040 con 0x55d8ee1edf40 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 10 client.25342 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 10 client.25342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 20 client.25342 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee273000 0x55d8ee275420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 4 client.25342 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8ee156400 con 0x55d8ee273000 2024-08-19T01:37:21.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.496+0000 7f4ccd393640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee273000 0x55d8ee275420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:21.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4ccd393640 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee273000 0x55d8ee275420 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b5ea0b3d5b9ce0 server_cookie=287faa70091f4137 in_seq=0 out_seq=0 2024-08-19T01:37:21.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cb0ff9640 10 client.25342 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cb0ff9640 1 -- 172.21.15.103:0/1454026333 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d8ee156400 con 0x55d8ee273000 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cb0ff9640 10 client.25342 handle_command_reply: tid=0 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 client.25342 shutdown 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 msgr2=0x55d8ee1fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1fc050 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4cb801b000 tx=0x7f4cb8005760 comp rx=0 tx=0).stop 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 shutdown_connections 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d8ee273000 0x55d8ee275420 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8ee1f8800 0x55d8ee1fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d8ee1edf40 0x55d8ee1fc050 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 --2- 172.21.15.103:0/1454026333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d8ee1ed620 0x55d8ee200d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 >> 172.21.15.103:0/1454026333 conn(0x55d8ee1e9bb0 msgr2=0x55d8ee1f5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cb0ff9640 0 client.25342 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:21.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 shutdown_connections 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.497+0000 7f4cd19f5740 1 -- 172.21.15.103:0/1454026333 wait complete. 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4cd19f5740 20 client.25342 trim_cache size 0 max 16384 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: "1a3e886e-7c59-4ad4-a60d-b75a04d39e91": { 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "1a3e886e-7c59-4ad4-a60d-b75a04d39e91", 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:37:21.504 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:37:21.505 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:37:21.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4cb40a2b80 msgr2=0x7f4cb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4cb40a2b80 0x7f4cb40a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0049000 tx=0x7f4cc0030eb0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8102a60 msgr2=0x7f4cc8198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:21.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8102a60 0x7f4cc8198130 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c920 tx=0x7f4cc400cde0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 shutdown_connections 2024-08-19T01:37:21.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4cb40a2b80 0x7f4cb40a5040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cc8107fd0 0x7f4cc819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cc8103410 0x7f4cc8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 --2- 172.21.15.103:0/2734367986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cc8102a60 0x7f4cc8198130 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:21.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 >> 172.21.15.103:0/2734367986 conn(0x7f4cc80fc500 msgr2=0x7f4cc8105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:21.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 shutdown_connections 2024-08-19T01:37:21.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:21.498+0000 7f4ccf5f7640 1 -- 172.21.15.103:0/2734367986 wait complete. 2024-08-19T01:37:22.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:21 smithi146 ceph-mon[22098]: pgmap v386: 97 pgs: 97 active+clean; 994 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 27 op/s 2024-08-19T01:37:22.234 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:21 smithi103 ceph-mon[18436]: pgmap v386: 97 pgs: 97 active+clean; 994 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 27 op/s 2024-08-19T01:37:22.263 INFO:tasks.cephfs.filesystem:scrub status for tag:1a3e886e-7c59-4ad4-a60d-b75a04d39e91 - {'path': '/', 'tag': '1a3e886e-7c59-4ad4-a60d-b75a04d39e91', 'options': 'recursive,force'} 2024-08-19T01:37:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:21 smithi190 ceph-mon[22032]: pgmap v386: 97 pgs: 97 active+clean; 994 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 27 op/s 2024-08-19T01:37:24.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:23 smithi146 ceph-mon[22098]: pgmap v387: 97 pgs: 97 active+clean; 1.1 GiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 44 op/s 2024-08-19T01:37:24.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:23 smithi103 ceph-mon[18436]: pgmap v387: 97 pgs: 97 active+clean; 1.1 GiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 44 op/s 2024-08-19T01:37:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:23 smithi190 ceph-mon[22032]: pgmap v387: 97 pgs: 97 active+clean; 1.1 GiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 44 op/s 2024-08-19T01:37:26.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:25 smithi146 ceph-mon[22098]: pgmap v388: 97 pgs: 97 active+clean; 1.1 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 46 op/s 2024-08-19T01:37:26.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:25 smithi103 ceph-mon[18436]: pgmap v388: 97 pgs: 97 active+clean; 1.1 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 46 op/s 2024-08-19T01:37:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:25 smithi190 ceph-mon[22032]: pgmap v388: 97 pgs: 97 active+clean; 1.1 GiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 46 op/s 2024-08-19T01:37:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 -- 172.21.15.103:0/1939194580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 msgr2=0x7f5d6c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 --2- 172.21.15.103:0/1939194580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c108bb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009ca0 tx=0x7f5d6402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 -- 172.21.15.103:0/1939194580 shutdown_connections 2024-08-19T01:37:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 --2- 172.21.15.103:0/1939194580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c108bb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 --2- 172.21.15.103:0/1939194580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d6c101960 0x7f5d6c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 --2- 172.21.15.103:0/1939194580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c100770 0x7f5d6c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 -- 172.21.15.103:0/1939194580 >> 172.21.15.103:0/1939194580 conn(0x7f5d6c0fbf10 msgr2=0x7f5d6c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:26.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 -- 172.21.15.103:0/1939194580 shutdown_connections 2024-08-19T01:37:26.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.388+0000 7f5d71a17640 1 -- 172.21.15.103:0/1939194580 wait complete. 2024-08-19T01:37:26.391 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 Processor -- start 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 -- start start 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d6c100770 0x7f5d6c06a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c101960 0x7f5d6c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10c2d0 con 0x7f5d6c102320 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10c150 con 0x7f5d6c101960 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d71a17640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10c450 con 0x7f5d6c100770 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d6a7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c101960 0x7f5d6c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d6a7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c101960 0x7f5d6c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:45658/0 (socket says 172.21.15.103:45658) 2024-08-19T01:37:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48238/0 (socket says 172.21.15.103:48238) 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/2379634352 learned_addr learned my addr 172.21.15.103:0/2379634352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.389+0000 7f5d6affd640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d6c100770 0x7f5d6c06a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d6b7fe640 1 -- 172.21.15.103:0/2379634352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d6c100770 msgr2=0x7f5d6c06a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d6c100770 0x7f5d6c06a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d6b7fe640 1 -- 172.21.15.103:0/2379634352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c101960 msgr2=0x7f5d6c19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c101960 0x7f5d6c19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d6b7fe640 1 -- 172.21.15.103:0/2379634352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c19f640 con 0x7f5d6c102320 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c101960 0x7f5d6c19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c19ed30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5d6402ade0 tx=0x7f5d64003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d70a15640 1 -- 172.21.15.103:0/2379634352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64032ec0 con 0x7f5d6c102320 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d70a15640 1 -- 172.21.15.103:0/2379634352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d640044f0 con 0x7f5d6c102320 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d70a15640 1 -- 172.21.15.103:0/2379634352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64030ce0 con 0x7f5d6c102320 2024-08-19T01:37:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c19f870 con 0x7f5d6c102320 2024-08-19T01:37:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.390+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c19fde0 con 0x7f5d6c102320 2024-08-19T01:37:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d70a15640 1 -- 172.21.15.103:0/2379634352 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5d64004040 con 0x7f5d6c102320 2024-08-19T01:37:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 Processor -- start 2024-08-19T01:37:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 -- start start 2024-08-19T01:37:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9fc0 0x55d38a8cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d38a82c9d0 con 0x55d38a8cf050 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d38a7d4820 con 0x55d38a8c9680 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d747d4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d38a828990 con 0x55d38a8c9fc0 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48240/0 (socket says 172.21.15.103:48240) 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/1677708494 learned_addr learned my addr 172.21.15.103:0/1677708494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.393+0000 7f5d6affd640 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9fc0 0x55d38a8cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/1677708494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9fc0 msgr2=0x55d38a8cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9fc0 0x55d38a8cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/1677708494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 msgr2=0x55d38a8c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/1677708494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d38a540210 con 0x55d38a8cf050 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d70a15640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5d540a2e50 0x7f5d540a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8d1510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5d60012c40 tx=0x7f5d60005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a88d0deae6fe0ed server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d70a15640 1 -- 172.21.15.103:0/2379634352 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5d640f0410 con 0x7f5d6c102320 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d6affd640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5d540a2e50 0x7f5d540a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d51ffb640 1 -- 172.21.15.103:0/1677708494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60012d30 con 0x55d38a8cf050 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d51ffb640 1 -- 172.21.15.103:0/1677708494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d60010c00 con 0x55d38a8cf050 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.394+0000 7f5d51ffb640 1 -- 172.21.15.103:0/1677708494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d600047a0 con 0x55d38a8cf050 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 -- 172.21.15.103:0/1677708494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 msgr2=0x55d38a8d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8d1510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5d60012c40 tx=0x7f5d60005f30 comp rx=0 tx=0).stop 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d6affd640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5d540a2e50 0x7f5d540a5310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55d38a8ca9f0 tx=0x7f5d5c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 -- 172.21.15.103:0/1677708494 shutdown_connections 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8d1510 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9fc0 0x55d38a8cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 --2- 172.21.15.103:0/1677708494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 -- 172.21.15.103:0/1677708494 >> 172.21.15.103:0/1677708494 conn(0x55d38a8c4cf0 msgr2=0x55d38a8c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 -- 172.21.15.103:0/1677708494 shutdown_connections 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 -- 172.21.15.103:0/1677708494 wait complete. 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.395+0000 7f5d747d4740 1 -- start start 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 1 Processor -- start 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9680 0x55d38a8d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d38a8d2020 con 0x55d38a8cf050 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d38a7e8520 con 0x55d38a8c9fc0 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d38a8d21a0 con 0x55d38a8c9680 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d747d4740 10 client.0 _ll_register_callbacks cb 0x55d38a53fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9680 0x55d38a8d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8dadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48252/0 (socket says 172.21.15.103:48252) 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 -- 172.21.15.103:0/4013596945 learned_addr learned my addr 172.21.15.103:0/4013596945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 -- 172.21.15.103:0/4013596945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9680 msgr2=0x55d38a8d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9680 0x55d38a8d3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 -- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 msgr2=0x55d38a8d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6b7fe640 1 -- 172.21.15.103:0/4013596945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d38a60c620 con 0x55d38a8cf050 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6affd640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9680 0x55d38a8d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.396+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8dadb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5d64006010 tx=0x7f5d640bc680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cb0b75418ec358 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d4affd640 1 -- 172.21.15.103:0/4013596945 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d640bcca0 con 0x55d38a8cf050 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d4affd640 1 -- 172.21.15.103:0/4013596945 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5d640046a0 con 0x55d38a8cf050 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d60010c00 con 0x55d38a8cf050 2024-08-19T01:37:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d4affd640 1 -- 172.21.15.103:0/4013596945 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d640c1050 con 0x55d38a8cf050 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 10 client.15870 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d38a8d2020 con 0x55d38a8cf050 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d4affd640 1 -- 172.21.15.103:0/4013596945 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d60010c00 con 0x55d38a8cf050 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d4affd640 1 -- 172.21.15.103:0/4013596945 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f5d640f00d0 con 0x55d38a8cf050 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 20 client.15870 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a8df370 0x55d38a8e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 4 client.15870 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.397+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d38a83a100 con 0x55d38a8df370 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.398+0000 7f5d6affd640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a8df370 0x55d38a8e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.398+0000 7f5d4affd640 10 client.15870 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.398+0000 7f5d6affd640 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a8df370 0x55d38a8e17d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e68530ccca9625 server_cookie=b01c42ce5e5dbd07 in_seq=0 out_seq=0 2024-08-19T01:37:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d4affd640 1 -- 172.21.15.103:0/4013596945 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d38a83a100 con 0x55d38a8df370 2024-08-19T01:37:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d4affd640 10 client.15870 handle_command_reply: tid=0 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 client.15870 shutdown 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 msgr2=0x55d38a8dadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8dadb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5d64006010 tx=0x7f5d640bc680 comp rx=0 tx=0).stop 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 shutdown_connections 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a8df370 0x55d38a8e17d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8cf050 0x55d38a8dadb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d4affd640 0 client.15870 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 --2- 172.21.15.103:0/4013596945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8c9680 0x55d38a8d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 >> 172.21.15.103:0/4013596945 conn(0x55d38a8c4cf0 msgr2=0x55d38a8c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 shutdown_connections 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 1 -- 172.21.15.103:0/4013596945 wait complete. 2024-08-19T01:37:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.399+0000 7f5d747d4740 20 client.15870 trim_cache size 0 max 16384 2024-08-19T01:37:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 Processor -- start 2024-08-19T01:37:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 -- start start 2024-08-19T01:37:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9680 0x55d38a8c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a8a30 con 0x55d38a8c9680 2024-08-19T01:37:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1aa5a0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d747d4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a9100 con 0x55d38a8cf050 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9680 0x55d38a8c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9680 0x55d38a8c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48256/0 (socket says 172.21.15.103:48256) 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 -- 172.21.15.103:0/449364282 learned_addr learned my addr 172.21.15.103:0/449364282 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8dad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 -- 172.21.15.103:0/449364282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 msgr2=0x55d38a8c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 -- 172.21.15.103:0/449364282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 msgr2=0x55d38a8dad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8dad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6affd640 1 -- 172.21.15.103:0/449364282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d38a8e1e00 con 0x55d38a8c9680 2024-08-19T01:37:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.415+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.416+0000 7f5d6affd640 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9680 0x55d38a8c5ba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55d38a8d2e00 tx=0x7f5d5c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c5d2d6bc186f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.416+0000 7f5d517fa640 1 -- 172.21.15.103:0/449364282 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c01ea10 con 0x55d38a8c9680 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.416+0000 7f5d517fa640 1 -- 172.21.15.103:0/449364282 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d5c01ebb0 con 0x55d38a8c9680 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.416+0000 7f5d517fa640 1 -- 172.21.15.103:0/449364282 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c011680 con 0x55d38a8c9680 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- 172.21.15.103:0/449364282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9680 msgr2=0x55d38a8c5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9680 0x55d38a8c5ba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55d38a8d2e00 tx=0x7f5d5c004610 comp rx=0 tx=0).stop 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- 172.21.15.103:0/449364282 shutdown_connections 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9fc0 0x55d38a8dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 --2- 172.21.15.103:0/449364282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9680 0x55d38a8c5ba0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- 172.21.15.103:0/449364282 >> 172.21.15.103:0/449364282 conn(0x55d38a8c4cf0 msgr2=0x55d38a8d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- 172.21.15.103:0/449364282 shutdown_connections 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- 172.21.15.103:0/449364282 wait complete. 2024-08-19T01:37:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- start start 2024-08-19T01:37:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 Processor -- start 2024-08-19T01:37:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a9ea0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a97d0 con 0x55d38a8c9680 2024-08-19T01:37:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c1a6a10 con 0x55d38a8cf050 2024-08-19T01:37:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.417+0000 7f5d747d4740 10 client.0 _ll_register_callbacks cb 0x7f5d64009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48266/0 (socket says 172.21.15.103:48266) 2024-08-19T01:37:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/2707290940 learned_addr learned my addr 172.21.15.103:0/2707290940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6b7fe640 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6affd640 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/2707290940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 msgr2=0x55d38a8d96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8d96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 msgr2=0x55d38a8d8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8d8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 -- 172.21.15.103:0/2707290940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d38a8d9cb0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d6a7fc640 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d8690 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5d6001d000 tx=0x7f5d60014f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd21468132d7d21 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d4affd640 1 -- 172.21.15.103:0/2707290940 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60010de0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d4affd640 1 -- 172.21.15.103:0/2707290940 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5d60005020 con 0x55d38a8c9fc0 2024-08-19T01:37:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5c01ebb0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.418+0000 7f5d4affd640 1 -- 172.21.15.103:0/2707290940 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d60005220 con 0x55d38a8c9fc0 2024-08-19T01:37:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 10 client.15876 fetch_fsmap learned FSMap version 28 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d38a8dafd0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d4affd640 1 -- 172.21.15.103:0/2707290940 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d5c01ebb0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d4affd640 1 -- 172.21.15.103:0/2707290940 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f5d6002b4e0 con 0x55d38a8c9fc0 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 20 client.15876 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a94a5a0 0x55d38a94c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 4 client.15876 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d38a834450 con 0x55d38a94a5a0 2024-08-19T01:37:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.419+0000 7f5d6affd640 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a94a5a0 0x55d38a94c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.420+0000 7f5d4affd640 10 client.15876 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.420+0000 7f5d6affd640 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a94a5a0 0x55d38a94c9c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ff629a614b0b8b server_cookie=7e657ac1f7288bf9 in_seq=0 out_seq=0 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d4affd640 1 -- 172.21.15.103:0/2707290940 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d38a834450 con 0x55d38a94a5a0 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d4affd640 10 client.15876 handle_command_reply: tid=0 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 client.15876 shutdown 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9fc0 msgr2=0x55d38a8d8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d8690 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5d6001d000 tx=0x7f5d60014f80 comp rx=0 tx=0).stop 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 shutdown_connections 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d38a94a5a0 0x55d38a94c9c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d38a8cf050 0x55d38a8d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d38a8c9fc0 0x55d38a8d8690 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 --2- 172.21.15.103:0/2707290940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d38a8c9680 0x55d38a8d8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d4affd640 0 client.15876 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 >> 172.21.15.103:0/2707290940 conn(0x55d38a8c4cf0 msgr2=0x55d38a8c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 shutdown_connections 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 1 -- 172.21.15.103:0/2707290940 wait complete. 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.428+0000 7f5d747d4740 20 client.15876 trim_cache size 0 max 16384 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:37:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:37:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5d540a2e50 msgr2=0x7f5d540a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5d540a2e50 0x7f5d540a5310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55d38a8ca9f0 tx=0x7f5d5c00b390 comp rx=0 tx=0).stop 2024-08-19T01:37:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 msgr2=0x7f5d6c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c19ed30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5d6402ade0 tx=0x7f5d64003ba0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 shutdown_connections 2024-08-19T01:37:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5d540a2e50 0x7f5d540a5310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d6c102320 0x7f5d6c19ed30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d6c101960 0x7f5d6c19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 --2- 172.21.15.103:0/2379634352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d6c100770 0x7f5d6c06a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 >> 172.21.15.103:0/2379634352 conn(0x7f5d6c0fbf10 msgr2=0x7f5d6c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 shutdown_connections 2024-08-19T01:37:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:26.429+0000 7f5d71a17640 1 -- 172.21.15.103:0/2379634352 wait complete. 2024-08-19T01:37:27.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:26 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:37:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:26 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:37:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:26 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:37:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:26 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:37:27.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:26 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:37:27.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:26 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:37:28.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:27 smithi146 ceph-mon[22098]: pgmap v389: 97 pgs: 97 active+clean; 1.1 GiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 46 op/s 2024-08-19T01:37:28.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:27 smithi103 ceph-mon[18436]: pgmap v389: 97 pgs: 97 active+clean; 1.1 GiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 46 op/s 2024-08-19T01:37:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:27 smithi190 ceph-mon[22032]: pgmap v389: 97 pgs: 97 active+clean; 1.1 GiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 46 op/s 2024-08-19T01:37:30.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:29 smithi146 ceph-mon[22098]: pgmap v390: 97 pgs: 97 active+clean; 1.2 GiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 51 op/s 2024-08-19T01:37:30.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:29 smithi103 ceph-mon[18436]: pgmap v390: 97 pgs: 97 active+clean; 1.2 GiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 51 op/s 2024-08-19T01:37:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:29 smithi190 ceph-mon[22032]: pgmap v390: 97 pgs: 97 active+clean; 1.2 GiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 51 op/s 2024-08-19T01:37:32.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:31 smithi146 ceph-mon[22098]: pgmap v391: 97 pgs: 97 active+clean; 1.3 GiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:37:32.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:31 smithi103 ceph-mon[18436]: pgmap v391: 97 pgs: 97 active+clean; 1.3 GiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:37:32.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:31 smithi190 ceph-mon[22032]: pgmap v391: 97 pgs: 97 active+clean; 1.3 GiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:37:34.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:33 smithi146 ceph-mon[22098]: pgmap v392: 97 pgs: 97 active+clean; 1.3 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 MiB/s wr, 47 op/s 2024-08-19T01:37:34.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:33 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:37:34.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:33 smithi103 ceph-mon[18436]: pgmap v392: 97 pgs: 97 active+clean; 1.3 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 MiB/s wr, 47 op/s 2024-08-19T01:37:34.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:33 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:37:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:33 smithi190 ceph-mon[22032]: pgmap v392: 97 pgs: 97 active+clean; 1.3 GiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 MiB/s wr, 47 op/s 2024-08-19T01:37:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:33 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:37:35.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:37:35.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:34 smithi103 ceph-mon[18436]: pgmap v393: 97 pgs: 97 active+clean; 1.4 GiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 29 op/s 2024-08-19T01:37:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:34 smithi190 ceph-mon[22032]: pgmap v393: 97 pgs: 97 active+clean; 1.4 GiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 29 op/s 2024-08-19T01:37:35.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:37:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:34 smithi146 ceph-mon[22098]: pgmap v393: 97 pgs: 97 active+clean; 1.4 GiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 29 op/s 2024-08-19T01:37:37.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:37 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:37:37.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:37 smithi190 ceph-mon[22032]: pgmap v394: 97 pgs: 97 active+clean; 1.4 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:37:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:37 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:37:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:37 smithi146 ceph-mon[22098]: pgmap v394: 97 pgs: 97 active+clean; 1.4 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:37:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:37 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:37:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:37 smithi103 ceph-mon[18436]: pgmap v394: 97 pgs: 97 active+clean; 1.4 GiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:37:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 -- 172.21.15.103:0/731796356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 msgr2=0x7fdccc100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 --2- 172.21.15.103:0/731796356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc100b80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009900 tx=0x7fdcc802a410 comp rx=0 tx=0).stop 2024-08-19T01:37:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 -- 172.21.15.103:0/731796356 shutdown_connections 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 --2- 172.21.15.103:0/731796356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdccc102330 0x7fdccc108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 --2- 172.21.15.103:0/731796356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdccc101970 0x7fdccc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 --2- 172.21.15.103:0/731796356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc100b80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 -- 172.21.15.103:0/731796356 >> 172.21.15.103:0/731796356 conn(0x7fdccc0fbee0 msgr2=0x7fdccc0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 -- 172.21.15.103:0/731796356 shutdown_connections 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.535+0000 7fdcd458a640 1 -- 172.21.15.103:0/731796356 wait complete. 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 Processor -- start 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 -- start start 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdccc101970 0x7fdccc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdccc102330 0x7fdccc1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc10c290 con 0x7fdccc100780 2024-08-19T01:37:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc10c110 con 0x7fdccc102330 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd458a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc10c410 con 0x7fdccc101970 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd2326640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd2326640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55864/0 (socket says 172.21.15.103:55864) 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd2326640 1 -- 172.21.15.103:0/3341663532 learned_addr learned my addr 172.21.15.103:0/3341663532 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd1b25640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdccc101970 0x7fdccc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.536+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdccc102330 0x7fdccc1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd2326640 1 -- 172.21.15.103:0/3341663532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdccc101970 msgr2=0x7fdccc19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd2326640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdccc101970 0x7fdccc19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd2326640 1 -- 172.21.15.103:0/3341663532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdccc102330 msgr2=0x7fdccc1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd2326640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdccc102330 0x7fdccc1a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd2326640 1 -- 172.21.15.103:0/3341663532 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc1a18d0 con 0x7fdccc100780 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd2326640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc199c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80314e0 tx=0x7fdcc8031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcc37fe640 1 -- 172.21.15.103:0/3341663532 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc803f030 con 0x7fdccc100780 2024-08-19T01:37:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc1a1b60 con 0x7fdccc100780 2024-08-19T01:37:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcc37fe640 1 -- 172.21.15.103:0/3341663532 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc80044b0 con 0x7fdccc100780 2024-08-19T01:37:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc1a2050 con 0x7fdccc100780 2024-08-19T01:37:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.537+0000 7fdcc37fe640 1 -- 172.21.15.103:0/3341663532 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8049060 con 0x7fdccc100780 2024-08-19T01:37:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.540+0000 7fdcc37fe640 1 -- 172.21.15.103:0/3341663532 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdcc8042030 con 0x7fdccc100780 2024-08-19T01:37:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 Processor -- start 2024-08-19T01:37:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 -- start start 2024-08-19T01:37:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x563091934a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091939a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x56309193c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcc37fe640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdcb80a2c30 0x7fdcb80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630918979d0 con 0x56309193a050 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56309183f820 con 0x563091934680 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd741d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563091893990 con 0x563091934fc0 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd1b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x56309193c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd1b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x56309193c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55872/0 (socket says 172.21.15.103:55872) 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd1b25640 1 -- 172.21.15.103:0/1717501637 learned_addr learned my addr 172.21.15.103:0/1717501637 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091939a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.541+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdcb80a2c30 0x7fdcb80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x563091934a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcc37fe640 1 -- 172.21.15.103:0/3341663532 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdcc80ef850 con 0x7fdccc100780 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2326640 1 -- 172.21.15.103:0/1717501637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 msgr2=0x563091934a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x563091934a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2326640 1 -- 172.21.15.103:0/1717501637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 msgr2=0x56309193c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x56309193c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2326640 1 -- 172.21.15.103:0/1717501637 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630915ab210 con 0x563091934fc0 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd1b25640 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x56309193c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdcb80a2c30 0x7fdcb80a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40098b0 tx=0x7fdcc4009840 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091939a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcc804b000 tx=0x7fdcc808f250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a96e9b9c93acf3a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcc0bfe640 1 -- 172.21.15.103:0/1717501637 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8035070 con 0x563091934fc0 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcc0bfe640 1 -- 172.21.15.103:0/1717501637 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc803b460 con 0x563091934fc0 2024-08-19T01:37:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.542+0000 7fdcc0bfe640 1 -- 172.21.15.103:0/1717501637 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80c4d50 con 0x563091934fc0 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 -- 172.21.15.103:0/1717501637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 msgr2=0x563091939a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091939a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcc804b000 tx=0x7fdcc808f250 comp rx=0 tx=0).stop 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 -- 172.21.15.103:0/1717501637 shutdown_connections 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x56309193c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091939a80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1717501637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x563091934a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 -- 172.21.15.103:0/1717501637 >> 172.21.15.103:0/1717501637 conn(0x56309192fcf0 msgr2=0x5630919300f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 -- 172.21.15.103:0/1717501637 shutdown_connections 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.543+0000 7fdcd741d740 1 -- 172.21.15.103:0/1717501637 wait complete. 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 -- start start 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 Processor -- start 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x56309193e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934fc0 0x56309193eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56309193a050 0x563091945b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56309193cde0 con 0x563091934fc0 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563091853520 con 0x563091934680 2024-08-19T01:37:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56309193cf60 con 0x56309193a050 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd741d740 10 client.0 _ll_register_callbacks cb 0x5630915aaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd1b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934fc0 0x56309193eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2b27640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56309193a050 0x563091945b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd1b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934fc0 0x56309193eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55880/0 (socket says 172.21.15.103:55880) 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd1b25640 1 -- 172.21.15.103:0/2614362680 learned_addr learned my addr 172.21.15.103:0/2614362680 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56309193a050 0x563091945b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:46238/0 (socket says 172.21.15.103:46238) 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2326640 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x56309193e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2b27640 1 -- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 msgr2=0x56309193e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x56309193e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2b27640 1 -- 172.21.15.103:0/2614362680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934fc0 msgr2=0x56309193eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934fc0 0x56309193eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.544+0000 7fdcd2b27640 1 -- 172.21.15.103:0/2614362680 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563091677620 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56309193a050 0x563091945b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009fd0 tx=0x7fdcc4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0aaf15d0af024e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/2614362680 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401fe30 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630918a5100 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/2614362680 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdcc4004b20 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/2614362680 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdcc4004d20 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 10 client.25372 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563091948a40 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/2614362680 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5630918a5100 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/2614362680 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdcc400f040 con 0x56309193a050 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 10 client.25372 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 20 client.25372 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56309194a130 0x56309194c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 4 client.25372 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.545+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56309189f450 con 0x56309194a130 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.546+0000 7fdcd2326640 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56309194a130 0x56309194c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.546+0000 7fdcb5ffb640 10 client.25372 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.546+0000 7fdcd2326640 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56309194a130 0x56309194c560 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944fdc5f3cdcf5c8 server_cookie=2f3e508f0b3d2a18 in_seq=0 out_seq=0 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/2614362680 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56309189f450 con 0x56309194a130 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcb5ffb640 10 client.25372 handle_command_reply: tid=0 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 client.25372 shutdown 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56309193a050 msgr2=0x563091945b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56309193a050 0x563091945b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009fd0 tx=0x7fdcc4004a40 comp rx=0 tx=0).stop 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 shutdown_connections 2024-08-19T01:37:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56309194a130 0x56309194c560 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56309193a050 0x563091945b30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934fc0 0x56309193eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2614362680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x56309193e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcb5ffb640 0 client.25372 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 >> 172.21.15.103:0/2614362680 conn(0x56309192fcf0 msgr2=0x563091931500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 shutdown_connections 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 1 -- 172.21.15.103:0/2614362680 wait complete. 2024-08-19T01:37:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.547+0000 7fdcd741d740 20 client.25372 trim_cache size 0 max 16384 2024-08-19T01:37:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 Processor -- start 2024-08-19T01:37:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 -- start start 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934680 0x56309192ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091945af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309193a050 0x563091950e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1aadb0 con 0x563091934680 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1ac950 con 0x56309193a050 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.565+0000 7fdcd741d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1ab480 con 0x563091934fc0 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934680 0x56309192ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934680 0x56309192ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55884/0 (socket says 172.21.15.103:55884) 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 -- 172.21.15.103:0/2187798961 learned_addr learned my addr 172.21.15.103:0/2187798961 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309193a050 0x563091950e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd1b25640 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091945af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 -- 172.21.15.103:0/2187798961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 msgr2=0x563091945af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091945af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 -- 172.21.15.103:0/2187798961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309193a050 msgr2=0x563091950e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309193a050 0x563091950e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 -- 172.21.15.103:0/2187798961 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630919354a0 con 0x563091934680 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309193a050 0x563091950e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcd2326640 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934680 0x56309192ed90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ed3d0 tx=0x7fdcc80dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413a2fbf2141d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcb7fff640 1 -- 172.21.15.103:0/2187798961 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8035040 con 0x563091934680 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcb7fff640 1 -- 172.21.15.103:0/2187798961 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc80d4070 con 0x563091934680 2024-08-19T01:37:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.566+0000 7fdcb7fff640 1 -- 172.21.15.103:0/2187798961 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80c0120 con 0x563091934680 2024-08-19T01:37:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 -- 172.21.15.103:0/2187798961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934680 msgr2=0x56309192ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934680 0x56309192ed90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ed3d0 tx=0x7fdcc80dc040 comp rx=0 tx=0).stop 2024-08-19T01:37:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 -- 172.21.15.103:0/2187798961 shutdown_connections 2024-08-19T01:37:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56309193a050 0x563091950e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091945af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 --2- 172.21.15.103:0/2187798961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563091934680 0x56309192ed90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 -- 172.21.15.103:0/2187798961 >> 172.21.15.103:0/2187798961 conn(0x56309192fcf0 msgr2=0x56309192e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 -- 172.21.15.103:0/2187798961 shutdown_connections 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.567+0000 7fdcd741d740 1 -- 172.21.15.103:0/2187798961 wait complete. 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 -- start start 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 Processor -- start 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x5630919431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091943710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x563091944720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1ac250 con 0x56309193a050 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1abb50 con 0x563091934680 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1a8d90 con 0x563091934fc0 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd741d740 10 client.0 _ll_register_callbacks cb 0x7fdcc4012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd2b27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x563091944720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd2b27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x563091944720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55898/0 (socket says 172.21.15.103:55898) 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd2b27640 1 -- 172.21.15.103:0/1298679020 learned_addr learned my addr 172.21.15.103:0/1298679020 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x5630919431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.568+0000 7fdcd1b25640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091943710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd2326640 1 -- 172.21.15.103:0/1298679020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 msgr2=0x563091943710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091943710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd2326640 1 -- 172.21.15.103:0/1298679020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 msgr2=0x563091944720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x563091944720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd2326640 1 -- 172.21.15.103:0/1298679020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563091944d30 con 0x563091934680 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd2b27640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x563091944720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd1b25640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091943710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd2326640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x5630919431d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x563091939de0 tx=0x7fdcc80c0520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aede170e3204a87 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/1298679020 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8035040 con 0x563091934680 2024-08-19T01:37:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc80d4070 con 0x563091934680 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/1298679020 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdcc80dc0b0 con 0x563091934680 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/1298679020 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdcc80c4d50 con 0x563091934680 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd741d740 10 client.25415 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5630919b4e40 con 0x563091934680 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.569+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/1298679020 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcc80d4070 con 0x563091934680 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/1298679020 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdcc80e6ce0 con 0x563091934680 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd741d740 10 client.25415 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd741d740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd741d740 20 client.25415 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630919b5680 0x5630919b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd741d740 4 client.25415 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563091939e10 con 0x5630919b5680 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd1b25640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630919b5680 0x5630919b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcb5ffb640 10 client.25415 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.570+0000 7fdcd1b25640 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630919b5680 0x5630919b7aa0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=789cbd701240d656 server_cookie=4df74d8016263d92 in_seq=0 out_seq=0 2024-08-19T01:37:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcb5ffb640 1 -- 172.21.15.103:0/1298679020 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563091939e10 con 0x5630919b5680 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcb5ffb640 10 client.25415 handle_command_reply: tid=0 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 client.25415 shutdown 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 msgr2=0x5630919431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x5630919431d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x563091939de0 tx=0x7fdcc80c0520 comp rx=0 tx=0).stop 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 shutdown_connections 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5630919b5680 0x5630919b7aa0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56309193a050 0x563091944720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563091934fc0 0x563091943710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 --2- 172.21.15.103:0/1298679020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563091934680 0x5630919431d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcb5ffb640 0 client.25415 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 >> 172.21.15.103:0/1298679020 conn(0x56309192fcf0 msgr2=0x56309192eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 shutdown_connections 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 1 -- 172.21.15.103:0/1298679020 wait complete. 2024-08-19T01:37:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.578+0000 7fdcd741d740 20 client.25415 trim_cache size 0 max 16384 2024-08-19T01:37:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:37:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:37:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:37:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdcb80a2c30 msgr2=0x7fdcb80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdcb80a2c30 0x7fdcb80a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40098b0 tx=0x7fdcc4009840 comp rx=0 tx=0).stop 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 msgr2=0x7fdccc199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc199c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80314e0 tx=0x7fdcc8031a30 comp rx=0 tx=0).stop 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 shutdown_connections 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdcb80a2c30 0x7fdcb80a50f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdccc102330 0x7fdccc1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdccc101970 0x7fdccc19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 --2- 172.21.15.103:0/3341663532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc100780 0x7fdccc199c90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 >> 172.21.15.103:0/3341663532 conn(0x7fdccc0fbee0 msgr2=0x7fdccc069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 shutdown_connections 2024-08-19T01:37:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:39.579+0000 7fdcd458a640 1 -- 172.21.15.103:0/3341663532 wait complete. 2024-08-19T01:37:40.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:39 smithi146 ceph-mon[22098]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:37:40.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:39 smithi103 ceph-mon[18436]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:37:40.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:39 smithi190 ceph-mon[22032]: pgmap v395: 97 pgs: 97 active+clean; 1.5 GiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:37:42.183 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:41 smithi190 ceph-mon[22032]: pgmap v396: 97 pgs: 97 active+clean; 1.6 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:37:42.360 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:41 smithi103 ceph-mon[18436]: pgmap v396: 97 pgs: 97 active+clean; 1.6 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:37:42.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:41 smithi146 ceph-mon[22098]: pgmap v396: 97 pgs: 97 active+clean; 1.6 GiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:37:44.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:43 smithi146 ceph-mon[22098]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:37:44.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:43 smithi103 ceph-mon[18436]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:37:44.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:43 smithi190 ceph-mon[22032]: pgmap v397: 97 pgs: 97 active+clean; 1.6 GiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:37:46.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:45 smithi146 ceph-mon[22098]: pgmap v398: 97 pgs: 97 active+clean; 1.7 GiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:37:46.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:45 smithi103 ceph-mon[18436]: pgmap v398: 97 pgs: 97 active+clean; 1.7 GiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:37:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:45 smithi190 ceph-mon[22032]: pgmap v398: 97 pgs: 97 active+clean; 1.7 GiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:37:48.202 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:47 smithi146 ceph-mon[22098]: pgmap v399: 97 pgs: 97 active+clean; 1.7 GiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:37:48.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:47 smithi103 ceph-mon[18436]: pgmap v399: 97 pgs: 97 active+clean; 1.7 GiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:37:48.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:47 smithi190 ceph-mon[22032]: pgmap v399: 97 pgs: 97 active+clean; 1.7 GiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:37:50.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:49 smithi103 ceph-mon[18436]: pgmap v400: 97 pgs: 97 active+clean; 1.8 GiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:37:50.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:49 smithi146 ceph-mon[22098]: pgmap v400: 97 pgs: 97 active+clean; 1.8 GiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:37:50.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:49 smithi190 ceph-mon[22032]: pgmap v400: 97 pgs: 97 active+clean; 1.8 GiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:37:52.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:51 smithi146 ceph-mon[22098]: pgmap v401: 97 pgs: 97 active+clean; 1.9 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:37:52.264 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:37:52.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:51 smithi103 ceph-mon[18436]: pgmap v401: 97 pgs: 97 active+clean; 1.9 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:37:52.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:51 smithi190 ceph-mon[22032]: pgmap v401: 97 pgs: 97 active+clean; 1.9 GiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:37:52.566 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 -- 172.21.15.103:0/4188646010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300101870 msgr2=0x7f5300101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 --2- 172.21.15.103:0/4188646010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300101870 0x7f5300101c70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009900 tx=0x7f52f002a410 comp rx=0 tx=0).stop 2024-08-19T01:37:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 -- 172.21.15.103:0/4188646010 shutdown_connections 2024-08-19T01:37:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 --2- 172.21.15.103:0/4188646010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5300103420 0x7f5300109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 --2- 172.21.15.103:0/4188646010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300102a60 0x7f5300102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 --2- 172.21.15.103:0/4188646010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300101870 0x7f5300101c70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 -- 172.21.15.103:0/4188646010 >> 172.21.15.103:0/4188646010 conn(0x7f53000fd010 msgr2=0x7f53000ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 -- 172.21.15.103:0/4188646010 shutdown_connections 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.670+0000 7f530688f640 1 -- 172.21.15.103:0/4188646010 wait complete. 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 Processor -- start 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 -- start start 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5300101870 0x7f5300197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300102a60 0x7f5300198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300103420 0x7f530019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f530010d470 con 0x7f5300102a60 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f530010d2f0 con 0x7f5300101870 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.671+0000 7f530688f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f530010d5f0 con 0x7f5300103420 2024-08-19T01:37:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300102a60 0x7f5300198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300102a60 0x7f5300198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40066/0 (socket says 172.21.15.103:40066) 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 -- 172.21.15.103:0/4036681933 learned_addr learned my addr 172.21.15.103:0/4036681933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52fffff640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5300101870 0x7f5300197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f5304e2c640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300103420 0x7f530019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 -- 172.21.15.103:0/4036681933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300103420 msgr2=0x7f530019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300103420 0x7f530019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 -- 172.21.15.103:0/4036681933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5300101870 msgr2=0x7f5300197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5300101870 0x7f5300197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 -- 172.21.15.103:0/4036681933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530019f6f0 con 0x7f5300102a60 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300102a60 0x7f5300198010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f52f400d770 tx=0x7f52f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52fd7fa640 1 -- 172.21.15.103:0/4036681933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4017070 con 0x7f5300102a60 2024-08-19T01:37:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52fd7fa640 1 -- 172.21.15.103:0/4036681933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52f40054d0 con 0x7f5300102a60 2024-08-19T01:37:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530019f9e0 con 0x7f5300102a60 2024-08-19T01:37:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f52fd7fa640 1 -- 172.21.15.103:0/4036681933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4004cf0 con 0x7f5300102a60 2024-08-19T01:37:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.672+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530019ffb0 con 0x7f5300102a60 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.675+0000 7f52fd7fa640 1 -- 172.21.15.103:0/4036681933 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f52f4004e90 con 0x7f5300102a60 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.675+0000 7f530964c740 1 Processor -- start 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.675+0000 7f530964c740 1 -- start start 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.675+0000 7f52fd7fa640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52e80a2cb0 0x7f52e80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f530964c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81bd6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f530964c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81bdba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f530964c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 0x560c81bde510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f530964c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560c81b399d0 con 0x560c81bd6680 2024-08-19T01:37:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f530964c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560c81ae1820 con 0x560c81bd6fc0 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f530964c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560c81b35990 con 0x560c81bdc050 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fd7fa640 1 -- 172.21.15.103:0/4036681933 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f52f40cea00 con 0x7f5300102a60 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f5304e2c640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52e80a2cb0 0x7f52e80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81bd6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81bd6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40082/0 (socket says 172.21.15.103:40082) 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 -- 172.21.15.103:0/3470105796 learned_addr learned my addr 172.21.15.103:0/3470105796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81bdba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 -- 172.21.15.103:0/3470105796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 msgr2=0x560c81bde510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 0x560c81bde510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 -- 172.21.15.103:0/3470105796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 msgr2=0x560c81bdba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81bdba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 -- 172.21.15.103:0/3470105796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8184d210 con 0x560c81bd6680 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81bdba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f5304e2c640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52e80a2cb0 0x7f52e80a5170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52f8008da0 tx=0x7f52f80087e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.676+0000 7f52fffff640 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81bd6a80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009900 tx=0x7f52f0004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7427f93288149f7d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f52e67fc640 1 -- 172.21.15.103:0/3470105796 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0035030 con 0x560c81bd6680 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f52e67fc640 1 -- 172.21.15.103:0/3470105796 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52f0040070 con 0x560c81bd6680 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f52e67fc640 1 -- 172.21.15.103:0/3470105796 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0031ab0 con 0x560c81bd6680 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f530964c740 1 -- 172.21.15.103:0/3470105796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 msgr2=0x560c81bd6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f530964c740 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81bd6a80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009900 tx=0x7f52f0004310 comp rx=0 tx=0).stop 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f530964c740 1 -- 172.21.15.103:0/3470105796 shutdown_connections 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f530964c740 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 0x560c81bde510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f530964c740 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81bdba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.677+0000 7f530964c740 1 --2- 172.21.15.103:0/3470105796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81bd6a80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 -- 172.21.15.103:0/3470105796 >> 172.21.15.103:0/3470105796 conn(0x560c81bd1cf0 msgr2=0x560c81bd20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 -- 172.21.15.103:0/3470105796 shutdown_connections 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 -- 172.21.15.103:0/3470105796 wait complete. 2024-08-19T01:37:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 -- start start 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 Processor -- start 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 0x560c81bec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560c81be33d0 con 0x560c81bd6680 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560c81af5520 con 0x560c81bd6fc0 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.678+0000 7f530964c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560c81be3550 con 0x560c81bdc050 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f530964c740 10 client.0 _ll_register_callbacks cb 0x560c8184ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40092/0 (socket says 172.21.15.103:40092) 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 -- 172.21.15.103:0/3706070806 learned_addr learned my addr 172.21.15.103:0/3706070806 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f5304e2c640 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 0x560c81bec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 -- 172.21.15.103:0/3706070806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 msgr2=0x560c81bec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 0x560c81bec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 -- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 msgr2=0x560c81be5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 -- 172.21.15.103:0/3706070806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c81919620 con 0x560c81bd6680 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52fffff640 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f52f002aa00 tx=0x7f52f0031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461eca52867001df server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52df7fe640 1 -- 172.21.15.103:0/3706070806 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0035030 con 0x560c81bd6680 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52df7fe640 1 -- 172.21.15.103:0/3706070806 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f52f0045670 con 0x560c81bd6680 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f0040070 con 0x560c81bd6680 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f52df7fe640 1 -- 172.21.15.103:0/3706070806 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52f0045870 con 0x560c81bd6680 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f530964c740 10 client.15906 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.679+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c81beedd0 con 0x560c81bd6680 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f52df7fe640 1 -- 172.21.15.103:0/3706070806 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52f0040070 con 0x560c81bd6680 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f52df7fe640 1 -- 172.21.15.103:0/3706070806 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f52f003cb30 con 0x560c81bd6680 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f530964c740 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f530964c740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f530964c740 20 client.15906 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f530964c740 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81bf0120 0x560c81bf2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f530964c740 4 client.15906 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c81b47100 con 0x560c81bf0120 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.680+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81bf0120 0x560c81bf2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.681+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81bf0120 0x560c81bf2510 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2604f61387848c9 server_cookie=a18b66280794b275 in_seq=0 out_seq=0 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.681+0000 7f52df7fe640 10 client.15906 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.681+0000 7f52df7fe640 1 -- 172.21.15.103:0/3706070806 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c81b47100 con 0x560c81bf0120 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.681+0000 7f52df7fe640 10 client.15906 handle_command_reply: tid=0 2024-08-19T01:37:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.681+0000 7f530964c740 1 client.15906 shutdown 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 msgr2=0x560c81be4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4af0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f52f002aa00 tx=0x7f52f0031ae0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 shutdown_connections 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81bf0120 0x560c81bf2510 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bdc050 0x560c81bec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f52df7fe640 0 client.15906 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 --2- 172.21.15.103:0/3706070806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4af0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 >> 172.21.15.103:0/3706070806 conn(0x560c81bd1cf0 msgr2=0x560c81bd3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 shutdown_connections 2024-08-19T01:37:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 1 -- 172.21.15.103:0/3706070806 wait complete. 2024-08-19T01:37:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.682+0000 7f530964c740 20 client.15906 trim_cache size 0 max 16384 2024-08-19T01:37:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.696+0000 7f530964c740 1 Processor -- start 2024-08-19T01:37:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.696+0000 7f530964c740 1 -- start start 2024-08-19T01:37:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f530964c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f530964c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6fc0 0x560c81bec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f530964c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bdc050 0x560c81bd30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f530964c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53001a6920 con 0x560c81bd6680 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f530964c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53001a84c0 con 0x560c81bdc050 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f530964c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53001a6ff0 con 0x560c81bd6fc0 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40094/0 (socket says 172.21.15.103:40094) 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 -- 172.21.15.103:0/2482489449 learned_addr learned my addr 172.21.15.103:0/2482489449 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6fc0 0x560c81bec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f5304e2c640 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bdc050 0x560c81bd30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 -- 172.21.15.103:0/2482489449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6fc0 msgr2=0x560c81bec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6fc0 0x560c81bec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 -- 172.21.15.103:0/2482489449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bdc050 msgr2=0x560c81bd30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bdc050 0x560c81bd30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 -- 172.21.15.103:0/2482489449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c81bd15b0 con 0x560c81bd6680 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f5304e2c640 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bdc050 0x560c81bd30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52fffff640 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4f10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f52f002a930 tx=0x7f52f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaf848bb193143e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52e5ffb640 1 -- 172.21.15.103:0/2482489449 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0035070 con 0x560c81bd6680 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.697+0000 7f52e5ffb640 1 -- 172.21.15.103:0/2482489449 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52f005d020 con 0x560c81bd6680 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f52e5ffb640 1 -- 172.21.15.103:0/2482489449 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f003c870 con 0x560c81bd6680 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 -- 172.21.15.103:0/2482489449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 msgr2=0x560c81be4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4f10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f52f002a930 tx=0x7f52f0000bc0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 -- 172.21.15.103:0/2482489449 shutdown_connections 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bdc050 0x560c81bd30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6fc0 0x560c81bec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 --2- 172.21.15.103:0/2482489449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bd6680 0x560c81be4f10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 -- 172.21.15.103:0/2482489449 >> 172.21.15.103:0/2482489449 conn(0x560c81bd1cf0 msgr2=0x560c81bd0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 -- 172.21.15.103:0/2482489449 shutdown_connections 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.698+0000 7f530964c740 1 -- 172.21.15.103:0/2482489449 wait complete. 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 -- start start 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 Processor -- start 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6680 0x560c81be9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bdc050 0x560c81beab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53001a7dc0 con 0x560c81bdc050 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53001a76c0 con 0x560c81bd6fc0 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f53001a4900 con 0x560c81bd6680 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f530964c740 10 client.0 _ll_register_callbacks cb 0x7f52f0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bdc050 0x560c81beab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bdc050 0x560c81beab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40102/0 (socket says 172.21.15.103:40102) 2024-08-19T01:37:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 -- 172.21.15.103:0/3486086829 learned_addr learned my addr 172.21.15.103:0/3486086829 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f52fffff640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6680 0x560c81be9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 -- 172.21.15.103:0/3486086829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6680 msgr2=0x560c81be9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6680 0x560c81be9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 -- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 msgr2=0x560c81be9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.699+0000 7f5304e2c640 1 -- 172.21.15.103:0/3486086829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c81beb170 con 0x560c81bdc050 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f52ff7fe640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f5304e2c640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bdc050 0x560c81beab60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f52f8014c20 tx=0x7f52f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2413f25ea2cb131f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f52df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f52df7fe640 1 -- 172.21.15.103:0/3486086829 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8014c50 con 0x560c81bdc050 2024-08-19T01:37:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f52df7fe640 1 -- 172.21.15.103:0/3486086829 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f52f8014df0 con 0x560c81bdc050 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f005d020 con 0x560c81bdc050 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f52df7fe640 1 -- 172.21.15.103:0/3486086829 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52f802e460 con 0x560c81bdc050 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f530964c740 10 client.15912 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c81c5ad50 con 0x560c81bdc050 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f52df7fe640 1 -- 172.21.15.103:0/3486086829 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52f005d020 con 0x560c81bdc050 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f52df7fe640 1 -- 172.21.15.103:0/3486086829 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f52f8022660 con 0x560c81bdc050 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f530964c740 10 client.15912 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.700+0000 7f530964c740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.701+0000 7f530964c740 20 client.15912 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.701+0000 7f530964c740 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81c5b6d0 0x560c81c5daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.701+0000 7f530964c740 4 client.15912 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:37:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.701+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c81b41450 con 0x560c81c5b6d0 2024-08-19T01:37:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.701+0000 7f52fffff640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81c5b6d0 0x560c81c5daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.701+0000 7f52df7fe640 10 client.15912 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.701+0000 7f52fffff640 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81c5b6d0 0x560c81c5daf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8514e6922a41560 server_cookie=db538758246ada8c in_seq=0 out_seq=0 2024-08-19T01:37:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f52df7fe640 1 -- 172.21.15.103:0/3486086829 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c81b41450 con 0x560c81c5b6d0 2024-08-19T01:37:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f52df7fe640 10 client.15912 handle_command_reply: tid=0 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 client.15912 shutdown 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bdc050 msgr2=0x560c81beab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bdc050 0x560c81beab60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f52f8014c20 tx=0x7f52f8004a40 comp rx=0 tx=0).stop 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 shutdown_connections 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560c81c5b6d0 0x560c81c5daf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c81bdc050 0x560c81beab60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560c81bd6fc0 0x560c81be9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 --2- 172.21.15.103:0/3486086829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c81bd6680 0x560c81be9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 >> 172.21.15.103:0/3486086829 conn(0x560c81bd1cf0 msgr2=0x560c81bd0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f52df7fe640 0 client.15912 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 shutdown_connections 2024-08-19T01:37:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 1 -- 172.21.15.103:0/3486086829 wait complete. 2024-08-19T01:37:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.709+0000 7f530964c740 20 client.15912 trim_cache size 0 max 16384 2024-08-19T01:37:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:37:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:37:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:37:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52e80a2cb0 msgr2=0x7f52e80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52e80a2cb0 0x7f52e80a5170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52f8008da0 tx=0x7f52f80087e0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300102a60 msgr2=0x7f5300198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300102a60 0x7f5300198010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f52f400d770 tx=0x7f52f400dc30 comp rx=0 tx=0).stop 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 shutdown_connections 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f52e80a2cb0 0x7f52e80a5170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5300103420 0x7f530019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5300102a60 0x7f5300198010 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 --2- 172.21.15.103:0/4036681933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5300101870 0x7f5300197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 >> 172.21.15.103:0/4036681933 conn(0x7f53000fd010 msgr2=0x7f5300105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 shutdown_connections 2024-08-19T01:37:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:37:52.710+0000 7f530688f640 1 -- 172.21.15.103:0/4036681933 wait complete. 2024-08-19T01:37:53.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 -- 172.21.15.103:0/2200664147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 msgr2=0x7f5cf810d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/2200664147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf810d670 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009cd0 tx=0x7f5cf002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 -- 172.21.15.103:0/2200664147 shutdown_connections 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/2200664147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf810d670 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/2200664147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf81007a0 0x7f5cf8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/2200664147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf81067a0 0x7f5cf8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 -- 172.21.15.103:0/2200664147 >> 172.21.15.103:0/2200664147 conn(0x7f5cf80fc4c0 msgr2=0x7f5cf80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 -- 172.21.15.103:0/2200664147 shutdown_connections 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.112+0000 7f5cff3aa640 1 -- 172.21.15.103:0/2200664147 wait complete. 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 Processor -- start 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 -- start start 2024-08-19T01:37:53.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf81007a0 0x7f5cf80fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf80fd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf81067a0 0x7f5cf80fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110a50 con 0x7f5cf8101140 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81108d0 con 0x7f5cf81007a0 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cff3aa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8110bd0 con 0x7f5cf81067a0 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cfc945640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf80fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cfc945640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf80fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40114/0 (socket says 172.21.15.103:40114) 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cfc945640 1 -- 172.21.15.103:0/1079997731 learned_addr learned my addr 172.21.15.103:0/1079997731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.113+0000 7f5cfd146640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf81007a0 0x7f5cf80fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfc945640 1 -- 172.21.15.103:0/1079997731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf81067a0 msgr2=0x7f5cf80fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfd947640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf81067a0 0x7f5cf80fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfc945640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf81067a0 0x7f5cf80fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfc945640 1 -- 172.21.15.103:0/1079997731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf81007a0 msgr2=0x7f5cf80fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfc945640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf81007a0 0x7f5cf80fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfc945640 1 -- 172.21.15.103:0/1079997731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81a6500 con 0x7f5cf8101140 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfd947640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf81067a0 0x7f5cf80fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfd146640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf81007a0 0x7f5cf80fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cfc945640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf80fd090 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800d770 tx=0x7f5ce800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cee7fc640 1 -- 172.21.15.103:0/1079997731 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8017070 con 0x7f5cf8101140 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81a67f0 con 0x7f5cf8101140 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cee7fc640 1 -- 172.21.15.103:0/1079997731 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce80054d0 con 0x7f5cf8101140 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cee7fc640 1 -- 172.21.15.103:0/1079997731 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8004cf0 con 0x7f5cf8101140 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.114+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a6d30 con 0x7f5cf8101140 2024-08-19T01:37:53.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.116+0000 7f5cee7fc640 1 -- 172.21.15.103:0/1079997731 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5ce8004e90 con 0x7f5cf8101140 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.117+0000 7f5cee7fc640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5ce40a2b80 0x7f5ce40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.117+0000 7f5d017a8740 1 Processor -- start 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.117+0000 7f5d017a8740 1 -- start start 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.117+0000 7f5cee7fc640 1 -- 172.21.15.103:0/1079997731 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5ce80ceb90 con 0x7f5cf8101140 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5ce40a2b80 0x7f5ce40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c20805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080b30 0x55d7c208aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c208d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5d017a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7c1fe5980 con 0x55d7c208b3f0 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5d017a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d7c1f7da70 con 0x55d7c2080b30 2024-08-19T01:37:53.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5d017a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d7c1fe1940 con 0x55d7c2080210 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c20805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c20805f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:37018/0 (socket says 172.21.15.103:37018) 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 -- 172.21.15.103:0/2417357314 learned_addr learned my addr 172.21.15.103:0/2417357314 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfc945640 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080b30 0x55d7c208aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd947640 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c208d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5ce40a2b80 0x7f5ce40a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4005e00 tx=0x7f5cf400f360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 -- 172.21.15.103:0/2417357314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080b30 msgr2=0x55d7c208aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080b30 0x55d7c208aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 -- 172.21.15.103:0/2417357314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 msgr2=0x55d7c208d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.118+0000 7f5cfd146640 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c208d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.119+0000 7f5cfd146640 1 -- 172.21.15.103:0/2417357314 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c1fe06e0 con 0x55d7c2080210 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.119+0000 7f5cfd947640 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c208d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.119+0000 7f5cfd146640 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c20805f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ffd0 tx=0x7f5cf401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576f4ab239664f62 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.119+0000 7f5ce37fe640 1 -- 172.21.15.103:0/2417357314 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf401d620 con 0x55d7c2080210 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.119+0000 7f5ce37fe640 1 -- 172.21.15.103:0/2417357314 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cf401dc70 con 0x55d7c2080210 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.119+0000 7f5ce37fe640 1 -- 172.21.15.103:0/2417357314 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4010bb0 con 0x55d7c2080210 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 -- 172.21.15.103:0/2417357314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 msgr2=0x55d7c20805f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c20805f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ffd0 tx=0x7f5cf401a000 comp rx=0 tx=0).stop 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 -- 172.21.15.103:0/2417357314 shutdown_connections 2024-08-19T01:37:53.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c208d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080b30 0x55d7c208aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 --2- 172.21.15.103:0/2417357314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c20805f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 -- 172.21.15.103:0/2417357314 >> 172.21.15.103:0/2417357314 conn(0x55d7c207c7a0 msgr2=0x55d7c207cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 -- 172.21.15.103:0/2417357314 shutdown_connections 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 -- 172.21.15.103:0/2417357314 wait complete. 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.120+0000 7f5d017a8740 1 -- start start 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 1 Processor -- start 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c2088250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2088790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c208b3f0 0x55d7c2088cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7c208e010 con 0x55d7c2080b30 2024-08-19T01:37:53.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d7c1fa0aa0 con 0x55d7c208b3f0 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d7c208e190 con 0x55d7c2080210 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5d017a8740 10 client.0 _ll_register_callbacks cb 0x55d7c1f8fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5cfd146640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c2088250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5cfd146640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c2088250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:37028/0 (socket says 172.21.15.103:37028) 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5cfd146640 1 -- 172.21.15.103:0/3948522654 learned_addr learned my addr 172.21.15.103:0/3948522654 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5cfc945640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2088790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.121+0000 7f5cfd947640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c208b3f0 0x55d7c2088cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5cfd146640 1 -- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c208b3f0 msgr2=0x55d7c2088cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5cfd146640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c208b3f0 0x55d7c2088cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5cfd146640 1 -- 172.21.15.103:0/3948522654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 msgr2=0x55d7c2088790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5cfd146640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2088790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5cfd146640 1 -- 172.21.15.103:0/3948522654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c1f84650 con 0x55d7c2080210 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5cfc945640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2088790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5cfd146640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c2088250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4010b60 tx=0x7f5cf4010b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f019005434f166 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5ce0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:53.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/3948522654 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf401d420 con 0x55d7c2080210 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf401dc70 con 0x55d7c2080210 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.122+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/3948522654 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5cf401d5c0 con 0x55d7c2080210 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/3948522654 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5cf401d7c0 con 0x55d7c2080210 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 10 client.25399 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7c1d573c0 con 0x55d7c2080210 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/3948522654 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cf401dc70 con 0x55d7c2080210 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/3948522654 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5cf4039ad0 con 0x55d7c2080210 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 10 client.25399 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:53.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 20 client.25399 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c209ade0 0x55d7c209d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 4 client.25399 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.123+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c1ff2ef0 con 0x55d7c209ade0 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.124+0000 7f5cfc945640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c209ade0 0x55d7c209d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.124+0000 7f5cfc945640 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c209ade0 0x55d7c209d1d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59f039d3ed4200d server_cookie=2506f60543e95c6 in_seq=0 out_seq=0 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.124+0000 7f5ce0ff9640 10 client.25399 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/3948522654 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7c1ff2ef0 con 0x55d7c209ade0 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5ce0ff9640 10 client.25399 handle_command_reply: tid=0 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 client.25399 shutdown 2024-08-19T01:37:53.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 msgr2=0x55d7c2088250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c2088250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4010b60 tx=0x7f5cf4010b90 comp rx=0 tx=0).stop 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 shutdown_connections 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c209ade0 0x55d7c209d1d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c208b3f0 0x55d7c2088cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2088790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 --2- 172.21.15.103:0/3948522654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080210 0x55d7c2088250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 >> 172.21.15.103:0/3948522654 conn(0x55d7c207c7a0 msgr2=0x55d7c2082480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5ce0ff9640 0 client.25399 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 shutdown_connections 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 1 -- 172.21.15.103:0/3948522654 wait complete. 2024-08-19T01:37:53.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.125+0000 7f5d017a8740 20 client.25399 trim_cache size 0 max 16384 2024-08-19T01:37:53.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.148+0000 7f5d017a8740 1 Processor -- start 2024-08-19T01:37:53.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.148+0000 7f5d017a8740 1 -- start start 2024-08-19T01:37:53.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.148+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c208a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.148+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080b30 0x55d7c20a2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c20a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5d017a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81aaf00 con 0x55d7c208b3f0 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5d017a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81acaa0 con 0x55d7c2080210 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5d017a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ab5d0 con 0x55d7c2080b30 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c20a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c20a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40150/0 (socket says 172.21.15.103:40150) 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 -- 172.21.15.103:0/4039031160 learned_addr learned my addr 172.21.15.103:0/4039031160 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd146640 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c208a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfc945640 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080b30 0x55d7c20a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 -- 172.21.15.103:0/4039031160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080b30 msgr2=0x55d7c20a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080b30 0x55d7c20a2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 -- 172.21.15.103:0/4039031160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 msgr2=0x55d7c208a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c208a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 -- 172.21.15.103:0/4039031160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c207c5d0 con 0x55d7c208b3f0 2024-08-19T01:37:53.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.149+0000 7f5cfd947640 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c20a2dc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009cd0 tx=0x7f5cf0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26f3fae41c21c49 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:53.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5ce2ffd640 1 -- 172.21.15.103:0/4039031160 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0040d20 con 0x55d7c208b3f0 2024-08-19T01:37:53.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5ce2ffd640 1 -- 172.21.15.103:0/4039031160 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cf0036040 con 0x55d7c208b3f0 2024-08-19T01:37:53.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5ce2ffd640 1 -- 172.21.15.103:0/4039031160 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00042a0 con 0x55d7c208b3f0 2024-08-19T01:37:53.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 -- 172.21.15.103:0/4039031160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 msgr2=0x55d7c20a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c20a2dc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009cd0 tx=0x7f5cf0005d80 comp rx=0 tx=0).stop 2024-08-19T01:37:53.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 -- 172.21.15.103:0/4039031160 shutdown_connections 2024-08-19T01:37:53.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c208b3f0 0x55d7c20a2dc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c2080b30 0x55d7c20a2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4039031160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c208a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 -- 172.21.15.103:0/4039031160 >> 172.21.15.103:0/4039031160 conn(0x55d7c207c7a0 msgr2=0x55d7c2082480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.150+0000 7f5d017a8740 1 -- 172.21.15.103:0/4039031160 shutdown_connections 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 -- 172.21.15.103:0/4039031160 wait complete. 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 -- start start 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 Processor -- start 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c20935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2093b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c208b3f0 0x55d7c2084ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81ac3a0 con 0x55d7c2080b30 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81abca0 con 0x55d7c2080210 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81a8ee0 con 0x55d7c208b3f0 2024-08-19T01:37:53.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.151+0000 7f5d017a8740 10 client.0 _ll_register_callbacks cb 0x7f5cf0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2093b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfd146640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c20935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2093b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40158/0 (socket says 172.21.15.103:40158) 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 -- 172.21.15.103:0/4263093300 learned_addr learned my addr 172.21.15.103:0/4263093300 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfd947640 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c208b3f0 0x55d7c2084ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 -- 172.21.15.103:0/4263093300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c208b3f0 msgr2=0x55d7c2084ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c208b3f0 0x55d7c2084ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 -- 172.21.15.103:0/4263093300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 msgr2=0x55d7c20935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c20935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 -- 172.21.15.103:0/4263093300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c2085190 con 0x55d7c2080b30 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5cfc945640 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2093b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5ce809b0c0 tx=0x7f5ce80ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b694ca8cead0ecf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5ce0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:53.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/4263093300 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8020040 con 0x55d7c2080b30 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/4263093300 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5ce80996c0 con 0x55d7c2080b30 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.152+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf0036040 con 0x55d7c2080b30 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/4263093300 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ce809f770 con 0x55d7c2080b30 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 10 client.15930 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7c21051e0 con 0x55d7c2080b30 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/4263093300 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cf0036040 con 0x55d7c2080b30 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/4263093300 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5ce80ccb20 con 0x55d7c2080b30 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 20 client.15930 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c21059c0 0x55d7c2107de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 4 client.15930 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7c1fed400 con 0x55d7c21059c0 2024-08-19T01:37:53.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.153+0000 7f5cfd146640 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c21059c0 0x55d7c2107de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:53.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5ce0ff9640 10 client.15930 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:53.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5cfd146640 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c21059c0 0x55d7c2107de0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff646f02b7da77f9 server_cookie=10be6473447076b7 in_seq=0 out_seq=0 2024-08-19T01:37:53.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5ce0ff9640 1 -- 172.21.15.103:0/4263093300 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7c1fed400 con 0x55d7c21059c0 2024-08-19T01:37:53.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5ce0ff9640 10 client.15930 handle_command_reply: tid=0 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5d017a8740 1 client.15930 shutdown 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 msgr2=0x55d7c2093b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2093b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5ce809b0c0 tx=0x7f5ce80ce040 comp rx=0 tx=0).stop 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 shutdown_connections 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.154+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d7c21059c0 0x55d7c2107de0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d7c208b3f0 0x55d7c2084ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5ce0ff9640 0 client.15930 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c2080b30 0x55d7c2093b00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5d017a8740 1 --2- 172.21.15.103:0/4263093300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d7c2080210 0x55d7c20935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 >> 172.21.15.103:0/4263093300 conn(0x55d7c207c7a0 msgr2=0x55d7c2081c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 shutdown_connections 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5d017a8740 1 -- 172.21.15.103:0/4263093300 wait complete. 2024-08-19T01:37:53.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.155+0000 7f5d017a8740 20 client.15930 trim_cache size 0 max 16384 2024-08-19T01:37:53.161 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:37:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:37:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:37:53.161 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:37:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5ce40a2b80 msgr2=0x7f5ce40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5ce40a2b80 0x7f5ce40a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4005e00 tx=0x7f5cf400f360 comp rx=0 tx=0).stop 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 msgr2=0x7f5cf80fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf80fd090 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800d770 tx=0x7f5ce800dc30 comp rx=0 tx=0).stop 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 shutdown_connections 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5ce40a2b80 0x7f5ce40a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cf81067a0 0x7f5cf80fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cf8101140 0x7f5cf80fd090 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 --2- 172.21.15.103:0/1079997731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5cf81007a0 0x7f5cf80fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 >> 172.21.15.103:0/1079997731 conn(0x7f5cf80fc4c0 msgr2=0x7f5cf806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 shutdown_connections 2024-08-19T01:37:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:53.156+0000 7f5cff3aa640 1 -- 172.21.15.103:0/1079997731 wait complete. 2024-08-19T01:37:54.048 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:53 smithi103 ceph-mon[18436]: pgmap v402: 97 pgs: 97 active+clean; 2.0 GiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:37:54.089 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:37:54.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:37:54.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:53 smithi146 ceph-mon[22098]: pgmap v402: 97 pgs: 97 active+clean; 2.0 GiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:37:54.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:53 smithi190 ceph-mon[22032]: pgmap v402: 97 pgs: 97 active+clean; 2.0 GiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:37:54.444 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 -- 172.21.15.103:0/4238047655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 msgr2=0x7fd2141078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 --2- 172.21.15.103:0/4238047655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141078b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd204009930 tx=0x7fd20402a420 comp rx=0 tx=0).stop 2024-08-19T01:37:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 -- 172.21.15.103:0/4238047655 shutdown_connections 2024-08-19T01:37:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 --2- 172.21.15.103:0/4238047655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd214107fc0 0x7fd21410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 --2- 172.21.15.103:0/4238047655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141078b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 --2- 172.21.15.103:0/4238047655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd214102ac0 0x7fd214102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 -- 172.21.15.103:0/4238047655 >> 172.21.15.103:0/4238047655 conn(0x7fd2140fc500 msgr2=0x7fd2140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 -- 172.21.15.103:0/4238047655 shutdown_connections 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 -- 172.21.15.103:0/4238047655 wait complete. 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.971+0000 7fd21c44b640 1 Processor -- start 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21c44b640 1 -- start start 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21c44b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd214102ac0 0x7fd214198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21c44b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21c44b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd214107fc0 0x7fd21419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21c44b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd214111c90 con 0x7fd214103470 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21c44b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd214111b10 con 0x7fd214107fc0 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21c44b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd214111e10 con 0x7fd214102ac0 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40168/0 (socket says 172.21.15.103:40168) 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 -- 172.21.15.103:0/3794423556 learned_addr learned my addr 172.21.15.103:0/3794423556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21a9e8640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd214107fc0 0x7fd21419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:54.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd21a1e7640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd214102ac0 0x7fd214198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 -- 172.21.15.103:0/3794423556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd214102ac0 msgr2=0x7fd214198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd214102ac0 0x7fd214198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 -- 172.21.15.103:0/3794423556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd214107fc0 msgr2=0x7fd21419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd214107fc0 0x7fd21419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 -- 172.21.15.103:0/3794423556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21419d220 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.972+0000 7fd2199e6640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141987b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd204031440 tx=0x7fd204004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.973+0000 7fd20b7fe640 1 -- 172.21.15.103:0/3794423556 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20403f030 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.973+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21419d4b0 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.973+0000 7fd20b7fe640 1 -- 172.21.15.103:0/3794423556 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd204003aa0 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.973+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2141a4df0 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.973+0000 7fd20b7fe640 1 -- 172.21.15.103:0/3794423556 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204035070 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.974+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd214102ea0 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.975+0000 7fd20b7fe640 1 -- 172.21.15.103:0/3794423556 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd204042030 con 0x7fd214103470 2024-08-19T01:37:54.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.976+0000 7fd20b7fe640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd2000a2b80 0x7fd2000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:54.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.976+0000 7fd21a1e7640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd2000a2b80 0x7fd2000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:54.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.976+0000 7fd20b7fe640 1 -- 172.21.15.103:0/3794423556 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd2040ef4c0 con 0x7fd214103470 2024-08-19T01:37:54.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.977+0000 7fd21a1e7640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd2000a2b80 0x7fd2000a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd210005fd0 tx=0x7fd210005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:54.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:54.982+0000 7fd20b7fe640 1 -- 172.21.15.103:0/3794423556 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd2040bc450 con 0x7fd214103470 2024-08-19T01:37:55.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.222+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd214109fd0 con 0x7fd214103470 2024-08-19T01:37:55.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.223+0000 7fd20b7fe640 1 -- 172.21.15.103:0/3794423556 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11991 (secure 0 0 0) 0x7fd2040bbd50 con 0x7fd214103470 2024-08-19T01:37:55.224 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:37:55.224 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-08-19T01:37:36:616371+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:37:36.566253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:37:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-08-19T01:37:55.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd2000a2b80 msgr2=0x7fd2000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:55.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd2000a2b80 0x7fd2000a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd210005fd0 tx=0x7fd210005850 comp rx=0 tx=0).stop 2024-08-19T01:37:55.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 msgr2=0x7fd2141987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:55.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141987b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd204031440 tx=0x7fd204004270 comp rx=0 tx=0).stop 2024-08-19T01:37:55.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 shutdown_connections 2024-08-19T01:37:55.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd2000a2b80 0x7fd2000a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:55.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd214107fc0 0x7fd21419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:55.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd214103470 0x7fd2141987b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:55.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 --2- 172.21.15.103:0/3794423556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd214102ac0 0x7fd214198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:55.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.225+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 >> 172.21.15.103:0/3794423556 conn(0x7fd2140fc500 msgr2=0x7fd214105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:55.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.226+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 shutdown_connections 2024-08-19T01:37:55.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:55.226+0000 7fd21c44b640 1 -- 172.21.15.103:0/3794423556 wait complete. 2024-08-19T01:37:56.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:37:56.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:55 smithi146 ceph-mon[22098]: pgmap v403: 97 pgs: 97 active+clean; 2.0 GiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:37:56.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:55 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3794423556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:56.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:55 smithi103 ceph-mon[18436]: pgmap v403: 97 pgs: 97 active+clean; 2.0 GiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:37:56.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:55 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3794423556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:56.346 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:56.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:55 smithi190 ceph-mon[22032]: pgmap v403: 97 pgs: 97 active+clean; 2.0 GiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:37:56.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:55 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3794423556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:37:56.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 -- 172.21.15.103:0/1838419564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8102ae0 msgr2=0x7f72c8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/1838419564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8102ae0 0x7f72c8102ec0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009cd0 tx=0x7f72b802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 -- 172.21.15.103:0/1838419564 shutdown_connections 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/1838419564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c8108010 0x7f72c810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/1838419564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c8103490 0x7f72c81078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/1838419564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8102ae0 0x7f72c8102ec0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 -- 172.21.15.103:0/1838419564 >> 172.21.15.103:0/1838419564 conn(0x7f72c80fc500 msgr2=0x7f72c80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 -- 172.21.15.103:0/1838419564 shutdown_connections 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.906+0000 7f72cd8a8640 1 -- 172.21.15.103:0/1838419564 wait complete. 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 Processor -- start 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 -- start start 2024-08-19T01:37:56.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c8102ae0 0x7f72c8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c8103490 0x7f72c81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8108010 0x7f72c819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8111cf0 con 0x7f72c8108010 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8111b70 con 0x7f72c8102ae0 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72cd8a8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8111e70 con 0x7f72c8103490 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72c77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8108010 0x7f72c819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72c77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8108010 0x7f72c819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40176/0 (socket says 172.21.15.103:40176) 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72c77fe640 1 -- 172.21.15.103:0/398196316 learned_addr learned my addr 172.21.15.103:0/398196316 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.907+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c8102ae0 0x7f72c8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72bffff640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c8103490 0x7f72c81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72c77fe640 1 -- 172.21.15.103:0/398196316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c8103490 msgr2=0x7f72c81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72c77fe640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c8103490 0x7f72c81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72c77fe640 1 -- 172.21.15.103:0/398196316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c8102ae0 msgr2=0x7f72c8198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72c77fe640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c8102ae0 0x7f72c8198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72c77fe640 1 -- 172.21.15.103:0/398196316 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c819d230 con 0x7f72c8108010 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72c77fe640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8108010 0x7f72c819cb50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009fd0 tx=0x7f72c000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72cc8a6640 1 -- 172.21.15.103:0/398196316 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0011830 con 0x7f72c8108010 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72cc8a6640 1 -- 172.21.15.103:0/398196316 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72c0016070 con 0x7f72c8108010 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c819d520 con 0x7f72c8108010 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72cc8a6640 1 -- 172.21.15.103:0/398196316 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0010de0 con 0x7f72c8108010 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.908+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c81a4e00 con 0x7f72c8108010 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cc8a6640 1 -- 172.21.15.103:0/398196316 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f72c00119d0 con 0x7f72c8108010 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 Processor -- start 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 -- start start 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06b953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 0x557f06ba25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f06afaa40 con 0x557f06b95910 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557f06a929d0 con 0x557f06ba01d0 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.911+0000 7f72cfca6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557f06af6a00 con 0x557f06b94ff0 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40180/0 (socket says 172.21.15.103:40180) 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2056139867 learned_addr learned my addr 172.21.15.103:0/2056139867 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72bffff640 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06b953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72bffff640 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 0x557f06ba25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72cc8a6640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72b40a2b80 0x7f72b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2056139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 msgr2=0x557f06b953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06b953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2056139867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 msgr2=0x557f06ba25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 0x557f06ba25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2056139867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f06a99650 con 0x557f06b95910 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72cc8a6640 1 -- 172.21.15.103:0/398196316 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f72c00ce720 con 0x7f72c8108010 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.912+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9fc90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009cd0 tx=0x7f72b80314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8322b62caf4375 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.913+0000 7f72c77fe640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72b40a2b80 0x7f72b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.913+0000 7f72bdffb640 1 -- 172.21.15.103:0/2056139867 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8031aa0 con 0x557f06b95910 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.913+0000 7f72bdffb640 1 -- 172.21.15.103:0/2056139867 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72b8036040 con 0x557f06b95910 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.913+0000 7f72bdffb640 1 -- 172.21.15.103:0/2056139867 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80046d0 con 0x557f06b95910 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 -- 172.21.15.103:0/2056139867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 msgr2=0x557f06b9fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9fc90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009cd0 tx=0x7f72b80314e0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72c77fe640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72b40a2b80 0x7f72b40a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72c0097410 tx=0x7f72c00047d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 -- 172.21.15.103:0/2056139867 shutdown_connections 2024-08-19T01:37:56.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 0x557f06ba25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9fc90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2056139867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06b953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 -- 172.21.15.103:0/2056139867 >> 172.21.15.103:0/2056139867 conn(0x557f06b91580 msgr2=0x557f06b91960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 -- 172.21.15.103:0/2056139867 shutdown_connections 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 -- 172.21.15.103:0/2056139867 wait complete. 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.914+0000 7f72cfca6740 1 -- start start 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72cfca6740 1 Processor -- start 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06ba01d0 0x557f06b9dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72cfca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f06ba2da0 con 0x557f06b95910 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72cfca6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557f06ab5b20 con 0x557f06b94ff0 2024-08-19T01:37:56.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72bffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40190/0 (socket says 172.21.15.103:40190) 2024-08-19T01:37:56.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.915+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2371065171 learned_addr learned my addr 172.21.15.103:0/2371065171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:56.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557f06ba2f20 con 0x557f06ba01d0 2024-08-19T01:37:56.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72bffff640 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06ba01d0 0x557f06b9dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72cfca6740 10 client.0 _ll_register_callbacks cb 0x557f06ab3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:56.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72bffff640 1 -- 172.21.15.103:0/2371065171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 msgr2=0x557f06b9d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72bffff640 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72bffff640 1 -- 172.21.15.103:0/2371065171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 msgr2=0x557f06b9d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72bffff640 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72bffff640 1 -- 172.21.15.103:0/2371065171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0686c3c0 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72bffff640 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06ba01d0 0x557f06b9dad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557f06b92900 tx=0x7f72b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cffaa3f1e05b9fc7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.916+0000 7f72a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.917+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2371065171 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0004260 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.917+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2371065171 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f72b0004400 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.917+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b8036040 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.917+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2371065171 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72b0026050 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.917+0000 7f72cfca6740 10 client.25417 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.917+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f06bae930 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.917+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2371065171 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72b8036040 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2371065171 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f72b000d3a0 con 0x557f06ba01d0 2024-08-19T01:37:56.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72cfca6740 10 client.25417 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72cfca6740 10 client.25417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72cfca6740 20 client.25417 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06bafce0 0x557f06bb2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72cfca6740 4 client.25417 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f06b07fb0 con 0x557f06bafce0 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.918+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06bafce0 0x557f06bb2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.919+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06bafce0 0x557f06bb2150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89489855d4f22193 server_cookie=d7754d074e5853c in_seq=0 out_seq=0 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.919+0000 7f72a6ffd640 10 client.25417 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.919+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2371065171 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f06b07fb0 con 0x557f06bafce0 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.919+0000 7f72a6ffd640 10 client.25417 handle_command_reply: tid=0 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.919+0000 7f72cfca6740 1 client.25417 shutdown 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06ba01d0 msgr2=0x557f06b9dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06ba01d0 0x557f06b9dad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557f06b92900 tx=0x7f72b00046c0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 shutdown_connections 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06bafce0 0x557f06bb2150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06ba01d0 0x557f06b9dad0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06b9d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2371065171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72a6ffd640 0 client.25417 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 >> 172.21.15.103:0/2371065171 conn(0x557f06b91580 msgr2=0x557f06b97260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 shutdown_connections 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 1 -- 172.21.15.103:0/2371065171 wait complete. 2024-08-19T01:37:56.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.920+0000 7f72cfca6740 20 client.25417 trim_cache size 0 max 16384 2024-08-19T01:37:56.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.935+0000 7f72cfca6740 1 Processor -- start 2024-08-19T01:37:56.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.935+0000 7f72cfca6740 1 -- start start 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.935+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06bb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.935+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06bb7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 0x557f06b9b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72cfca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81ad1e0 con 0x557f06b95910 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06bb7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72cfca6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81aed80 con 0x557f06ba01d0 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72cfca6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81ad8b0 con 0x557f06b94ff0 2024-08-19T01:37:56.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06bb7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40206/0 (socket says 172.21.15.103:40206) 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 -- 172.21.15.103:0/407247300 learned_addr learned my addr 172.21.15.103:0/407247300 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72bffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06bb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 -- 172.21.15.103:0/407247300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 msgr2=0x557f06bb7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/407247300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06bb7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 -- 172.21.15.103:0/407247300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 msgr2=0x557f06b9b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/407247300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 0x557f06b9b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 -- 172.21.15.103:0/407247300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f06b9fb00 con 0x557f06b95910 2024-08-19T01:37:56.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.936+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/407247300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06bb7fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009410 tx=0x7f72b803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe980ae1658860f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:56.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.939+0000 7f72bd7fa640 1 -- 172.21.15.103:0/407247300 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8004720 con 0x557f06b95910 2024-08-19T01:37:56.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.939+0000 7f72bd7fa640 1 -- 172.21.15.103:0/407247300 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72b8036070 con 0x557f06b95910 2024-08-19T01:37:56.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.939+0000 7f72bd7fa640 1 -- 172.21.15.103:0/407247300 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80523b0 con 0x557f06b95910 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 -- 172.21.15.103:0/407247300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 msgr2=0x557f06bb7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 --2- 172.21.15.103:0/407247300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06bb7fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009410 tx=0x7f72b803e040 comp rx=0 tx=0).stop 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 -- 172.21.15.103:0/407247300 shutdown_connections 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 --2- 172.21.15.103:0/407247300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06ba01d0 0x557f06b9b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 --2- 172.21.15.103:0/407247300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06b95910 0x557f06bb7fb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 --2- 172.21.15.103:0/407247300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b94ff0 0x557f06bb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 -- 172.21.15.103:0/407247300 >> 172.21.15.103:0/407247300 conn(0x557f06b91580 msgr2=0x557f06b8c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 -- 172.21.15.103:0/407247300 shutdown_connections 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 -- 172.21.15.103:0/407247300 wait complete. 2024-08-19T01:37:56.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.940+0000 7f72cfca6740 1 -- start start 2024-08-19T01:37:56.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 1 Processor -- start 2024-08-19T01:37:56.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b95910 0x557f06b9a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06ba01d0 0x557f06bb6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81ae680 con 0x557f06ba01d0 2024-08-19T01:37:56.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81adf80 con 0x557f06b94ff0 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81ab1c0 con 0x557f06b95910 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72cfca6740 10 client.0 _ll_register_callbacks cb 0x7f72b000ec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b95910 0x557f06b9a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72c6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b95910 0x557f06b9a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:37096/0 (socket says 172.21.15.103:37096) 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2234070412 learned_addr learned my addr 172.21.15.103:0/2234070412 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72bffff640 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06ba01d0 0x557f06bb6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.941+0000 7f72bffff640 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 msgr2=0x557f06b9a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2234070412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06ba01d0 msgr2=0x557f06bb6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06ba01d0 0x557f06bb6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72c6ffd640 1 -- 172.21.15.103:0/2234070412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f06bb7180 con 0x557f06b95910 2024-08-19T01:37:56.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72c6ffd640 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b95910 0x557f06b9a810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f72b80093e0 tx=0x7f72b80041d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3a1f1d30bc8a71 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:56.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:37:56.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2234070412 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b805f020 con 0x557f06b95910 2024-08-19T01:37:56.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2234070412 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f72b8003d40 con 0x557f06b95910 2024-08-19T01:37:56.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.942+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b8036070 con 0x557f06b95910 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2234070412 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72b8003970 con 0x557f06b95910 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 10 client.25423 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f06c1e150 con 0x557f06b95910 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2234070412 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72b8036070 con 0x557f06b95910 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2234070412 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f72b8030070 con 0x557f06b95910 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 10 client.25423 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 10 client.25423 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 20 client.25423 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06c1e1a0 0x557f06c20590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 4 client.25423 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f06b024c0 con 0x557f06c1e1a0 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.944+0000 7f72bffff640 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06c1e1a0 0x557f06c20590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:56.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.945+0000 7f72bffff640 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06c1e1a0 0x557f06c20590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c28ac6e34002b0b server_cookie=e2b7238940231b4a in_seq=0 out_seq=0 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.945+0000 7f72a6ffd640 10 client.25423 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.945+0000 7f72a6ffd640 1 -- 172.21.15.103:0/2234070412 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f06b024c0 con 0x557f06c1e1a0 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.945+0000 7f72a6ffd640 10 client.25423 handle_command_reply: tid=0 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.945+0000 7f72cfca6740 1 client.25423 shutdown 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.945+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b95910 msgr2=0x557f06b9a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.945+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b95910 0x557f06b9a810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f72b80093e0 tx=0x7f72b80041d0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 shutdown_connections 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x557f06c1e1a0 0x557f06c20590 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f06ba01d0 0x557f06bb6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f06b95910 0x557f06b9a810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72a6ffd640 0 client.25423 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 --2- 172.21.15.103:0/2234070412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557f06b94ff0 0x557f06b9a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 >> 172.21.15.103:0/2234070412 conn(0x557f06b91580 msgr2=0x557f06b9b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:56.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 shutdown_connections 2024-08-19T01:37:56.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 1 -- 172.21.15.103:0/2234070412 wait complete. 2024-08-19T01:37:56.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cfca6740 20 client.25423 trim_cache size 0 max 16384 2024-08-19T01:37:56.949 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72b40a2b80 msgr2=0x7f72b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72b40a2b80 0x7f72b40a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f72c0097410 tx=0x7f72c00047d0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8108010 msgr2=0x7f72c819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8108010 0x7f72c819cb50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009fd0 tx=0x7f72c000b8e0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.946+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 shutdown_connections 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.947+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f72b40a2b80 0x7f72b40a5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.947+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72c8108010 0x7f72c819cb50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.947+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72c8103490 0x7f72c81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.947+0000 7f72cd8a8640 1 --2- 172.21.15.103:0/398196316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72c8102ae0 0x7f72c8198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.947+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 >> 172.21.15.103:0/398196316 conn(0x7f72c80fc500 msgr2=0x7f72c8105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.947+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 shutdown_connections 2024-08-19T01:37:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:56.947+0000 7f72cd8a8640 1 -- 172.21.15.103:0/398196316 wait complete. 2024-08-19T01:37:57.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:56 smithi146 ceph-mon[22098]: pgmap v404: 97 pgs: 97 active+clean; 2.0 GiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-08-19T01:37:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:56 smithi103 ceph-mon[18436]: pgmap v404: 97 pgs: 97 active+clean; 2.0 GiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-08-19T01:37:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:56 smithi190 ceph-mon[22032]: pgmap v404: 97 pgs: 97 active+clean; 2.0 GiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-08-19T01:37:57.751 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:37:58.024 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:37:58.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 -- 172.21.15.103:0/409823295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44102a60 msgr2=0x7fac44102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 --2- 172.21.15.103:0/409823295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44102a60 0x7fac44102e40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fac34009930 tx=0x7fac3402a420 comp rx=0 tx=0).stop 2024-08-19T01:37:58.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 -- 172.21.15.103:0/409823295 shutdown_connections 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 --2- 172.21.15.103:0/409823295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac44107fd0 0x7fac4410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 --2- 172.21.15.103:0/409823295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac44103410 0x7fac441078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 --2- 172.21.15.103:0/409823295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44102a60 0x7fac44102e40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 -- 172.21.15.103:0/409823295 >> 172.21.15.103:0/409823295 conn(0x7fac440fc500 msgr2=0x7fac440fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 -- 172.21.15.103:0/409823295 shutdown_connections 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.523+0000 7fac493fe640 1 -- 172.21.15.103:0/409823295 wait complete. 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.524+0000 7fac493fe640 1 Processor -- start 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.524+0000 7fac493fe640 1 -- start start 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.524+0000 7fac493fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac44102a60 0x7fac44198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.524+0000 7fac493fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44103410 0x7fac44198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.524+0000 7fac493fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac44107fd0 0x7fac4419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44103410 0x7fac44198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac493fe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac44111c80 con 0x7fac44103410 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac493fe640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fac44111b00 con 0x7fac44107fd0 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac493fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac44111e00 con 0x7fac44102a60 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44103410 0x7fac44198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40236/0 (socket says 172.21.15.103:40236) 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 -- 172.21.15.103:0/1132637256 learned_addr learned my addr 172.21.15.103:0/1132637256 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac42ffd640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac44102a60 0x7fac44198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac437fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac44107fd0 0x7fac4419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 -- 172.21.15.103:0/1132637256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac44102a60 msgr2=0x7fac44198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac44102a60 0x7fac44198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 -- 172.21.15.103:0/1132637256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac44107fd0 msgr2=0x7fac4419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac44107fd0 0x7fac4419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 -- 172.21.15.103:0/1132637256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4419d1d0 con 0x7fac44103410 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac437fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac44107fd0 0x7fac4419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:37:58.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac427fc640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44103410 0x7fac44198760 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c970 tx=0x7fac3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac2bfff640 1 -- 172.21.15.103:0/1132637256 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38007bb0 con 0x7fac44103410 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac2bfff640 1 -- 172.21.15.103:0/1132637256 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac38007d50 con 0x7fac44103410 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4419d4c0 con 0x7fac44103410 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac2bfff640 1 -- 172.21.15.103:0/1132637256 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3800f450 con 0x7fac44103410 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.525+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac441005f0 con 0x7fac44103410 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac2bfff640 1 -- 172.21.15.103:0/1132637256 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fac38004eb0 con 0x7fac44103410 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 Processor -- start 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 -- start start 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da77e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da788c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da78b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b8da6e3a40 con 0x55b8da77e910 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b8da67b9d0 con 0x55b8da7891d0 2024-08-19T01:37:58.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b8da6dfa00 con 0x55b8da77dff0 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.528+0000 7fac2bfff640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fac2c0a2b80 0x7fac2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da788c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac437fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fac2c0a2b80 0x7fac2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da788c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40242/0 (socket says 172.21.15.103:40242) 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 -- 172.21.15.103:0/2603364702 learned_addr learned my addr 172.21.15.103:0/2603364702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac427fc640 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da78b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac437fe640 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da77e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac2bfff640 1 -- 172.21.15.103:0/1132637256 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fac380cee60 con 0x7fac44103410 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 -- 172.21.15.103:0/2603364702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 msgr2=0x55b8da77e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da77e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 -- 172.21.15.103:0/2603364702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 msgr2=0x55b8da78b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da78b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 -- 172.21.15.103:0/2603364702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8da682650 con 0x55b8da77e910 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac427fc640 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da78b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:58.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac437fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fac2c0a2b80 0x7fac2c0a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005c40 tx=0x7fac3c009cd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.529+0000 7fac42ffd640 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da788c90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fac34009930 tx=0x7fac34032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bfaaae13c0c46b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.530+0000 7fac297fa640 1 -- 172.21.15.103:0/2603364702 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3403f070 con 0x55b8da77e910 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.530+0000 7fac297fa640 1 -- 172.21.15.103:0/2603364702 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac340316a0 con 0x55b8da77e910 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.530+0000 7fac297fa640 1 -- 172.21.15.103:0/2603364702 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34035070 con 0x55b8da77e910 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.530+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2603364702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 msgr2=0x55b8da788c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.530+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da788c90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fac34009930 tx=0x7fac34032cc0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2603364702 shutdown_connections 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da78b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da788c90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2603364702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da77e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2603364702 >> 172.21.15.103:0/2603364702 conn(0x55b8da77a580 msgr2=0x55b8da77a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2603364702 shutdown_connections 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2603364702 wait complete. 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- start start 2024-08-19T01:37:58.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 Processor -- start 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77dff0 0x55b8da786080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77e910 0x55b8da7865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da786b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b8da78bda0 con 0x55b8da77dff0 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b8da69eb20 con 0x55b8da7891d0 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b8da78bf20 con 0x55b8da77e910 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac4b7fc740 10 client.0 _ll_register_callbacks cb 0x55b8da69cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.531+0000 7fac42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77dff0 0x55b8da786080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77dff0 0x55b8da786080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40252/0 (socket says 172.21.15.103:40252) 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 -- 172.21.15.103:0/2435736340 learned_addr learned my addr 172.21.15.103:0/2435736340 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac437fe640 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da786b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 -- 172.21.15.103:0/2435736340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77e910 msgr2=0x55b8da7865c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77e910 0x55b8da7865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 -- 172.21.15.103:0/2435736340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 msgr2=0x55b8da786b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da786b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 -- 172.21.15.103:0/2435736340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8da4553c0 con 0x55b8da77dff0 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac437fe640 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da786b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac42ffd640 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77dff0 0x55b8da786080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fac34009900 tx=0x7fac34009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d36636506d3f99 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac26ff5640 1 -- 172.21.15.103:0/2435736340 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3404a020 con 0x55b8da77dff0 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac26ff5640 1 -- 172.21.15.103:0/2435736340 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fac34004250 con 0x55b8da77dff0 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac340316a0 con 0x55b8da77dff0 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac26ff5640 1 -- 172.21.15.103:0/2435736340 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac3403f070 con 0x55b8da77dff0 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac4b7fc740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:58.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.532+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b8da7978e0 con 0x55b8da77dff0 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac26ff5640 1 -- 172.21.15.103:0/2435736340 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac340316a0 con 0x55b8da77dff0 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac26ff5640 1 -- 172.21.15.103:0/2435736340 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fac34004770 con 0x55b8da77dff0 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac4b7fc740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac4b7fc740 10 client.15966 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac4b7fc740 20 client.15966 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da798c30 0x55b8da79b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac4b7fc740 4 client.15966 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8da6f0fb0 con 0x55b8da798c30 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.533+0000 7fac427fc640 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da798c30 0x55b8da79b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.534+0000 7fac26ff5640 10 client.15966 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.534+0000 7fac427fc640 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da798c30 0x55b8da79b020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2909261a8b25408d server_cookie=bce8ef10cdad9ce1 in_seq=0 out_seq=0 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.534+0000 7fac26ff5640 1 -- 172.21.15.103:0/2435736340 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8da6f0fb0 con 0x55b8da798c30 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.534+0000 7fac26ff5640 10 client.15966 handle_command_reply: tid=0 2024-08-19T01:37:58.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.534+0000 7fac4b7fc740 1 client.15966 shutdown 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77dff0 msgr2=0x55b8da786080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77dff0 0x55b8da786080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fac34009900 tx=0x7fac34009660 comp rx=0 tx=0).stop 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 shutdown_connections 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da798c30 0x55b8da79b020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac26ff5640 0 client.15966 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da786b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77e910 0x55b8da7865c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/2435736340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77dff0 0x55b8da786080 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 >> 172.21.15.103:0/2435736340 conn(0x55b8da77a580 msgr2=0x55b8da780260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 shutdown_connections 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 1 -- 172.21.15.103:0/2435736340 wait complete. 2024-08-19T01:37:58.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.535+0000 7fac4b7fc740 20 client.15966 trim_cache size 0 max 16384 2024-08-19T01:37:58.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.549+0000 7fac4b7fc740 1 Processor -- start 2024-08-19T01:37:58.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.549+0000 7fac4b7fc740 1 -- start start 2024-08-19T01:37:58.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.549+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da778d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.549+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da7792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da779810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac441af360 con 0x55b8da77e910 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fac441b0f00 con 0x55b8da7891d0 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac441afa30 con 0x55b8da77dff0 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da7792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da7792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40268/0 (socket says 172.21.15.103:40268) 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 -- 172.21.15.103:0/980627696 learned_addr learned my addr 172.21.15.103:0/980627696 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac42ffd640 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da778d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac437fe640 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da779810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 -- 172.21.15.103:0/980627696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 msgr2=0x55b8da778d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da778d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 -- 172.21.15.103:0/980627696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 msgr2=0x55b8da779810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da779810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 -- 172.21.15.103:0/980627696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8da77e740 con 0x55b8da77e910 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac437fe640 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da779810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac427fc640 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da7792d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fac3809b3f0 tx=0x7fac380048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493a63610bbcaec0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.550+0000 7fac28ff9640 1 -- 172.21.15.103:0/980627696 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3801d070 con 0x55b8da77e910 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac28ff9640 1 -- 172.21.15.103:0/980627696 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac380cce50 con 0x55b8da77e910 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac28ff9640 1 -- 172.21.15.103:0/980627696 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3809ab40 con 0x55b8da77e910 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac4b7fc740 1 -- 172.21.15.103:0/980627696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 msgr2=0x55b8da7792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da7792d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fac3809b3f0 tx=0x7fac380048b0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac4b7fc740 1 -- 172.21.15.103:0/980627696 shutdown_connections 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da7891d0 0x55b8da779810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da77e910 0x55b8da7792d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.551+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/980627696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da77dff0 0x55b8da778d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 -- 172.21.15.103:0/980627696 >> 172.21.15.103:0/980627696 conn(0x55b8da77a580 msgr2=0x55b8da7751b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 -- 172.21.15.103:0/980627696 shutdown_connections 2024-08-19T01:37:58.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 -- 172.21.15.103:0/980627696 wait complete. 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 -- start start 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 Processor -- start 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da77dff0 0x55b8da79b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da7891d0 0x55b8da79b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da798c30 0x55b8da79bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac441b0800 con 0x55b8da7891d0 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fac441b0100 con 0x55b8da77dff0 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fac441ad360 con 0x55b8da798c30 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac4b7fc740 10 client.0 _ll_register_callbacks cb 0x7fac3c0145b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.552+0000 7fac427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da7891d0 0x55b8da79b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da7891d0 0x55b8da79b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40280/0 (socket says 172.21.15.103:40280) 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 -- 172.21.15.103:0/3140149441 learned_addr learned my addr 172.21.15.103:0/3140149441 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac437fe640 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da798c30 0x55b8da79bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac42ffd640 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da77dff0 0x55b8da79b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 -- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da798c30 msgr2=0x55b8da79bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da798c30 0x55b8da79bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 -- 172.21.15.103:0/3140149441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da77dff0 msgr2=0x55b8da79b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da77dff0 0x55b8da79b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 -- 172.21.15.103:0/3140149441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8da801850 con 0x55b8da7891d0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac427fc640 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da7891d0 0x55b8da79b5c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fac380cc9e0 tx=0x7fac3809faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b25a2797b3ac7a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac26ff5640 1 -- 172.21.15.103:0/3140149441 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3801d040 con 0x55b8da7891d0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac26ff5640 1 -- 172.21.15.103:0/3140149441 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fac3809add0 con 0x55b8da7891d0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.553+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac380cce50 con 0x55b8da7891d0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac26ff5640 1 -- 172.21.15.103:0/3140149441 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac3809ab40 con 0x55b8da7891d0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b8da802b80 con 0x55b8da7891d0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac26ff5640 1 -- 172.21.15.103:0/3140149441 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac380cce50 con 0x55b8da7891d0 2024-08-19T01:37:58.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac26ff5640 1 -- 172.21.15.103:0/3140149441 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fac38097d60 con 0x55b8da7891d0 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 10 client.15972 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 20 client.15972 populate_metadata read hostname 'smithi103' 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da803250 0x55b8da805670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 4 client.15972 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8da6eb4c0 con 0x55b8da803250 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.554+0000 7fac42ffd640 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da803250 0x55b8da805670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac26ff5640 10 client.15972 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac42ffd640 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da803250 0x55b8da805670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e6b0085ec35165d server_cookie=b8d58c37347db954 in_seq=0 out_seq=0 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac26ff5640 1 -- 172.21.15.103:0/3140149441 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8da6eb4c0 con 0x55b8da803250 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac26ff5640 10 client.15972 handle_command_reply: tid=0 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 client.15972 shutdown 2024-08-19T01:37:58.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da7891d0 msgr2=0x55b8da79b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da7891d0 0x55b8da79b5c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fac380cc9e0 tx=0x7fac3809faa0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 shutdown_connections 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55b8da803250 0x55b8da805670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8da798c30 0x55b8da79bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8da7891d0 0x55b8da79b5c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 --2- 172.21.15.103:0/3140149441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8da77dff0 0x55b8da79b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 >> 172.21.15.103:0/3140149441 conn(0x55b8da77a580 msgr2=0x55b8da778d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac26ff5640 0 client.15972 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 shutdown_connections 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 1 -- 172.21.15.103:0/3140149441 wait complete. 2024-08-19T01:37:58.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.555+0000 7fac4b7fc740 20 client.15972 trim_cache size 0 max 16384 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fac2c0a2b80 msgr2=0x7fac2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fac2c0a2b80 0x7fac2c0a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005c40 tx=0x7fac3c009cd0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44103410 msgr2=0x7fac44198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44103410 0x7fac44198760 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c970 tx=0x7fac3800ce30 comp rx=0 tx=0).stop 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 shutdown_connections 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fac2c0a2b80 0x7fac2c0a5040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac44107fd0 0x7fac4419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac44103410 0x7fac44198760 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 --2- 172.21.15.103:0/1132637256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac44102a60 0x7fac44198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:37:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 >> 172.21.15.103:0/1132637256 conn(0x7fac440fc500 msgr2=0x7fac44105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:37:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 shutdown_connections 2024-08-19T01:37:58.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:37:58.556+0000 7fac493fe640 1 -- 172.21.15.103:0/1132637256 wait complete. 2024-08-19T01:37:59.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:37:59.687 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:00.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:37:59 smithi103 ceph-mon[18436]: pgmap v405: 97 pgs: 97 active+clean; 2.1 GiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:00.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:37:59 smithi146 ceph-mon[22098]: pgmap v405: 97 pgs: 97 active+clean; 2.1 GiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.199+0000 7fbe75f6d640 1 -- 172.21.15.103:0/1562994509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe701007a0 msgr2=0x7fbe70100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.199+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/1562994509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe701007a0 0x7fbe70100c00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009930 tx=0x7fbe6402a420 comp rx=0 tx=0).stop 2024-08-19T01:38:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 -- 172.21.15.103:0/1562994509 shutdown_connections 2024-08-19T01:38:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/1562994509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe70101140 0x7fbe7010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/1562994509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe701007a0 0x7fbe70100c00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/1562994509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe701067a0 0x7fbe70106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 -- 172.21.15.103:0/1562994509 >> 172.21.15.103:0/1562994509 conn(0x7fbe700fc4c0 msgr2=0x7fbe700fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 -- 172.21.15.103:0/1562994509 shutdown_connections 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 -- 172.21.15.103:0/1562994509 wait complete. 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 Processor -- start 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.200+0000 7fbe75f6d640 1 -- start start 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe75f6d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe701007a0 0x7fbe7019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe75f6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe70101140 0x7fbe7019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe75f6d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe701067a0 0x7fbe7019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe75f6d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70110a50 con 0x7fbe70101140 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe75f6d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701108d0 con 0x7fbe701067a0 2024-08-19T01:38:00.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe75f6d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70110bd0 con 0x7fbe701007a0 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe70101140 0x7fbe7019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe70101140 0x7fbe7019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40306/0 (socket says 172.21.15.103:40306) 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 -- 172.21.15.103:0/4125206997 learned_addr learned my addr 172.21.15.103:0/4125206997 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe701007a0 0x7fbe7019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe701067a0 0x7fbe7019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 -- 172.21.15.103:0/4125206997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe701007a0 msgr2=0x7fbe7019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe701007a0 0x7fbe7019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 -- 172.21.15.103:0/4125206997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe701067a0 msgr2=0x7fbe7019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe701067a0 0x7fbe7019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 -- 172.21.15.103:0/4125206997 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7019f230 con 0x7fbe70101140 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.201+0000 7fbe6effd640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe70101140 0x7fbe7019a7c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005940 tx=0x7fbe640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.202+0000 7fbe6cff9640 1 -- 172.21.15.103:0/4125206997 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6403f030 con 0x7fbe70101140 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.202+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7019f4c0 con 0x7fbe70101140 2024-08-19T01:38:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.202+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a6e00 con 0x7fbe70101140 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.202+0000 7fbe6cff9640 1 -- 172.21.15.103:0/4125206997 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe64003cb0 con 0x7fbe70101140 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.202+0000 7fbe6cff9640 1 -- 172.21.15.103:0/4125206997 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64035070 con 0x7fbe70101140 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.204+0000 7fbe7836b740 1 Processor -- start 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.204+0000 7fbe7836b740 1 -- start start 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.204+0000 7fbe6cff9640 1 -- 172.21.15.103:0/4125206997 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbe64042030 con 0x7fbe70101140 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec226e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226e910 0x564ec2278c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec227b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe7836b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ec21d3a40 con 0x564ec226dff0 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe7836b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564ec216b9d0 con 0x564ec226e910 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe7836b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ec21cfa00 con 0x564ec22791d0 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec226e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec226e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40314/0 (socket says 172.21.15.103:40314) 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2614370863 learned_addr learned my addr 172.21.15.103:0/2614370863 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6effd640 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec227b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226e910 0x564ec2278c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2614370863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 msgr2=0x564ec227b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec227b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2614370863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226e910 msgr2=0x564ec2278c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226e910 0x564ec2278c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2614370863 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec2172650 con 0x564ec226dff0 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6effd640 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec227b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6cff9640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe580a2c00 0x7fbe580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe580a2c00 0x7fbe580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.205+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec226e3d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe68007f40 tx=0x7fbe6800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2ad5c383c7f76 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:00.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.206+0000 7fbe6cff9640 1 -- 172.21.15.103:0/4125206997 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbe640ef300 con 0x7fbe70101140 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.206+0000 7fbe55ffb640 1 -- 172.21.15.103:0/2614370863 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68010900 con 0x564ec226dff0 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.206+0000 7fbe55ffb640 1 -- 172.21.15.103:0/2614370863 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe680183f0 con 0x564ec226dff0 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.206+0000 7fbe55ffb640 1 -- 172.21.15.103:0/2614370863 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68017600 con 0x564ec226dff0 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 -- 172.21.15.103:0/2614370863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 msgr2=0x564ec226e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec226e3d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe68007f40 tx=0x7fbe6800f850 comp rx=0 tx=0).stop 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe580a2c00 0x7fbe580a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x564ec226b3e0 tx=0x7fbe6000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 -- 172.21.15.103:0/2614370863 shutdown_connections 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec227b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226e910 0x564ec2278c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2614370863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec226e3d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 -- 172.21.15.103:0/2614370863 >> 172.21.15.103:0/2614370863 conn(0x564ec226a580 msgr2=0x564ec226a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 -- 172.21.15.103:0/2614370863 shutdown_connections 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 -- 172.21.15.103:0/2614370863 wait complete. 2024-08-19T01:38:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 -- start start 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.207+0000 7fbe7836b740 1 Processor -- start 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec226dff0 0x564ec2278c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226e910 0x564ec2275f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2276710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe7836b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564ec227bdf0 con 0x564ec226e910 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe7836b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564ec218eb20 con 0x564ec22791d0 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe7836b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ec227bf70 con 0x564ec226dff0 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe7836b740 10 client.0 _ll_register_callbacks cb 0x564ec218cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226e910 0x564ec2275f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226e910 0x564ec2275f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40322/0 (socket says 172.21.15.103:40322) 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 -- 172.21.15.103:0/2067518943 learned_addr learned my addr 172.21.15.103:0/2067518943 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec226dff0 0x564ec2278c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2276710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 -- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec226dff0 msgr2=0x564ec2278c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec226dff0 0x564ec2278c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 -- 172.21.15.103:0/2067518943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 msgr2=0x564ec2276710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2276710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 -- 172.21.15.103:0/2067518943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec1f453c0 con 0x564ec226e910 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe6effd640 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226e910 0x564ec2275f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbe64032fd0 tx=0x7fbe640b7e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc25b2b2ed5ab30c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.208+0000 7fbe4effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe4effd640 1 -- 172.21.15.103:0/2067518943 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe64049070 con 0x564ec226e910 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe4effd640 1 -- 172.21.15.103:0/2067518943 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbe640b8200 con 0x564ec226e910 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe680183f0 con 0x564ec226e910 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe4effd640 1 -- 172.21.15.103:0/2067518943 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe640bb4f0 con 0x564ec226e910 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 10 client.15984 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ec2287960 con 0x564ec226e910 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe4effd640 1 -- 172.21.15.103:0/2067518943 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe680183f0 con 0x564ec226e910 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe4effd640 1 -- 172.21.15.103:0/2067518943 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbe640ee070 con 0x564ec226e910 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 10 client.15984 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 10 client.15984 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 20 client.15984 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2288df0 0x564ec228b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 4 client.15984 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.209+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ec21e0fb0 con 0x564ec2288df0 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.210+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2288df0 0x564ec228b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.210+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2288df0 0x564ec228b210 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5baccacfebbc74c2 server_cookie=bfa4b4d2ae270fc3 in_seq=0 out_seq=0 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.210+0000 7fbe4effd640 10 client.15984 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe4effd640 1 -- 172.21.15.103:0/2067518943 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ec21e0fb0 con 0x564ec2288df0 2024-08-19T01:38:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe4effd640 10 client.15984 handle_command_reply: tid=0 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 client.15984 shutdown 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226e910 msgr2=0x564ec2275f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226e910 0x564ec2275f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbe64032fd0 tx=0x7fbe640b7e30 comp rx=0 tx=0).stop 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 shutdown_connections 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2288df0 0x564ec228b210 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2276710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226e910 0x564ec2275f80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe4effd640 0 client.15984 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2067518943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec226dff0 0x564ec2278c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 >> 172.21.15.103:0/2067518943 conn(0x564ec226a580 msgr2=0x564ec22702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 shutdown_connections 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 1 -- 172.21.15.103:0/2067518943 wait complete. 2024-08-19T01:38:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.211+0000 7fbe7836b740 20 client.15984 trim_cache size 0 max 16384 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 Processor -- start 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 -- start start 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226dff0 0x564ec2271270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec22717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec2274850 0x564ec2274ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c00e280 con 0x564ec2274850 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c00fe20 con 0x564ec226dff0 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe7836b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c00e950 con 0x564ec22791d0 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec2274850 0x564ec2274ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec2274850 0x564ec2274ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40326/0 (socket says 172.21.15.103:40326) 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe6ffff640 1 -- 172.21.15.103:0/1103369188 learned_addr learned my addr 172.21.15.103:0/1103369188 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe6effd640 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec22717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.226+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226dff0 0x564ec2271270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe6ffff640 1 -- 172.21.15.103:0/1103369188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 msgr2=0x564ec22717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec22717b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe6ffff640 1 -- 172.21.15.103:0/1103369188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226dff0 msgr2=0x564ec2271270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226dff0 0x564ec2271270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe6ffff640 1 -- 172.21.15.103:0/1103369188 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec228b890 con 0x564ec2274850 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226dff0 0x564ec2271270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec2274850 0x564ec2274ca0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800fd50 tx=0x7fbe68018eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b57bd22cd49980 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe557fa640 1 -- 172.21.15.103:0/1103369188 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68007960 con 0x564ec2274850 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe557fa640 1 -- 172.21.15.103:0/1103369188 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe68003e20 con 0x564ec2274850 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.227+0000 7fbe557fa640 1 -- 172.21.15.103:0/1103369188 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe680183f0 con 0x564ec2274850 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 -- 172.21.15.103:0/1103369188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec2274850 msgr2=0x564ec2274ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec2274850 0x564ec2274ca0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800fd50 tx=0x7fbe68018eb0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 -- 172.21.15.103:0/1103369188 shutdown_connections 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec2274850 0x564ec2274ca0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec22791d0 0x564ec22717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 --2- 172.21.15.103:0/1103369188 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec226dff0 0x564ec2271270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 -- 172.21.15.103:0/1103369188 >> 172.21.15.103:0/1103369188 conn(0x564ec226a580 msgr2=0x564ec22651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 -- 172.21.15.103:0/1103369188 shutdown_connections 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.228+0000 7fbe7836b740 1 -- 172.21.15.103:0/1103369188 wait complete. 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 -- start start 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 Processor -- start 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec2273a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec2274850 0x564ec2281250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2281790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c00f720 con 0x564ec226dff0 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c00f020 con 0x564ec22791d0 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe7836b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c00c260 con 0x564ec2274850 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2281790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.229+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2281790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:46466/0 (socket says 172.21.15.103:46466) 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2765103270 learned_addr learned my addr 172.21.15.103:0/2765103270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec2273a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6effd640 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec2274850 0x564ec2281250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe7836b740 10 client.0 _ll_register_callbacks cb 0x7fbe64009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec2274850 msgr2=0x564ec2281250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec2274850 0x564ec2281250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2765103270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 msgr2=0x564ec2273a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec2273a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6ffff640 1 -- 172.21.15.103:0/2765103270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec227a180 con 0x564ec22791d0 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6effd640 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec2274850 0x564ec2281250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe6ffff640 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2281790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800f890 tx=0x7fbe68007230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8064a0c09b5b4863 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe4effd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:38:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe4effd640 1 -- 172.21.15.103:0/2765103270 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68030030 con 0x564ec22791d0 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe68003e20 con 0x564ec22791d0 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.230+0000 7fbe4effd640 1 -- 172.21.15.103:0/2765103270 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbe680183f0 con 0x564ec22791d0 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe4effd640 1 -- 172.21.15.103:0/2765103270 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe68003c40 con 0x564ec22791d0 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 10 client.25484 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ec2304bf0 con 0x564ec22791d0 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe4effd640 1 -- 172.21.15.103:0/2765103270 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe68003e20 con 0x564ec22791d0 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe4effd640 1 -- 172.21.15.103:0/2765103270 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbe680055e0 con 0x564ec22791d0 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 10 client.25484 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 10 client.25484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 20 client.25484 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2305120 0x564ec2307510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 4 client.25484 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ec21db4c0 con 0x564ec2305120 2024-08-19T01:38:00.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.231+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2305120 0x564ec2307510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe6f7fe640 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2305120 0x564ec2307510 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90b3fd346e9b6ea6 server_cookie=d29d461f5c5bcc22 in_seq=0 out_seq=0 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe4effd640 10 client.25484 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe4effd640 1 -- 172.21.15.103:0/2765103270 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ec21db4c0 con 0x564ec2305120 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe4effd640 10 client.25484 handle_command_reply: tid=0 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 client.25484 shutdown 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 msgr2=0x564ec2281790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2281790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800f890 tx=0x7fbe68007230 comp rx=0 tx=0).stop 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 shutdown_connections 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x564ec2305120 0x564ec2307510 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564ec22791d0 0x564ec2281790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ec2274850 0x564ec2281250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 --2- 172.21.15.103:0/2765103270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564ec226dff0 0x564ec2273a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 >> 172.21.15.103:0/2765103270 conn(0x564ec226a580 msgr2=0x564ec2278790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe4effd640 0 client.25484 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:38:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 shutdown_connections 2024-08-19T01:38:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.232+0000 7fbe7836b740 1 -- 172.21.15.103:0/2765103270 wait complete. 2024-08-19T01:38:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.233+0000 7fbe7836b740 20 client.25484 trim_cache size 0 max 16384 2024-08-19T01:38:00.249 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.233+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe580a2c00 msgr2=0x7fbe580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.233+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe580a2c00 0x7fbe580a50c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x564ec226b3e0 tx=0x7fbe6000a2d0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.233+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe70101140 msgr2=0x7fbe7019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.233+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe70101140 0x7fbe7019a7c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005940 tx=0x7fbe640042f0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 shutdown_connections 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbe580a2c00 0x7fbe580a50c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe701067a0 0x7fbe7019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe70101140 0x7fbe7019a7c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 --2- 172.21.15.103:0/4125206997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe701007a0 0x7fbe7019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 >> 172.21.15.103:0/4125206997 conn(0x7fbe700fc4c0 msgr2=0x7fbe7010c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 shutdown_connections 2024-08-19T01:38:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:00.234+0000 7fbe75f6d640 1 -- 172.21.15.103:0/4125206997 wait complete. 2024-08-19T01:38:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:37:59 smithi190 ceph-mon[22032]: pgmap v405: 97 pgs: 97 active+clean; 2.1 GiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:01.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:38:01.572 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:01.986 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:01 smithi103 ceph-mon[18436]: pgmap v406: 97 pgs: 97 active+clean; 2.1 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:38:02.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 -- 172.21.15.103:0/2882421811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 msgr2=0x7f17341078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2882421811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f17341078c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1724009930 tx=0x7f172402a420 comp rx=0 tx=0).stop 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 -- 172.21.15.103:0/2882421811 shutdown_connections 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2882421811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1734107fd0 0x7f173410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2882421811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f17341078c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2882421811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1734102a60 0x7f1734102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 -- 172.21.15.103:0/2882421811 >> 172.21.15.103:0/2882421811 conn(0x7f17340fc500 msgr2=0x7f17340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 -- 172.21.15.103:0/2882421811 shutdown_connections 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.073+0000 7f173b4b8640 1 -- 172.21.15.103:0/2882421811 wait complete. 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 Processor -- start 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 -- start start 2024-08-19T01:38:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1734102a60 0x7f1734198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f1734198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1734107fd0 0x7f173419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1734111c50 con 0x7f1734103410 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1734111ad0 con 0x7f1734107fd0 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.074+0000 7f173b4b8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1734111dd0 con 0x7f1734102a60 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f1734198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f1734198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43966/0 (socket says 172.21.15.103:43966) 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 -- 172.21.15.103:0/2009680065 learned_addr learned my addr 172.21.15.103:0/2009680065 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1739254640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1734102a60 0x7f1734198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1739a55640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1734107fd0 0x7f173419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 -- 172.21.15.103:0/2009680065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1734102a60 msgr2=0x7f1734198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1734102a60 0x7f1734198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 -- 172.21.15.103:0/2009680065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1734107fd0 msgr2=0x7f173419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1734107fd0 0x7f173419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 -- 172.21.15.103:0/2009680065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173419d1e0 con 0x7f1734103410 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f1738a53640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f1734198770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1724031440 tx=0x7f1724004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f172a7fc640 1 -- 172.21.15.103:0/2009680065 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172403f030 con 0x7f1734103410 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f172a7fc640 1 -- 172.21.15.103:0/2009680065 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1724003aa0 con 0x7f1734103410 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173419d470 con 0x7f1734103410 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.075+0000 7f172a7fc640 1 -- 172.21.15.103:0/2009680065 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724035070 con 0x7f1734103410 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.076+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17341005f0 con 0x7f1734103410 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.078+0000 7f172a7fc640 1 -- 172.21.15.103:0/2009680065 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1724042030 con 0x7f1734103410 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 Processor -- start 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 -- start start 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5abff0 0x560fdb5ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5b71d0 0x560fdb5b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560fdb511a40 con 0x560fdb5abff0 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560fdb4a99d0 con 0x560fdb5b71d0 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f173d8b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560fdb50da00 con 0x560fdb5ac910 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f172a7fc640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17200a2c00 0x7f17200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f1739a55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5abff0 0x560fdb5ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f1739a55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5abff0 0x560fdb5ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43968/0 (socket says 172.21.15.103:43968) 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f1739a55640 1 -- 172.21.15.103:0/1265310216 learned_addr learned my addr 172.21.15.103:0/1265310216 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f172a7fc640 1 -- 172.21.15.103:0/2009680065 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f17240ef660 con 0x7f1734103410 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f1738a53640 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5b71d0 0x560fdb5b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.079+0000 7f1739254640 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 -- 172.21.15.103:0/1265310216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 msgr2=0x560fdb5b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 -- 172.21.15.103:0/1265310216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5b71d0 msgr2=0x560fdb5b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5b71d0 0x560fdb5b95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 -- 172.21.15.103:0/1265310216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fdb4b0650 con 0x560fdb5abff0 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17200a2c00 0x7f17200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5abff0 0x560fdb5ac3d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f172c012470 tx=0x7f172c012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466abbc2f8f05e32 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f171f7fe640 1 -- 172.21.15.103:0/1265310216 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c026620 con 0x560fdb5abff0 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f171f7fe640 1 -- 172.21.15.103:0/1265310216 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f172c01bd10 con 0x560fdb5abff0 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f1739a55640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17200a2c00 0x7f17200a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f172c022000 tx=0x7f172c01f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.080+0000 7f171f7fe640 1 -- 172.21.15.103:0/1265310216 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c01aa70 con 0x560fdb5abff0 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 -- 172.21.15.103:0/1265310216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5abff0 msgr2=0x560fdb5ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5abff0 0x560fdb5ac3d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f172c012470 tx=0x7f172c012930 comp rx=0 tx=0).stop 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 -- 172.21.15.103:0/1265310216 shutdown_connections 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5b71d0 0x560fdb5b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1265310216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5abff0 0x560fdb5ac3d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 -- 172.21.15.103:0/1265310216 >> 172.21.15.103:0/1265310216 conn(0x560fdb5a8580 msgr2=0x560fdb5a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 -- 172.21.15.103:0/1265310216 shutdown_connections 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 -- 172.21.15.103:0/1265310216 wait complete. 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.081+0000 7f173d8b6740 1 -- start start 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 1 Processor -- start 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560fdb5b9da0 con 0x560fdb5ac910 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560fdb4ccb20 con 0x560fdb5abff0 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560fdb5b9f20 con 0x560fdb5b71d0 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f173d8b6740 10 client.0 _ll_register_callbacks cb 0x560fdb4cabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f1738a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f1738a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43974/0 (socket says 172.21.15.103:43974) 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f1738a53640 1 -- 172.21.15.103:0/1844414822 learned_addr learned my addr 172.21.15.103:0/1844414822 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f1739254640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.082+0000 7f1739a55640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f1738a53640 1 -- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 msgr2=0x560fdb5b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f1738a53640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f1738a53640 1 -- 172.21.15.103:0/1844414822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 msgr2=0x560fdb5b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f1738a53640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f1738a53640 1 -- 172.21.15.103:0/1844414822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fdb2833c0 con 0x560fdb5ac910 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f1739254640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f1738a53640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5b45f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f17240bbf30 tx=0x7f17240bb900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ead911a2fa7188 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f171cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f171cff9640 1 -- 172.21.15.103:0/1844414822 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724049070 con 0x560fdb5ac910 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c01bd10 con 0x560fdb5ac910 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f171cff9640 1 -- 172.21.15.103:0/1844414822 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1724003c50 con 0x560fdb5ac910 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f171cff9640 1 -- 172.21.15.103:0/1844414822 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17240c4b60 con 0x560fdb5ac910 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f173d8b6740 10 client.16002 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fdb5c59a0 con 0x560fdb5ac910 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.083+0000 7f171cff9640 1 -- 172.21.15.103:0/1844414822 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f172c01bd10 con 0x560fdb5ac910 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f171cff9640 1 -- 172.21.15.103:0/1844414822 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f17240c46d0 con 0x560fdb5ac910 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f173d8b6740 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f173d8b6740 10 client.16002 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f173d8b6740 20 client.16002 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb5c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f173d8b6740 4 client.16002 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fdb51efb0 con 0x560fdb5c6d40 2024-08-19T01:38:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f1739254640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb5c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f1739254640 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb5c9190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf8f53e2d75ece03 server_cookie=6862cadc3a5eeae9 in_seq=0 out_seq=0 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.084+0000 7f171cff9640 10 client.16002 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f171cff9640 1 -- 172.21.15.103:0/1844414822 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fdb51efb0 con 0x560fdb5c6d40 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f171cff9640 10 client.16002 handle_command_reply: tid=0 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 client.16002 shutdown 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 msgr2=0x560fdb5b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5b45f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f17240bbf30 tx=0x7f17240bb900 comp rx=0 tx=0).stop 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 shutdown_connections 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb5c9190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f171cff9640 0 client.16002 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5b45f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 --2- 172.21.15.103:0/1844414822 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.085+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 >> 172.21.15.103:0/1844414822 conn(0x560fdb5a8580 msgr2=0x560fdb5ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:02.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.086+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 shutdown_connections 2024-08-19T01:38:02.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.086+0000 7f173d8b6740 1 -- 172.21.15.103:0/1844414822 wait complete. 2024-08-19T01:38:02.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.086+0000 7f173d8b6740 20 client.16002 trim_cache size 0 max 16384 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 Processor -- start 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 -- start start 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17341af2d0 con 0x560fdb5ac910 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17341b0e70 con 0x560fdb5abff0 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f173d8b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17341af9a0 con 0x560fdb5b71d0 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f1738a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f1738a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43978/0 (socket says 172.21.15.103:43978) 2024-08-19T01:38:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.100+0000 7f1738a53640 1 -- 172.21.15.103:0/2754258678 learned_addr learned my addr 172.21.15.103:0/2754258678 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1739254640 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1739a55640 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1738a53640 1 -- 172.21.15.103:0/2754258678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 msgr2=0x560fdb5cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1738a53640 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1738a53640 1 -- 172.21.15.103:0/2754258678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 msgr2=0x560fdb5aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1738a53640 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1738a53640 1 -- 172.21.15.103:0/2754258678 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fdb5b9900 con 0x560fdb5ac910 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1739254640 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f1738a53640 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5ceaa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f17240b8400 tx=0x7f17240c0420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2938df7184c858e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f171effd640 1 -- 172.21.15.103:0/2754258678 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724049070 con 0x560fdb5ac910 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f171effd640 1 -- 172.21.15.103:0/2754258678 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17240bbfa0 con 0x560fdb5ac910 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.101+0000 7f171effd640 1 -- 172.21.15.103:0/2754258678 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240bb3f0 con 0x560fdb5ac910 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 -- 172.21.15.103:0/2754258678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 msgr2=0x560fdb5ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5ceaa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f17240b8400 tx=0x7f17240c0420 comp rx=0 tx=0).stop 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 -- 172.21.15.103:0/2754258678 shutdown_connections 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5b71d0 0x560fdb5cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5ac910 0x560fdb5ceaa0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 --2- 172.21.15.103:0/2754258678 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 -- 172.21.15.103:0/2754258678 >> 172.21.15.103:0/2754258678 conn(0x560fdb5a8580 msgr2=0x560fdb5b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 -- 172.21.15.103:0/2754258678 shutdown_connections 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 -- 172.21.15.103:0/2754258678 wait complete. 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.102+0000 7f173d8b6740 1 -- start start 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 1 Processor -- start 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5b71d0 0x560fdb5b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17341b0770 con 0x560fdb5b71d0 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17341b0070 con 0x560fdb5abff0 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17341ad2b0 con 0x560fdb5ac910 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f173d8b6740 10 client.0 _ll_register_callbacks cb 0x7f17240dc0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f1738a53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f1739254640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f1738a53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5bf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:41278/0 (socket says 172.21.15.103:41278) 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f1739a55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5b71d0 0x560fdb5b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.103+0000 7f1738a53640 1 -- 172.21.15.103:0/3193303324 learned_addr learned my addr 172.21.15.103:0/3193303324 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1739254640 1 -- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 msgr2=0x560fdb5bf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1739254640 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5bf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1739254640 1 -- 172.21.15.103:0/3193303324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5b71d0 msgr2=0x560fdb5b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1739254640 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5b71d0 0x560fdb5b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1739254640 1 -- 172.21.15.103:0/3193303324 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fdb5b09a0 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1739a55640 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5b71d0 0x560fdb5b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1738a53640 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f1739254640 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5bf170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1730004700 tx=0x7f173000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e09d08f1491d393 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f171cff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f171cff9640 1 -- 172.21.15.103:0/3193303324 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173001d380 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17240bbfa0 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f171cff9640 1 -- 172.21.15.103:0/3193303324 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f173001d520 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f171cff9640 1 -- 172.21.15.103:0/3193303324 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f173001d720 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f173d8b6740 10 client.25502 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fdb631100 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.104+0000 7f171cff9640 1 -- 172.21.15.103:0/3193303324 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17240bbfa0 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f171cff9640 1 -- 172.21.15.103:0/3193303324 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1730011af0 con 0x560fdb5abff0 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f173d8b6740 10 client.25502 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:02.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f173d8b6740 10 client.25502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f173d8b6740 20 client.25502 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f173d8b6740 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb6338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f173d8b6740 4 client.25502 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fdb5194c0 con 0x560fdb5c6d40 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f1738a53640 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb6338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f1738a53640 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb6338f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae6bb86f8a871ed5 server_cookie=dff8d6ee93ede451 in_seq=0 out_seq=0 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.105+0000 7f171cff9640 10 client.25502 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f171cff9640 1 -- 172.21.15.103:0/3193303324 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fdb5194c0 con 0x560fdb5c6d40 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f171cff9640 10 client.25502 handle_command_reply: tid=0 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 client.25502 shutdown 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 msgr2=0x560fdb5bf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5bf170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1730004700 tx=0x7f173000bda0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 shutdown_connections 2024-08-19T01:38:02.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560fdb5c6d40 0x560fdb6338f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fdb5b71d0 0x560fdb5b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fdb5ac910 0x560fdb5bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 --2- 172.21.15.103:0/3193303324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560fdb5abff0 0x560fdb5bf170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 >> 172.21.15.103:0/3193303324 conn(0x560fdb5a8580 msgr2=0x560fdb5b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f171cff9640 0 client.25502 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 shutdown_connections 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 1 -- 172.21.15.103:0/3193303324 wait complete. 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.106+0000 7f173d8b6740 20 client.25502 trim_cache size 0 max 16384 2024-08-19T01:38:02.115 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:38:02.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17200a2c00 msgr2=0x7f17200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17200a2c00 0x7f17200a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f172c022000 tx=0x7f172c01f000 comp rx=0 tx=0).stop 2024-08-19T01:38:02.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 msgr2=0x7f1734198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:02.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f1734198770 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1724031440 tx=0x7f1724004270 comp rx=0 tx=0).stop 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 shutdown_connections 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17200a2c00 0x7f17200a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1734107fd0 0x7f173419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1734103410 0x7f1734198770 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 --2- 172.21.15.103:0/2009680065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1734102a60 0x7f1734198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 >> 172.21.15.103:0/2009680065 conn(0x7f17340fc500 msgr2=0x7f1734105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 shutdown_connections 2024-08-19T01:38:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:02.107+0000 7f173b4b8640 1 -- 172.21.15.103:0/2009680065 wait complete. 2024-08-19T01:38:02.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:01 smithi146 ceph-mon[22098]: pgmap v406: 97 pgs: 97 active+clean; 2.1 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:38:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:01 smithi190 ceph-mon[22032]: pgmap v406: 97 pgs: 97 active+clean; 2.1 GiB data, 818 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:38:02.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:38:03.222 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:03.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 -- 172.21.15.103:0/2149827269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4102c20 msgr2=0x7f42d410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 --2- 172.21.15.103:0/2149827269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4102c20 0x7f42d410d770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009ca0 tx=0x7f42cc02a950 comp rx=0 tx=0).stop 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 -- 172.21.15.103:0/2149827269 shutdown_connections 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 --2- 172.21.15.103:0/2149827269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4102c20 0x7f42d410d770 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 --2- 172.21.15.103:0/2149827269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42d41022a0 0x7f42d41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 --2- 172.21.15.103:0/2149827269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4100b50 0x7f42d4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 -- 172.21.15.103:0/2149827269 >> 172.21.15.103:0/2149827269 conn(0x7f42d40fc500 msgr2=0x7f42d40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 -- 172.21.15.103:0/2149827269 shutdown_connections 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.736+0000 7f42db3c3640 1 -- 172.21.15.103:0/2149827269 wait complete. 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 Processor -- start 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 -- start start 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4100b50 0x7f42d41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42d41022a0 0x7f42d41985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4102c20 0x7f42d419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4110b60 con 0x7f42d4100b50 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41109e0 con 0x7f42d41022a0 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42db3c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4110ce0 con 0x7f42d4102c20 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4100b50 0x7f42d41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4100b50 0x7f42d41980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44004/0 (socket says 172.21.15.103:44004) 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42d915f640 1 -- 172.21.15.103:0/3049806186 learned_addr learned my addr 172.21.15.103:0/3049806186 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.737+0000 7f42d9960640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4102c20 0x7f42d419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d895e640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42d41022a0 0x7f42d41985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d915f640 1 -- 172.21.15.103:0/3049806186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4102c20 msgr2=0x7f42d419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d915f640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4102c20 0x7f42d419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d915f640 1 -- 172.21.15.103:0/3049806186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42d41022a0 msgr2=0x7f42d41985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d915f640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42d41022a0 0x7f42d41985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d915f640 1 -- 172.21.15.103:0/3049806186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d419d050 con 0x7f42d4100b50 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d895e640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42d41022a0 0x7f42d41985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42d915f640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4100b50 0x7f42d41980a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d950 tx=0x7f42d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42ca7fc640 1 -- 172.21.15.103:0/3049806186 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0017030 con 0x7f42d4100b50 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d419d340 con 0x7f42d4100b50 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42ca7fc640 1 -- 172.21.15.103:0/3049806186 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d0004020 con 0x7f42d4100b50 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42ca7fc640 1 -- 172.21.15.103:0/3049806186 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00047a0 con 0x7f42d4100b50 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.738+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d40fc8e0 con 0x7f42d4100b50 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42ca7fc640 1 -- 172.21.15.103:0/3049806186 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f42d00041d0 con 0x7f42d4100b50 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 Processor -- start 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 -- start start 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd8ff0 0x555af5cd93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5ce65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555af5c3ea40 con 0x555af5cd9910 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555af5bd69d0 con 0x555af5cd8ff0 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.741+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555af5c3aa00 con 0x555af5ce41d0 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d895e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5ce65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d9960640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd8ff0 0x555af5cd93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44012/0 (socket says 172.21.15.103:44012) 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d915f640 1 -- 172.21.15.103:0/4182258499 learned_addr learned my addr 172.21.15.103:0/4182258499 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d895e640 1 -- 172.21.15.103:0/4182258499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd8ff0 msgr2=0x555af5cd93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d895e640 1 --2- 172.21.15.103:0/4182258499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd8ff0 0x555af5cd93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d895e640 1 -- 172.21.15.103:0/4182258499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 msgr2=0x555af5ce3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d895e640 1 --2- 172.21.15.103:0/4182258499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d895e640 1 -- 172.21.15.103:0/4182258499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af5bdd650 con 0x555af5ce41d0 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42ca7fc640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f42c00a2b80 0x7f42c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d9960640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f42c00a2b80 0x7f42c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.742+0000 7f42d895e640 1 --2- 172.21.15.103:0/4182258499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5ce65c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f42c4005fd0 tx=0x7f42c400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f5022d22f26074 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.743+0000 7f42ca7fc640 1 -- 172.21.15.103:0/3049806186 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f42d00ceaf0 con 0x7f42d4100b50 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.743+0000 7f42bf7fe640 1 -- 172.21.15.103:0/4182258499 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4015070 con 0x555af5ce41d0 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.743+0000 7f42bf7fe640 1 -- 172.21.15.103:0/4182258499 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c40035f0 con 0x555af5ce41d0 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.743+0000 7f42bf7fe640 1 -- 172.21.15.103:0/4182258499 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c40116a0 con 0x555af5ce41d0 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 -- 172.21.15.103:0/4182258499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 msgr2=0x555af5ce65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42d9960640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f42c00a2b80 0x7f42c00a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009430 tx=0x7f42cc005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/4182258499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5ce65c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f42c4005fd0 tx=0x7f42c400f730 comp rx=0 tx=0).stop 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 -- 172.21.15.103:0/4182258499 shutdown_connections 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/4182258499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5ce65c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/4182258499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/4182258499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd8ff0 0x555af5cd93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 -- 172.21.15.103:0/4182258499 >> 172.21.15.103:0/4182258499 conn(0x555af5cd5580 msgr2=0x555af5cd5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 -- 172.21.15.103:0/4182258499 shutdown_connections 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 -- 172.21.15.103:0/4182258499 wait complete. 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 -- start start 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.744+0000 7f42dd7c1740 1 Processor -- start 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5cd8ff0 0x555af5ce1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5ce41d0 0x555af5ce1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555af5ce6e40 con 0x555af5cd9910 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555af5bf9b20 con 0x555af5ce41d0 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555af5ce6fc0 con 0x555af5cd8ff0 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42dd7c1740 10 client.0 _ll_register_callbacks cb 0x555af5bf7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d895e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d895e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44022/0 (socket says 172.21.15.103:44022) 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d895e640 1 -- 172.21.15.103:0/704702154 learned_addr learned my addr 172.21.15.103:0/704702154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d915f640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5cd8ff0 0x555af5ce1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d9960640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5ce41d0 0x555af5ce1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d915f640 1 -- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5ce41d0 msgr2=0x555af5ce1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d915f640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5ce41d0 0x555af5ce1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d915f640 1 -- 172.21.15.103:0/704702154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 msgr2=0x555af5ce1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d915f640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d915f640 1 -- 172.21.15.103:0/704702154 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af59b03c0 con 0x555af5cd8ff0 2024-08-19T01:38:03.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d895e640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:03.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.745+0000 7f42d9960640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5ce41d0 0x555af5ce1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42d915f640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5cd8ff0 0x555af5ce1020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009610 tx=0x7f42d00d21c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4a1cb478201f34 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42bcff9640 1 -- 172.21.15.103:0/704702154 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00d6040 con 0x555af5cd8ff0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42bcff9640 1 -- 172.21.15.103:0/704702154 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f42d00979f0 con 0x555af5cd8ff0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42c40035f0 con 0x555af5cd8ff0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42bcff9640 1 -- 172.21.15.103:0/704702154 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42d009aec0 con 0x555af5cd8ff0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42dd7c1740 10 client.25483 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555af5cf29d0 con 0x555af5cd8ff0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42bcff9640 1 -- 172.21.15.103:0/704702154 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42c40035f0 con 0x555af5cd8ff0 2024-08-19T01:38:03.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.746+0000 7f42bcff9640 1 -- 172.21.15.103:0/704702154 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f42d0009a30 con 0x555af5cd8ff0 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42dd7c1740 10 client.25483 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42dd7c1740 10 client.25483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42dd7c1740 20 client.25483 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5cf43c0 0x555af5cf6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42dd7c1740 4 client.25483 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555af5c4bfb0 con 0x555af5cf43c0 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42d895e640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5cf43c0 0x555af5cf6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42d895e640 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5cf43c0 0x555af5cf6860 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d62c2447ff56c0fd server_cookie=b873e87521781cdc in_seq=0 out_seq=0 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.747+0000 7f42bcff9640 10 client.25483 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42bcff9640 1 -- 172.21.15.103:0/704702154 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555af5c4bfb0 con 0x555af5cf43c0 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42bcff9640 10 client.25483 handle_command_reply: tid=0 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 client.25483 shutdown 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5cd8ff0 msgr2=0x555af5ce1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5cd8ff0 0x555af5ce1020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009610 tx=0x7f42d00d21c0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 shutdown_connections 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5cf43c0 0x555af5cf6860 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5ce41d0 0x555af5ce1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd9910 0x555af5ce1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.748+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/704702154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5cd8ff0 0x555af5ce1020 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.749+0000 7f42bcff9640 0 client.25483 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.749+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 >> 172.21.15.103:0/704702154 conn(0x555af5cd5580 msgr2=0x555af5cdb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.749+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 shutdown_connections 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.749+0000 7f42dd7c1740 1 -- 172.21.15.103:0/704702154 wait complete. 2024-08-19T01:38:03.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.749+0000 7f42dd7c1740 20 client.25483 trim_cache size 0 max 16384 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.764+0000 7f42dd7c1740 1 Processor -- start 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42dd7c1740 1 -- start start 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5ce3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 0x555af5cd5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5cfc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41af2c0 con 0x555af5cd8ff0 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41b0e60 con 0x555af5cd9910 2024-08-19T01:38:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41af990 con 0x555af5ce41d0 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5ce3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5ce3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44034/0 (socket says 172.21.15.103:44034) 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 -- 172.21.15.103:0/3320502392 learned_addr learned my addr 172.21.15.103:0/3320502392 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 -- 172.21.15.103:0/3320502392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 msgr2=0x555af5cfc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 --2- 172.21.15.103:0/3320502392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5cfc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 -- 172.21.15.103:0/3320502392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 msgr2=0x555af5cd5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 --2- 172.21.15.103:0/3320502392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 0x555af5cd5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 -- 172.21.15.103:0/3320502392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af5cd9740 con 0x555af5cd8ff0 2024-08-19T01:38:03.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.765+0000 7f42d915f640 1 --2- 172.21.15.103:0/3320502392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5ce3780 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005fd0 tx=0x7f42d000df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32bf36d48482b21 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.766+0000 7f42beffd640 1 -- 172.21.15.103:0/3320502392 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d009ace0 con 0x555af5cd8ff0 2024-08-19T01:38:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.766+0000 7f42beffd640 1 -- 172.21.15.103:0/3320502392 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d009ba80 con 0x555af5cd8ff0 2024-08-19T01:38:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.766+0000 7f42beffd640 1 -- 172.21.15.103:0/3320502392 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00979f0 con 0x555af5cd8ff0 2024-08-19T01:38:03.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.766+0000 7f42dd7c1740 1 -- 172.21.15.103:0/3320502392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 msgr2=0x555af5ce3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.766+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/3320502392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5ce3780 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005fd0 tx=0x7f42d000df40 comp rx=0 tx=0).stop 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 -- 172.21.15.103:0/3320502392 shutdown_connections 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/3320502392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5cfc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/3320502392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 0x555af5cd5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/3320502392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5ce3780 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 -- 172.21.15.103:0/3320502392 >> 172.21.15.103:0/3320502392 conn(0x555af5cd5580 msgr2=0x555af5ce0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 -- 172.21.15.103:0/3320502392 shutdown_connections 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 -- 172.21.15.103:0/3320502392 wait complete. 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 -- start start 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 Processor -- start 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5cec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 0x555af5cec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.767+0000 7f42dd7c1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5cdd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41b0760 con 0x555af5cd8ff0 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41b0060 con 0x555af5cd9910 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42dd7c1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42d41ad2a0 con 0x555af5ce41d0 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5cec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5cec4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44036/0 (socket says 172.21.15.103:44036) 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 -- 172.21.15.103:0/2529873213 learned_addr learned my addr 172.21.15.103:0/2529873213 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d9960640 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5cdd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42dd7c1740 10 client.0 _ll_register_callbacks cb 0x7f42d00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d895e640 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 0x555af5cec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 -- 172.21.15.103:0/2529873213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 msgr2=0x555af5cdd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5cdd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 -- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 msgr2=0x555af5cec9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 0x555af5cec9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 -- 172.21.15.103:0/2529873213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af5cddae0 con 0x555af5cd8ff0 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42d915f640 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5cec4a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f42d00977e0 tx=0x7f42d0009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d890d39cb6332c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42bcff9640 1 -- 172.21.15.103:0/2529873213 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d009ace0 con 0x555af5cd8ff0 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42bcff9640 1 -- 172.21.15.103:0/2529873213 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f42d009a7d0 con 0x555af5cd8ff0 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.768+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d009ba80 con 0x555af5cd8ff0 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42bcff9640 1 -- 172.21.15.103:0/2529873213 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42d00d22c0 con 0x555af5cd8ff0 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 10 client.16026 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:03.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555af5cddae0 con 0x555af5cd8ff0 2024-08-19T01:38:03.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42bcff9640 1 -- 172.21.15.103:0/2529873213 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d009ba80 con 0x555af5cd8ff0 2024-08-19T01:38:03.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42bcff9640 1 -- 172.21.15.103:0/2529873213 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f42d0097160 con 0x555af5cd8ff0 2024-08-19T01:38:03.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 10 client.16026 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:03.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:03.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 20 client.16026 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5d5e140 0x555af5d60530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 4 client.16026 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.769+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555af5c464c0 con 0x555af5d5e140 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42d895e640 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5d5e140 0x555af5d60530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42d895e640 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5d5e140 0x555af5d60530 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de190e36058731ea server_cookie=c4abce1e442648e5 in_seq=0 out_seq=0 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42bcff9640 10 client.16026 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42bcff9640 1 -- 172.21.15.103:0/2529873213 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555af5c464c0 con 0x555af5d5e140 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42bcff9640 10 client.16026 handle_command_reply: tid=0 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42dd7c1740 1 client.16026 shutdown 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 msgr2=0x555af5cec4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5cec4a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f42d00977e0 tx=0x7f42d0009740 comp rx=0 tx=0).stop 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 shutdown_connections 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.770+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555af5d5e140 0x555af5d60530 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555af5ce41d0 0x555af5cdd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555af5cd9910 0x555af5cec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42dd7c1740 1 --2- 172.21.15.103:0/2529873213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555af5cd8ff0 0x555af5cec4a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 >> 172.21.15.103:0/2529873213 conn(0x555af5cd5580 msgr2=0x555af5cdaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42bcff9640 0 client.16026 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 shutdown_connections 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42dd7c1740 1 -- 172.21.15.103:0/2529873213 wait complete. 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42dd7c1740 20 client.16026 trim_cache size 0 max 16384 2024-08-19T01:38:03.777 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:38:03.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f42c00a2b80 msgr2=0x7f42c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42db3c3640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f42c00a2b80 0x7f42c00a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009430 tx=0x7f42cc005c50 comp rx=0 tx=0).stop 2024-08-19T01:38:03.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4100b50 msgr2=0x7f42d41980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.771+0000 7f42db3c3640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4100b50 0x7f42d41980a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d950 tx=0x7f42d000de10 comp rx=0 tx=0).stop 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 shutdown_connections 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f42c00a2b80 0x7f42c00a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4102c20 0x7f42d419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42d41022a0 0x7f42d41985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 --2- 172.21.15.103:0/3049806186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42d4100b50 0x7f42d41980a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 >> 172.21.15.103:0/3049806186 conn(0x7f42d40fc500 msgr2=0x7f42d4069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 shutdown_connections 2024-08-19T01:38:03.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:03.772+0000 7f42db3c3640 1 -- 172.21.15.103:0/3049806186 wait complete. 2024-08-19T01:38:04.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:03 smithi146 ceph-mon[22098]: pgmap v407: 97 pgs: 97 active+clean; 2.2 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:38:04.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:03 smithi103 ceph-mon[18436]: pgmap v407: 97 pgs: 97 active+clean; 2.2 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:38:04.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:03 smithi190 ceph-mon[22032]: pgmap v407: 97 pgs: 97 active+clean; 2.2 GiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:38:05.516 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:38:05.516 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:38:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.814+0000 7f8311d4d640 1 -- 172.21.15.103:0/277191503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c102a80 msgr2=0x7f830c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.814+0000 7f8311d4d640 1 --2- 172.21.15.103:0/277191503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c102a80 0x7f830c102f00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f830000fb10 tx=0x7f830002e570 comp rx=0 tx=0).stop 2024-08-19T01:38:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 -- 172.21.15.103:0/277191503 shutdown_connections 2024-08-19T01:38:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 --2- 172.21.15.103:0/277191503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f830c103440 0x7f830c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 --2- 172.21.15.103:0/277191503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c102a80 0x7f830c102f00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 --2- 172.21.15.103:0/277191503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830c101890 0x7f830c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 -- 172.21.15.103:0/277191503 >> 172.21.15.103:0/277191503 conn(0x7f830c0fcff0 msgr2=0x7f830c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 -- 172.21.15.103:0/277191503 shutdown_connections 2024-08-19T01:38:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 -- 172.21.15.103:0/277191503 wait complete. 2024-08-19T01:38:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 Processor -- start 2024-08-19T01:38:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.815+0000 7f8311d4d640 1 -- start start 2024-08-19T01:38:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f8311d4d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830c101890 0x7f830c195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f8311d4d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f830c102a80 0x7f830c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f8311d4d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c103440 0x7f830c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f8311d4d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10b230 con 0x7f830c103440 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f8311d4d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10b0b0 con 0x7f830c102a80 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f8311d4d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10b3b0 con 0x7f830c101890 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c103440 0x7f830c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c103440 0x7f830c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44062/0 (socket says 172.21.15.103:44062) 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 -- 172.21.15.103:0/2530670322 learned_addr learned my addr 172.21.15.103:0/2530670322 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830b7fe640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830c101890 0x7f830c195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830affd640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f830c102a80 0x7f830c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 -- 172.21.15.103:0/2530670322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830c101890 msgr2=0x7f830c195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830c101890 0x7f830c195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 -- 172.21.15.103:0/2530670322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f830c102a80 msgr2=0x7f830c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f830c102a80 0x7f830c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 -- 172.21.15.103:0/2530670322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c19d490 con 0x7f830c103440 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.816+0000 7f830bfff640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c103440 0x7f830c19ce20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f830400efd0 tx=0x7f830400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.817+0000 7f8308ff9640 1 -- 172.21.15.103:0/2530670322 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304003b70 con 0x7f830c103440 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.817+0000 7f8308ff9640 1 -- 172.21.15.103:0/2530670322 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8304003d10 con 0x7f830c103440 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.817+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c19d780 con 0x7f830c103440 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.817+0000 7f8308ff9640 1 -- 172.21.15.103:0/2530670322 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304010980 con 0x7f830c103440 2024-08-19T01:38:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.817+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c19dc80 con 0x7f830c103440 2024-08-19T01:38:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.819+0000 7f8308ff9640 1 -- 172.21.15.103:0/2530670322 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8304004020 con 0x7f830c103440 2024-08-19T01:38:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 Processor -- start 2024-08-19T01:38:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 -- start start 2024-08-19T01:38:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d326a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d32ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d32e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f51d2899d0 con 0x55f51d326680 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f51d231820 con 0x55f51d32c050 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8314bea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f51d285990 con 0x55f51d326fc0 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.820+0000 7f8308ff9640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f82f40a2c30 0x7f82f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d326a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830bfff640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f82f40a2c30 0x7f82f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d32ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d326a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44078/0 (socket says 172.21.15.103:44078) 2024-08-19T01:38:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 -- 172.21.15.103:0/2245622732 learned_addr learned my addr 172.21.15.103:0/2245622732 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f8308ff9640 1 -- 172.21.15.103:0/2530670322 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f83040cf500 con 0x7f830c103440 2024-08-19T01:38:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d32e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 -- 172.21.15.103:0/2245622732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 msgr2=0x55f51d32ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d32ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 -- 172.21.15.103:0/2245622732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 msgr2=0x55f51d32e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d32e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.821+0000 7f830affd640 1 -- 172.21.15.103:0/2245622732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51cf9d210 con 0x55f51d326680 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.822+0000 7f830bfff640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f82f40a2c30 0x7f82f40a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8304098760 tx=0x7f8304097990 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.822+0000 7f830affd640 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d326a80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8300005940 tx=0x7f8300005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad22f6c5f0656605 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.822+0000 7f82f1ffb640 1 -- 172.21.15.103:0/2245622732 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300048020 con 0x55f51d326680 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.822+0000 7f82f1ffb640 1 -- 172.21.15.103:0/2245622732 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8300004900 con 0x55f51d326680 2024-08-19T01:38:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.822+0000 7f82f1ffb640 1 -- 172.21.15.103:0/2245622732 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000f6d0 con 0x55f51d326680 2024-08-19T01:38:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 -- 172.21.15.103:0/2245622732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 msgr2=0x55f51d326a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d326a80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8300005940 tx=0x7f8300005d50 comp rx=0 tx=0).stop 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 -- 172.21.15.103:0/2245622732 shutdown_connections 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d32e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d32ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 --2- 172.21.15.103:0/2245622732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d326a80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 -- 172.21.15.103:0/2245622732 >> 172.21.15.103:0/2245622732 conn(0x55f51d321cf0 msgr2=0x55f51d3220f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 -- 172.21.15.103:0/2245622732 shutdown_connections 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.823+0000 7f8314bea740 1 -- 172.21.15.103:0/2245622732 wait complete. 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 -- start start 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 Processor -- start 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d330860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d330da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d337de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f51d32efc0 con 0x55f51d326fc0 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f51d245520 con 0x55f51d326680 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.824+0000 7f8314bea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f51d32f140 con 0x55f51d32c050 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d330da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830affd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d330860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f8314bea740 10 client.0 _ll_register_callbacks cb 0x55f51cf9ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830affd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d330860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:43430/0 (socket says 172.21.15.103:43430) 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830affd640 1 -- 172.21.15.103:0/3515034194 learned_addr learned my addr 172.21.15.103:0/3515034194 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830affd640 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d337de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830b7fe640 1 -- 172.21.15.103:0/3515034194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 msgr2=0x55f51d337de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830b7fe640 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d337de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830b7fe640 1 -- 172.21.15.103:0/3515034194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 msgr2=0x55f51d330860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830b7fe640 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d330860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.825+0000 7f830b7fe640 1 -- 172.21.15.103:0/3515034194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51d069620 con 0x55f51d326fc0 2024-08-19T01:38:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.826+0000 7f830b7fe640 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d330da0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55f51d3279f0 tx=0x7f82fc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba62847c4a643e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.826+0000 7f82eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.826+0000 7f82eaffd640 1 -- 172.21.15.103:0/3515034194 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00bdc0 con 0x55f51d326fc0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.826+0000 7f82eaffd640 1 -- 172.21.15.103:0/3515034194 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f82fc011440 con 0x55f51d326fc0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.826+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8300004900 con 0x55f51d326fc0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.826+0000 7f82eaffd640 1 -- 172.21.15.103:0/3515034194 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82fc011640 con 0x55f51d326fc0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 10 client.16038 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f51d33aca0 con 0x55f51d326fc0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f82eaffd640 1 -- 172.21.15.103:0/3515034194 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82fc011c90 con 0x55f51d326fc0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f82eaffd640 1 -- 172.21.15.103:0/3515034194 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f82fc026aa0 con 0x55f51d326fc0 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 10 client.16038 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 20 client.16038 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d33c370 0x55f51d33e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 4 client.16038 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.827+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8300004900 con 0x55f51d33c370 2024-08-19T01:38:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.828+0000 7f830affd640 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d33c370 0x55f51d33e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.829+0000 7f82eaffd640 10 client.16038 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.829+0000 7f830affd640 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d33c370 0x55f51d33e780 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed01af9c5f711b5 server_cookie=f3c15173e293b3b3 in_seq=0 out_seq=0 2024-08-19T01:38:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.830+0000 7f82eaffd640 1 -- 172.21.15.103:0/3515034194 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8300004900 con 0x55f51d33c370 2024-08-19T01:38:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.830+0000 7f82eaffd640 10 client.16038 handle_command_reply: tid=0 2024-08-19T01:38:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.830+0000 7f8314bea740 1 client.16038 shutdown 2024-08-19T01:38:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.830+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 msgr2=0x55f51d330da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.830+0000 7f8314bea740 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d330da0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55f51d3279f0 tx=0x7f82fc0039c0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.830+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 shutdown_connections 2024-08-19T01:38:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d33c370 0x55f51d33e780 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f82eaffd640 0 client.16038 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d337de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d330da0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 1 --2- 172.21.15.103:0/3515034194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d330860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 >> 172.21.15.103:0/3515034194 conn(0x55f51d321cf0 msgr2=0x55f51d323500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 shutdown_connections 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 1 -- 172.21.15.103:0/3515034194 wait complete. 2024-08-19T01:38:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.831+0000 7f8314bea740 20 client.16038 trim_cache size 0 max 16384 2024-08-19T01:38:05.840 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 Processor -- start 2024-08-19T01:38:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 -- start start 2024-08-19T01:38:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d326a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d337da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d320d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1a68d0 con 0x55f51d326fc0 2024-08-19T01:38:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1a8470 con 0x55f51d326680 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f8314bea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1a6fa0 con 0x55f51d32c050 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d337da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f830b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d337da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44088/0 (socket says 172.21.15.103:44088) 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.850+0000 7f830b7fe640 1 -- 172.21.15.103:0/799444572 learned_addr learned my addr 172.21.15.103:0/799444572 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830affd640 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d320d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830affd640 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d326a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830b7fe640 1 -- 172.21.15.103:0/799444572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 msgr2=0x55f51d320d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830b7fe640 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d320d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830b7fe640 1 -- 172.21.15.103:0/799444572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 msgr2=0x55f51d326a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830b7fe640 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d326a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830b7fe640 1 -- 172.21.15.103:0/799444572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51d32e440 con 0x55f51d326fc0 2024-08-19T01:38:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830affd640 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d320d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f830b7fe640 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d337da0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0041a0 tx=0x7f82fc00b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af0e323edde67e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f82f17fa640 1 -- 172.21.15.103:0/799444572 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00bdc0 con 0x55f51d326fc0 2024-08-19T01:38:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f82f17fa640 1 -- 172.21.15.103:0/799444572 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82fc0054f0 con 0x55f51d326fc0 2024-08-19T01:38:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.851+0000 7f82f17fa640 1 -- 172.21.15.103:0/799444572 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc004680 con 0x55f51d326fc0 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 -- 172.21.15.103:0/799444572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 msgr2=0x55f51d337da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d337da0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0041a0 tx=0x7f82fc00b400 comp rx=0 tx=0).stop 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 -- 172.21.15.103:0/799444572 shutdown_connections 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d32c050 0x55f51d320d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326fc0 0x55f51d337da0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 --2- 172.21.15.103:0/799444572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d326680 0x55f51d326a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 -- 172.21.15.103:0/799444572 >> 172.21.15.103:0/799444572 conn(0x55f51d321cf0 msgr2=0x55f51d320680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 -- 172.21.15.103:0/799444572 shutdown_connections 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 -- 172.21.15.103:0/799444572 wait complete. 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.852+0000 7f8314bea740 1 -- start start 2024-08-19T01:38:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 1 Processor -- start 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d3351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d3356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d336700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1a7d70 con 0x55f51d326680 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1a7670 con 0x55f51d32c050 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1a48b0 con 0x55f51d326fc0 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f8314bea740 10 client.0 _ll_register_callbacks cb 0x7f82fc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d3351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d3351b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44094/0 (socket says 172.21.15.103:44094) 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 -- 172.21.15.103:0/1342400249 learned_addr learned my addr 172.21.15.103:0/1342400249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830b7fe640 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d3356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d336700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 -- 172.21.15.103:0/1342400249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 msgr2=0x55f51d3356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d3356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 -- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 msgr2=0x55f51d336700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d336700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.853+0000 7f830affd640 1 -- 172.21.15.103:0/1342400249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51d336d10 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f830affd640 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d3351b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8300005940 tx=0x7f8300005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb96b3b8f79c6ff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f82eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f82eaffd640 1 -- 172.21.15.103:0/1342400249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300061060 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82fc0054f0 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f82eaffd640 1 -- 172.21.15.103:0/1342400249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f830005de80 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f82eaffd640 1 -- 172.21.15.103:0/1342400249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8300048070 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f8314bea740 10 client.16044 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.854+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f51d337f90 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f82eaffd640 1 -- 172.21.15.103:0/1342400249 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82fc0054f0 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f82eaffd640 1 -- 172.21.15.103:0/1342400249 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f830005d590 con 0x55f51d326680 2024-08-19T01:38:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f8314bea740 10 client.16044 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f8314bea740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f8314bea740 20 client.16044 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f8314bea740 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d3a7200 0x55f51d3a9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f8314bea740 4 client.16044 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f51d297100 con 0x55f51d3a7200 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f830b7fe640 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d3a7200 0x55f51d3a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f82eaffd640 10 client.16044 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.855+0000 7f830b7fe640 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d3a7200 0x55f51d3a9620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fadba71a6a04b09d server_cookie=47a4f6a31fe60434 in_seq=0 out_seq=0 2024-08-19T01:38:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.862+0000 7f82eaffd640 1 -- 172.21.15.103:0/1342400249 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f51d297100 con 0x55f51d3a7200 2024-08-19T01:38:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.862+0000 7f82eaffd640 10 client.16044 handle_command_reply: tid=0 2024-08-19T01:38:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 client.16044 shutdown 2024-08-19T01:38:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 msgr2=0x55f51d3351b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d3351b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8300005940 tx=0x7f8300005fd0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 shutdown_connections 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f51d3a7200 0x55f51d3a9620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f51d32c050 0x55f51d336700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f51d326fc0 0x55f51d3356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 --2- 172.21.15.103:0/1342400249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f51d326680 0x55f51d3351b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 >> 172.21.15.103:0/1342400249 conn(0x55f51d321cf0 msgr2=0x55f51d320ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f82eaffd640 0 client.16044 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 shutdown_connections 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 1 -- 172.21.15.103:0/1342400249 wait complete. 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.863+0000 7f8314bea740 20 client.16044 trim_cache size 0 max 16384 2024-08-19T01:38:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:38:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:38:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:38:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:38:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f82f40a2c30 msgr2=0x7f82f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f82f40a2c30 0x7f82f40a50f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8304098760 tx=0x7f8304097990 comp rx=0 tx=0).stop 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c103440 msgr2=0x7f830c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c103440 0x7f830c19ce20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f830400efd0 tx=0x7f830400c6a0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 shutdown_connections 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f82f40a2c30 0x7f82f40a50f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f830c103440 0x7f830c19ce20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f830c102a80 0x7f830c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 --2- 172.21.15.103:0/2530670322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f830c101890 0x7f830c195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 >> 172.21.15.103:0/2530670322 conn(0x7f830c0fcff0 msgr2=0x7f830c105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 shutdown_connections 2024-08-19T01:38:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:05.864+0000 7f8311d4d640 1 -- 172.21.15.103:0/2530670322 wait complete. 2024-08-19T01:38:06.155 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:05 smithi103 ceph-mon[18436]: pgmap v408: 97 pgs: 97 active+clean; 2.2 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:06.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:05 smithi146 ceph-mon[22098]: pgmap v408: 97 pgs: 97 active+clean; 2.2 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:05 smithi190 ceph-mon[22032]: pgmap v408: 97 pgs: 97 active+clean; 2.2 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:06.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.387+0000 7f3f5fae7640 1 -- 172.21.15.103:0/3498588603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58102a70 msgr2=0x7f3f58102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.387+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/3498588603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58102a70 0x7f3f58102e50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009900 tx=0x7f3f5402a480 comp rx=0 tx=0).stop 2024-08-19T01:38:06.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.387+0000 7f3f5fae7640 1 -- 172.21.15.103:0/3498588603 shutdown_connections 2024-08-19T01:38:06.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.387+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/3498588603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f5810bea0 0x7f3f5810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.387+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/3498588603 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f58103390 0x7f3f5810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.387+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/3498588603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58102a70 0x7f3f58102e50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.387+0000 7f3f5fae7640 1 -- 172.21.15.103:0/3498588603 >> 172.21.15.103:0/3498588603 conn(0x7f3f580fc4d0 msgr2=0x7f3f580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.388+0000 7f3f5fae7640 1 -- 172.21.15.103:0/3498588603 shutdown_connections 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.388+0000 7f3f5fae7640 1 -- 172.21.15.103:0/3498588603 wait complete. 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.388+0000 7f3f5fae7640 1 Processor -- start 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.388+0000 7f3f5fae7640 1 -- start start 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5fae7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f58102a70 0x7f3f58198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5fae7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58103390 0x7f3f58198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5fae7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f5810bea0 0x7f3f5819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5fae7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58111b50 con 0x7f3f58103390 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5fae7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581119d0 con 0x7f3f5810bea0 2024-08-19T01:38:06.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5fae7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58111cd0 con 0x7f3f58102a70 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58103390 0x7f3f58198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58103390 0x7f3f58198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44120/0 (socket says 172.21.15.103:44120) 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d883640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f58102a70 0x7f3f58198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 -- 172.21.15.103:0/710538407 learned_addr learned my addr 172.21.15.103:0/710538407 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5e084640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f5810bea0 0x7f3f5819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 -- 172.21.15.103:0/710538407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f58102a70 msgr2=0x7f3f58198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f58102a70 0x7f3f58198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 -- 172.21.15.103:0/710538407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f5810bea0 msgr2=0x7f3f5819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f5810bea0 0x7f3f5819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 -- 172.21.15.103:0/710538407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5819cfc0 con 0x7f3f58103390 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f5d082640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58103390 0x7f3f58198550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800c970 tx=0x7f3f4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:06.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.389+0000 7f3f4effd640 1 -- 172.21.15.103:0/710538407 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48007bb0 con 0x7f3f58103390 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.390+0000 7f3f4effd640 1 -- 172.21.15.103:0/710538407 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f48007d50 con 0x7f3f58103390 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.390+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5819d2b0 con 0x7f3f58103390 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.390+0000 7f3f4effd640 1 -- 172.21.15.103:0/710538407 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48018780 con 0x7f3f58103390 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.390+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f581a4b90 con 0x7f3f58103390 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.392+0000 7f3f4effd640 1 -- 172.21.15.103:0/710538407 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3f4800f450 con 0x7f3f58103390 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.393+0000 7f3f4effd640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f440a2b80 0x7f3f440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.393+0000 7f3f61ee5740 1 Processor -- start 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.393+0000 7f3f4effd640 1 -- 172.21.15.103:0/710538407 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3f480cef90 con 0x7f3f58103390 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.393+0000 7f3f5d883640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f440a2b80 0x7f3f440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.393+0000 7f3f61ee5740 1 -- start start 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b700475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70051ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b700547d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b7000ae40 con 0x560b700523e0 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560b6ffac7f0 con 0x560b70047200 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560b6ff53fb0 con 0x560b70047b20 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b700547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5d883640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f440a2b80 0x7f3f440a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f54031440 tx=0x7f3f54005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b700547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44132/0 (socket says 172.21.15.103:44132) 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 -- 172.21.15.103:0/4268289713 learned_addr learned my addr 172.21.15.103:0/4268289713 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5d082640 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70051ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5d883640 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b700475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 -- 172.21.15.103:0/4268289713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 msgr2=0x560b70051ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70051ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 -- 172.21.15.103:0/4268289713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 msgr2=0x560b700475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b700475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 -- 172.21.15.103:0/4268289713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6fd1e6a0 con 0x560b700523e0 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5d883640 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b700475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.394+0000 7f3f5e084640 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b700547d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009f70 tx=0x7f3f50010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3047bd0baa7a3ca5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.395+0000 7f3f43fff640 1 -- 172.21.15.103:0/4268289713 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50016070 con 0x560b700523e0 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.395+0000 7f3f43fff640 1 -- 172.21.15.103:0/4268289713 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f50004d20 con 0x560b700523e0 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.395+0000 7f3f43fff640 1 -- 172.21.15.103:0/4268289713 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f500127d0 con 0x560b700523e0 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.395+0000 7f3f61ee5740 1 -- 172.21.15.103:0/4268289713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 msgr2=0x560b700547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.395+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b700547d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009f70 tx=0x7f3f50010800 comp rx=0 tx=0).stop 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 -- 172.21.15.103:0/4268289713 shutdown_connections 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b700547d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70051ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/4268289713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b700475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 -- 172.21.15.103:0/4268289713 >> 172.21.15.103:0/4268289713 conn(0x560b70043790 msgr2=0x560b70043b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 -- 172.21.15.103:0/4268289713 shutdown_connections 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 -- 172.21.15.103:0/4268289713 wait complete. 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 -- start start 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 Processor -- start 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.396+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047200 0x560b7004f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047b20 0x560b7004f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b7004fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b6ff67cb0 con 0x560b700523e0 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560b6ffa87b0 con 0x560b70047b20 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560b70054fb0 con 0x560b70047200 2024-08-19T01:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f61ee5740 10 client.0 _ll_register_callbacks cb 0x560b6fcbbba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b7004fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b7004fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44142/0 (socket says 172.21.15.103:44142) 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 -- 172.21.15.103:0/3176195049 learned_addr learned my addr 172.21.15.103:0/3176195049 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5d883640 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047200 0x560b7004f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5d082640 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047b20 0x560b7004f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 -- 172.21.15.103:0/3176195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047200 msgr2=0x560b7004f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047200 0x560b7004f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 -- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047b20 msgr2=0x560b7004f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047b20 0x560b7004f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 -- 172.21.15.103:0/3176195049 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6ff66bc0 con 0x560b700523e0 2024-08-19T01:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f5e084640 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b7004fd70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3f500058e0 tx=0x7f3f50004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e59df9f43a5638 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.397+0000 7f3f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.398+0000 7f3f417fa640 1 -- 172.21.15.103:0/3176195049 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50016070 con 0x560b700523e0 2024-08-19T01:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.398+0000 7f3f417fa640 1 -- 172.21.15.103:0/3176195049 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3f500051f0 con 0x560b700523e0 2024-08-19T01:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f50004d20 con 0x560b700523e0 2024-08-19T01:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f417fa640 1 -- 172.21.15.103:0/3176195049 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f50005410 con 0x560b700523e0 2024-08-19T01:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 10 client.16056 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b70060e10 con 0x560b700523e0 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f417fa640 1 -- 172.21.15.103:0/3176195049 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f500055b0 con 0x560b700523e0 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f417fa640 1 -- 172.21.15.103:0/3176195049 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3f5000f040 con 0x560b700523e0 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 10 client.16056 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 20 client.16056 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b70062270 0x560b70064660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 4 client.16056 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.399+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f50004d20 con 0x560b70062270 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.400+0000 7f3f5d883640 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b70062270 0x560b70064660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.400+0000 7f3f417fa640 10 client.16056 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.400+0000 7f3f5d883640 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b70062270 0x560b70064660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2fdaf4a2e41c2b server_cookie=76a1756d69c4bc15 in_seq=0 out_seq=0 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f417fa640 1 -- 172.21.15.103:0/3176195049 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3f50004d20 con 0x560b70062270 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f417fa640 10 client.16056 handle_command_reply: tid=0 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 client.16056 shutdown 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 msgr2=0x560b7004fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b7004fd70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3f500058e0 tx=0x7f3f50004c70 comp rx=0 tx=0).stop 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 shutdown_connections 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b70062270 0x560b70064660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b7004fd70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047b20 0x560b7004f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f417fa640 0 client.16056 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/3176195049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047200 0x560b7004f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 >> 172.21.15.103:0/3176195049 conn(0x560b70043790 msgr2=0x560b70049470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 shutdown_connections 2024-08-19T01:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 1 -- 172.21.15.103:0/3176195049 wait complete. 2024-08-19T01:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.401+0000 7f3f61ee5740 20 client.16056 trim_cache size 0 max 16384 2024-08-19T01:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 Processor -- start 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 -- start start 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7004ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70049360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581ad0b0 con 0x560b700523e0 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581aec50 con 0x560b70047200 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581ad780 con 0x560b70047b20 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f5d082640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70049360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70069ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44146/0 (socket says 172.21.15.103:44146) 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.420+0000 7f3f5e084640 1 -- 172.21.15.103:0/2887791283 learned_addr learned my addr 172.21.15.103:0/2887791283 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d883640 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7004ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d082640 1 -- 172.21.15.103:0/2887791283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 msgr2=0x560b7004ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d082640 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7004ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d082640 1 -- 172.21.15.103:0/2887791283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 msgr2=0x560b70069ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d082640 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70069ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d082640 1 -- 172.21.15.103:0/2887791283 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b70051e40 con 0x560b70047b20 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5e084640 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70069ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d883640 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7004ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f5d082640 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70049360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f480ccbb0 tx=0x7f3f4809b220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d19d154a7950ae89 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f437fe640 1 -- 172.21.15.103:0/2887791283 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f480b3030 con 0x560b70047b20 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f437fe640 1 -- 172.21.15.103:0/2887791283 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f48084e50 con 0x560b70047b20 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.421+0000 7f3f437fe640 1 -- 172.21.15.103:0/2887791283 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800f740 con 0x560b70047b20 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 -- 172.21.15.103:0/2887791283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 msgr2=0x560b70049360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70049360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f480ccbb0 tx=0x7f3f4809b220 comp rx=0 tx=0).stop 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 -- 172.21.15.103:0/2887791283 shutdown_connections 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70069ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70049360 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/2887791283 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7004ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 -- 172.21.15.103:0/2887791283 >> 172.21.15.103:0/2887791283 conn(0x560b70043790 msgr2=0x560b70048c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 -- 172.21.15.103:0/2887791283 shutdown_connections 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.422+0000 7f3f61ee5740 1 -- 172.21.15.103:0/2887791283 wait complete. 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 -- start start 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 Processor -- start 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7005a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70055af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70056030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581ae550 con 0x560b700523e0 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581ade50 con 0x560b70047200 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f581ab090 con 0x560b70047b20 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70056030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f61ee5740 10 client.0 _ll_register_callbacks cb 0x7f3f50009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70056030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44152/0 (socket says 172.21.15.103:44152) 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.423+0000 7f3f5d082640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70055af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 -- 172.21.15.103:0/1098577123 learned_addr learned my addr 172.21.15.103:0/1098577123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5d883640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7005a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 -- 172.21.15.103:0/1098577123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 msgr2=0x560b70055af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70055af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 -- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 msgr2=0x560b7005a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7005a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 -- 172.21.15.103:0/1098577123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b700566a0 con 0x560b700523e0 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5d082640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70055af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5d883640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7005a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f5e084640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70056030 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3f5002f750 tx=0x7f3f50010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6143f8c0e5f21528 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f417fa640 1 -- 172.21.15.103:0/1098577123 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50034030 con 0x560b700523e0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f417fa640 1 -- 172.21.15.103:0/1098577123 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3f50021030 con 0x560b700523e0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f48084e50 con 0x560b700523e0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f417fa640 1 -- 172.21.15.103:0/1098577123 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f50016040 con 0x560b700523e0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.424+0000 7f3f61ee5740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b700ccd60 con 0x560b700523e0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f417fa640 1 -- 172.21.15.103:0/1098577123 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f5002f9e0 con 0x560b700523e0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f417fa640 1 -- 172.21.15.103:0/1098577123 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3f50012450 con 0x560b700523e0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f61ee5740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f61ee5740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f61ee5740 20 client.16062 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b700cd4a0 0x560b700cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f61ee5740 4 client.16062 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3f48084e50 con 0x560b700cd4a0 2024-08-19T01:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.425+0000 7f3f5d883640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b700cd4a0 0x560b700cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.426+0000 7f3f417fa640 10 client.16062 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.426+0000 7f3f5d883640 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b700cd4a0 0x560b700cf8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fee39cb42719262 server_cookie=e2948d1f032b6283 in_seq=0 out_seq=0 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f417fa640 1 -- 172.21.15.103:0/1098577123 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3f48084e50 con 0x560b700cd4a0 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f417fa640 10 client.16062 handle_command_reply: tid=0 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 client.16062 shutdown 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 msgr2=0x560b70056030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70056030 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3f5002f750 tx=0x7f3f50010d00 comp rx=0 tx=0).stop 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 shutdown_connections 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560b700cd4a0 0x560b700cf8c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b700523e0 0x560b70056030 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560b70047b20 0x560b70055af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 --2- 172.21.15.103:0/1098577123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560b70047200 0x560b7005a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 >> 172.21.15.103:0/1098577123 conn(0x560b70043790 msgr2=0x560b70049470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f417fa640 0 client.16062 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 shutdown_connections 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 1 -- 172.21.15.103:0/1098577123 wait complete. 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.427+0000 7f3f61ee5740 20 client.16062 trim_cache size 0 max 16384 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "4156e333-a9d3-4872-9240-d3fdb26b0662", 2024-08-19T01:38:06.433 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:38:06.434 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f440a2b80 msgr2=0x7f3f440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f440a2b80 0x7f3f440a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f54031440 tx=0x7f3f54005b50 comp rx=0 tx=0).stop 2024-08-19T01:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58103390 msgr2=0x7f3f58198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58103390 0x7f3f58198550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800c970 tx=0x7f3f4800ce30 comp rx=0 tx=0).stop 2024-08-19T01:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 shutdown_connections 2024-08-19T01:38:06.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3f440a2b80 0x7f3f440a5040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3f5810bea0 0x7f3f5819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f58103390 0x7f3f58198550 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 --2- 172.21.15.103:0/710538407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f58102a70 0x7f3f58198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:06.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 >> 172.21.15.103:0/710538407 conn(0x7f3f580fc4d0 msgr2=0x7f3f580fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:06.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 shutdown_connections 2024-08-19T01:38:06.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:06.428+0000 7f3f5fae7640 1 -- 172.21.15.103:0/710538407 wait complete. 2024-08-19T01:38:07.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:06 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:38:07.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:06 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:06 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:38:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:06 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:06 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:38:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:06 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:06 smithi103 ceph-mon[18436]: pgmap v409: 97 pgs: 97 active+clean; 2.3 GiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:06 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:38:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:06 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:06 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:38:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:06 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:06 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:38:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:06 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:06 smithi190 ceph-mon[22032]: pgmap v409: 97 pgs: 97 active+clean; 2.3 GiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:07.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:06 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:38:07.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:06 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:06 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:38:07.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:06 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:06 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:38:07.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:06 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:07.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:06 smithi146 ceph-mon[22098]: pgmap v409: 97 pgs: 97 active+clean; 2.3 GiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:08.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:07 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:38:08.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:07 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:07 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:38:08.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:07 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:38:08.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:07 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:07 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:07 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:38:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:07 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:07 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:38:08.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:07 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:38:08.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:07 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:07 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:07 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:38:08.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:07 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:07 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:38:08.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:07 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:38:08.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:07 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:08.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:07 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:38:09.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:08 smithi103 ceph-mon[18436]: pgmap v410: 97 pgs: 97 active+clean; 2.4 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 37 op/s 2024-08-19T01:38:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:08 smithi190 ceph-mon[22032]: pgmap v410: 97 pgs: 97 active+clean; 2.4 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 37 op/s 2024-08-19T01:38:09.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:08 smithi146 ceph-mon[22098]: pgmap v410: 97 pgs: 97 active+clean; 2.4 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 MiB/s wr, 37 op/s 2024-08-19T01:38:12.154 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:11 smithi146 ceph-mon[22098]: pgmap v411: 97 pgs: 97 active+clean; 2.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 42 op/s 2024-08-19T01:38:12.261 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:11 smithi103 ceph-mon[18436]: pgmap v411: 97 pgs: 97 active+clean; 2.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 42 op/s 2024-08-19T01:38:12.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:38:12.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:11 smithi190 ceph-mon[22032]: pgmap v411: 97 pgs: 97 active+clean; 2.4 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 42 op/s 2024-08-19T01:38:12.572 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:14.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:13 smithi103 ceph-mon[18436]: pgmap v412: 97 pgs: 97 active+clean; 2.5 GiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 29 MiB/s wr, 48 op/s 2024-08-19T01:38:14.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:13 smithi146 ceph-mon[22098]: pgmap v412: 97 pgs: 97 active+clean; 2.5 GiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 29 MiB/s wr, 48 op/s 2024-08-19T01:38:14.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:13 smithi190 ceph-mon[22032]: pgmap v412: 97 pgs: 97 active+clean; 2.5 GiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 29 MiB/s wr, 48 op/s 2024-08-19T01:38:16.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:15 smithi103 ceph-mon[18436]: pgmap v413: 97 pgs: 97 active+clean; 2.5 GiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-08-19T01:38:16.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:15 smithi146 ceph-mon[22098]: pgmap v413: 97 pgs: 97 active+clean; 2.5 GiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-08-19T01:38:16.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:15 smithi190 ceph-mon[22032]: pgmap v413: 97 pgs: 97 active+clean; 2.5 GiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 43 op/s 2024-08-19T01:38:16.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/280406447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b04100b50 msgr2=0x7f7b04100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/280406447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b04100b50 0x7f7b04100f30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009900 tx=0x7f7b0002a480 comp rx=0 tx=0).stop 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/280406447 shutdown_connections 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/280406447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b04102c20 0x7f7b0410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/280406447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b041022a0 0x7f7b041026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/280406447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b04100b50 0x7f7b04100f30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/280406447 >> 172.21.15.103:0/280406447 conn(0x7f7b040fc500 msgr2=0x7f7b040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/280406447 shutdown_connections 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.854+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/280406447 wait complete. 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 Processor -- start 2024-08-19T01:38:16.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 -- start start 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b04100b50 0x7f7b0419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b041022a0 0x7f7b0419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b04102c20 0x7f7b0419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04110b60 con 0x7f7b041022a0 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041109e0 con 0x7f7b04100b50 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0c1b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04110ce0 con 0x7f7b04102c20 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0974b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b041022a0 0x7f7b0419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0974b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b041022a0 0x7f7b0419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44316/0 (socket says 172.21.15.103:44316) 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b09f4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b04100b50 0x7f7b0419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0974b640 1 -- 172.21.15.103:0/423843649 learned_addr learned my addr 172.21.15.103:0/423843649 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.855+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b04102c20 0x7f7b0419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0974b640 1 -- 172.21.15.103:0/423843649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b04102c20 msgr2=0x7f7b0419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0974b640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b04102c20 0x7f7b0419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0974b640 1 -- 172.21.15.103:0/423843649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b04100b50 msgr2=0x7f7b0419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0974b640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b04100b50 0x7f7b0419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0974b640 1 -- 172.21.15.103:0/423843649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0419f350 con 0x7f7b041022a0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b04102c20 0x7f7b0419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0974b640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b041022a0 0x7f7b0419ab10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7af400d950 tx=0x7f7af400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7afaffd640 1 -- 172.21.15.103:0/423843649 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4017070 con 0x7f7b041022a0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0419f640 con 0x7f7b041022a0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7afaffd640 1 -- 172.21.15.103:0/423843649 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af4009c30 con 0x7f7b041022a0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7afaffd640 1 -- 172.21.15.103:0/423843649 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af400f620 con 0x7f7b041022a0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.856+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a6f20 con 0x7f7b041022a0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.859+0000 7f7afaffd640 1 -- 172.21.15.103:0/423843649 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7af40109a0 con 0x7f7b041022a0 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.859+0000 7f7b0e5ae740 1 Processor -- start 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0e5ae740 1 -- start start 2024-08-19T01:38:16.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf21e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf228eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf22b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bcf183980 con 0x563bcf2293f0 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563bcf11ba70 con 0x563bcf21eb30 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563bcf17f940 con 0x563bcf21e210 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0974b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf22b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf21e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf21e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:59848/0 (socket says 172.21.15.103:59848) 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 -- 172.21.15.103:0/1204478130 learned_addr learned my addr 172.21.15.103:0/1204478130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf228eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7afaffd640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7af00a2b80 0x7f7af00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 -- 172.21.15.103:0/1204478130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 msgr2=0x563bcf228eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf228eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 -- 172.21.15.103:0/1204478130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 msgr2=0x563bcf22b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf22b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.860+0000 7f7b0a74d640 1 -- 172.21.15.103:0/1204478130 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bcf17e6e0 con 0x563bcf21e210 2024-08-19T01:38:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.861+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf228eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.861+0000 7f7afaffd640 1 -- 172.21.15.103:0/423843649 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7af40cec50 con 0x7f7b041022a0 2024-08-19T01:38:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.861+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7af00a2b80 0x7f7af00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.861+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf21e5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009fd0 tx=0x7f7afc0108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4377c9c54ce16dbf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.861+0000 7f7aeffff640 1 -- 172.21.15.103:0/1204478130 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc022070 con 0x563bcf21e210 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.861+0000 7f7aeffff640 1 -- 172.21.15.103:0/1204478130 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc01b380 con 0x563bcf21e210 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.861+0000 7f7aeffff640 1 -- 172.21.15.103:0/1204478130 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00e620 con 0x563bcf21e210 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/1204478130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 msgr2=0x563bcf21e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7af00a2b80 0x7f7af00a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7afc01f000 tx=0x7f7afc01c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf21e5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009fd0 tx=0x7f7afc0108e0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/1204478130 shutdown_connections 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf22b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf228eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/1204478130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf21e5f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/1204478130 >> 172.21.15.103:0/1204478130 conn(0x563bcf21a7a0 msgr2=0x563bcf21ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/1204478130 shutdown_connections 2024-08-19T01:38:16.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/1204478130 wait complete. 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.862+0000 7f7b0e5ae740 1 -- start start 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 1 Processor -- start 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21e210 0x563bcf2262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21eb30 0x563bcf226830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf226d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bcf22bfc0 con 0x563bcf2293f0 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563bcf13eaa0 con 0x563bcf21e210 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563bcf22c140 con 0x563bcf21eb30 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0a74d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf226d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0a74d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf226d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44342/0 (socket says 172.21.15.103:44342) 2024-08-19T01:38:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0a74d640 1 -- 172.21.15.103:0/346692523 learned_addr learned my addr 172.21.15.103:0/346692523 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21e210 0x563bcf2262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0e5ae740 10 client.0 _ll_register_callbacks cb 0x563bcf12de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.863+0000 7f7b0974b640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21eb30 0x563bcf226830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b0a74d640 1 -- 172.21.15.103:0/346692523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21eb30 msgr2=0x563bcf226830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21eb30 0x563bcf226830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b0a74d640 1 -- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21e210 msgr2=0x563bcf2262f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21e210 0x563bcf2262f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b0a74d640 1 -- 172.21.15.103:0/346692523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcf122650 con 0x563bcf2293f0 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21e210 0x563bcf2262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf226d70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011ef0 tx=0x7f7afc011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8ea8a66b8b083 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7aed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7aed7fa640 1 -- 172.21.15.103:0/346692523 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc022050 con 0x563bcf2293f0 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7aed7fa640 1 -- 172.21.15.103:0/346692523 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7afc004dd0 con 0x563bcf2293f0 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc01b380 con 0x563bcf2293f0 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.864+0000 7f7aed7fa640 1 -- 172.21.15.103:0/346692523 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7afc037070 con 0x563bcf2293f0 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 10 client.16074 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563bceef53c0 con 0x563bcf2293f0 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7aed7fa640 1 -- 172.21.15.103:0/346692523 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc01b380 con 0x563bcf2293f0 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7aed7fa640 1 -- 172.21.15.103:0/346692523 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7afc02f030 con 0x563bcf2293f0 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 20 client.16074 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf238db0 0x563bcf23b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 4 client.16074 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:16.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bcf190ef0 con 0x563bcf238db0 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.865+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf238db0 0x563bcf23b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.866+0000 7f7aed7fa640 10 client.16074 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.866+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf238db0 0x563bcf23b1a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6afbd05fca3440 server_cookie=5f0c3623b93d7a80 in_seq=0 out_seq=0 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7aed7fa640 1 -- 172.21.15.103:0/346692523 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563bcf190ef0 con 0x563bcf238db0 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7aed7fa640 10 client.16074 handle_command_reply: tid=0 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 client.16074 shutdown 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 msgr2=0x563bcf226d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf226d70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011ef0 tx=0x7f7afc011f20 comp rx=0 tx=0).stop 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 shutdown_connections 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf238db0 0x563bcf23b1a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf226d70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21eb30 0x563bcf226830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/346692523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21e210 0x563bcf2262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7aed7fa640 0 client.16074 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 >> 172.21.15.103:0/346692523 conn(0x563bcf21a7a0 msgr2=0x563bcf219900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 shutdown_connections 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.867+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/346692523 wait complete. 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.868+0000 7f7b0e5ae740 20 client.16074 trim_cache size 0 max 16384 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 Processor -- start 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 -- start start 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf220cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf225bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf2409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041af430 con 0x563bcf2293f0 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b0fd0 con 0x563bcf21eb30 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041afb00 con 0x563bcf21e210 2024-08-19T01:38:16.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf2409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf2409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44356/0 (socket says 172.21.15.103:44356) 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 -- 172.21.15.103:0/22316734 learned_addr learned my addr 172.21.15.103:0/22316734 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf220cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0974b640 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf225bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 -- 172.21.15.103:0/22316734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 msgr2=0x563bcf220cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf220cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 -- 172.21.15.103:0/22316734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 msgr2=0x563bcf225bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf225bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.887+0000 7f7b0a74d640 1 -- 172.21.15.103:0/22316734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bcf21e020 con 0x563bcf2293f0 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.888+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf2409a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011010 tx=0x7f7afc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faa95c9b1f66eff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.888+0000 7f7aef7fe640 1 -- 172.21.15.103:0/22316734 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc041070 con 0x563bcf2293f0 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.888+0000 7f7aef7fe640 1 -- 172.21.15.103:0/22316734 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc0117c0 con 0x563bcf2293f0 2024-08-19T01:38:16.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.888+0000 7f7aef7fe640 1 -- 172.21.15.103:0/22316734 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc022070 con 0x563bcf2293f0 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/22316734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 msgr2=0x563bcf2409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf2409a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7afc011010 tx=0x7f7afc0108e0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/22316734 shutdown_connections 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf2293f0 0x563bcf2409a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf21eb30 0x563bcf225bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/22316734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf220cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/22316734 >> 172.21.15.103:0/22316734 conn(0x563bcf21a7a0 msgr2=0x563bcf228260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/22316734 shutdown_connections 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/22316734 wait complete. 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 -- start start 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.889+0000 7f7b0e5ae740 1 Processor -- start 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf223df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf21eb30 0x563bcf231390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0e5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf2293f0 0x563bcf2318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b08d0 con 0x563bcf21eb30 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b01d0 con 0x563bcf2293f0 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0e5ae740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ad410 con 0x563bcf21e210 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0e5ae740 10 client.0 _ll_register_callbacks cb 0x7f7afc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf21eb30 0x563bcf231390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf21eb30 0x563bcf231390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44360/0 (socket says 172.21.15.103:44360) 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 -- 172.21.15.103:0/3067374669 learned_addr learned my addr 172.21.15.103:0/3067374669 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0a74d640 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf2293f0 0x563bcf2318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf223df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 -- 172.21.15.103:0/3067374669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 msgr2=0x563bcf223df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf223df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 -- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf2293f0 msgr2=0x563bcf2318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:16.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf2293f0 0x563bcf2318d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 -- 172.21.15.103:0/3067374669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bcf222560 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7b0974b640 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf21eb30 0x563bcf231390 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ccaf0 tx=0x7f7af400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff9417dca918fe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7aed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.890+0000 7f7aed7fa640 1 -- 172.21.15.103:0/3067374669 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40b4060 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc0117c0 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7aed7fa640 1 -- 172.21.15.103:0/3067374669 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7af40b4200 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7aed7fa640 1 -- 172.21.15.103:0/3067374669 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7af4020040 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 10 client.16080 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563bcf2a2b40 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7aed7fa640 1 -- 172.21.15.103:0/3067374669 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc0117c0 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7aed7fa640 1 -- 172.21.15.103:0/3067374669 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7af4010460 con 0x563bcf21eb30 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:16.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 20 client.16080 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf2a3260 0x563bcf2a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 4 client.16080 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563bcf18b400 con 0x563bcf2a3260 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.891+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf2a3260 0x563bcf2a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7aed7fa640 10 client.16080 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b09f4c640 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf2a3260 0x563bcf2a5650 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306f17d6d9509e3 server_cookie=c2b6fa9beb255baa in_seq=0 out_seq=0 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7aed7fa640 1 -- 172.21.15.103:0/3067374669 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563bcf18b400 con 0x563bcf2a3260 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7aed7fa640 10 client.16080 handle_command_reply: tid=0 2024-08-19T01:38:16.893 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: "4156e333-a9d3-4872-9240-d3fdb26b0662": { 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "4156e333-a9d3-4872-9240-d3fdb26b0662", 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:38:16.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 client.16080 shutdown 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf21eb30 msgr2=0x563bcf231390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf21eb30 0x563bcf231390 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ccaf0 tx=0x7f7af400d8d0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 shutdown_connections 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563bcf2a3260 0x563bcf2a5650 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563bcf2293f0 0x563bcf2318d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bcf21eb30 0x563bcf231390 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 --2- 172.21.15.103:0/3067374669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563bcf21e210 0x563bcf223df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7aed7fa640 0 client.16080 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 >> 172.21.15.103:0/3067374669 conn(0x563bcf21a7a0 msgr2=0x563bcf226220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 shutdown_connections 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 1 -- 172.21.15.103:0/3067374669 wait complete. 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.892+0000 7f7b0e5ae740 20 client.16080 trim_cache size 0 max 16384 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7af00a2b80 msgr2=0x7f7af00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7af00a2b80 0x7f7af00a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7afc01f000 tx=0x7f7afc01c000 comp rx=0 tx=0).stop 2024-08-19T01:38:16.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b041022a0 msgr2=0x7f7b0419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b041022a0 0x7f7b0419ab10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7af400d950 tx=0x7f7af400de10 comp rx=0 tx=0).stop 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 shutdown_connections 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7af00a2b80 0x7f7af00a5040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b04102c20 0x7f7b0419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b041022a0 0x7f7b0419ab10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 --2- 172.21.15.103:0/423843649 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b04100b50 0x7f7b0419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 >> 172.21.15.103:0/423843649 conn(0x7f7b040fc500 msgr2=0x7f7b04103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 shutdown_connections 2024-08-19T01:38:16.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:15.893+0000 7f7b0c1b0640 1 -- 172.21.15.103:0/423843649 wait complete. 2024-08-19T01:38:18.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:17 smithi146 ceph-mon[22098]: pgmap v414: 97 pgs: 97 active+clean; 2.5 GiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-08-19T01:38:18.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:17 smithi103 ceph-mon[18436]: pgmap v414: 97 pgs: 97 active+clean; 2.5 GiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-08-19T01:38:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:17 smithi190 ceph-mon[22032]: pgmap v414: 97 pgs: 97 active+clean; 2.5 GiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-08-19T01:38:18.528 INFO:tasks.cephfs.filesystem:scrub status for tag:4156e333-a9d3-4872-9240-d3fdb26b0662 - {'path': '/', 'tag': '4156e333-a9d3-4872-9240-d3fdb26b0662', 'options': 'recursive,force'} 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 -- 172.21.15.103:0/271453284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830102320 msgr2=0x7f5830108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 --2- 172.21.15.103:0/271453284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830102320 0x7f5830108bb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5828009ca0 tx=0x7f582802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 -- 172.21.15.103:0/271453284 shutdown_connections 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 --2- 172.21.15.103:0/271453284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830102320 0x7f5830108bb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 --2- 172.21.15.103:0/271453284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5830101960 0x7f5830101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 --2- 172.21.15.103:0/271453284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5830100770 0x7f5830100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 -- 172.21.15.103:0/271453284 >> 172.21.15.103:0/271453284 conn(0x7f58300fbf10 msgr2=0x7f58300fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 -- 172.21.15.103:0/271453284 shutdown_connections 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 -- 172.21.15.103:0/271453284 wait complete. 2024-08-19T01:38:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 Processor -- start 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.959+0000 7f58381da640 1 -- start start 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f58381da640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5830100770 0x7f5830104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f58381da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830101960 0x7f5830102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f58381da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5830102320 0x7f5830103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f58381da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f583010c2d0 con 0x7f5830101960 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f58381da640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f583010c150 con 0x7f5830100770 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f58381da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f583010c450 con 0x7f5830102320 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830101960 0x7f5830102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830101960 0x7f5830102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44368/0 (socket says 172.21.15.103:44368) 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 -- 172.21.15.103:0/777635051 learned_addr learned my addr 172.21.15.103:0/777635051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5836777640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5830102320 0x7f5830103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835f76640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5830100770 0x7f5830104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 -- 172.21.15.103:0/777635051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5830102320 msgr2=0x7f5830103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5830102320 0x7f5830103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 -- 172.21.15.103:0/777635051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5830100770 msgr2=0x7f5830104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5830100770 0x7f5830104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 -- 172.21.15.103:0/777635051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5830103b10 con 0x7f5830101960 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835f76640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5830100770 0x7f5830104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5835775640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830101960 0x7f5830102d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f582000d950 tx=0x7f582000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5826ffd640 1 -- 172.21.15.103:0/777635051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820017070 con 0x7f5830101960 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f5826ffd640 1 -- 172.21.15.103:0/777635051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5820009c30 con 0x7f5830101960 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.960+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58301a1ae0 con 0x7f5830101960 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.961+0000 7f5826ffd640 1 -- 172.21.15.103:0/777635051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582000f620 con 0x7f5830101960 2024-08-19T01:38:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.961+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58301a2080 con 0x7f5830101960 2024-08-19T01:38:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.963+0000 7f5826ffd640 1 -- 172.21.15.103:0/777635051 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f58200109a0 con 0x7f5830101960 2024-08-19T01:38:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.963+0000 7f583b06e740 1 Processor -- start 2024-08-19T01:38:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f583b06e740 1 -- start start 2024-08-19T01:38:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f583b06e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa77ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f583b06e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77efc0 0x560eaa783a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f583b06e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa786510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f583b06e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560eaa6e19d0 con 0x560eaa77e680 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f583b06e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560eaa689820 con 0x560eaa784050 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f583b06e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560eaa6dd990 con 0x560eaa77efc0 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5836777640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa77ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5835f76640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77efc0 0x560eaa783a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5836777640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa77ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44378/0 (socket says 172.21.15.103:44378) 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5836777640 1 -- 172.21.15.103:0/833727706 learned_addr learned my addr 172.21.15.103:0/833727706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5835775640 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa786510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5826ffd640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f581c0a2cb0 0x7f581c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5835f76640 1 -- 172.21.15.103:0/833727706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 msgr2=0x560eaa786510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5835f76640 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa786510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5835f76640 1 -- 172.21.15.103:0/833727706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 msgr2=0x560eaa77ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5835f76640 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa77ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5835f76640 1 -- 172.21.15.103:0/833727706 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eaa3f5210 con 0x560eaa77efc0 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.964+0000 7f5836777640 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa77ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.965+0000 7f5826ffd640 1 -- 172.21.15.103:0/777635051 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f58200cebd0 con 0x7f5830101960 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.965+0000 7f5836777640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f581c0a2cb0 0x7f581c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.965+0000 7f5835f76640 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77efc0 0x560eaa783a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58301017c0 tx=0x7f582c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e3084bb8df4be9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.965+0000 7f5836777640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f581c0a2cb0 0x7f581c0a5170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5828004af0 tx=0x7f58280095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.965+0000 7f581bfff640 1 -- 172.21.15.103:0/833727706 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c015070 con 0x560eaa77efc0 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.965+0000 7f581bfff640 1 -- 172.21.15.103:0/833727706 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c01a910 con 0x560eaa77efc0 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.965+0000 7f581bfff640 1 -- 172.21.15.103:0/833727706 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c011860 con 0x560eaa77efc0 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 -- 172.21.15.103:0/833727706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77efc0 msgr2=0x560eaa783a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77efc0 0x560eaa783a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58301017c0 tx=0x7f582c00f730 comp rx=0 tx=0).stop 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 -- 172.21.15.103:0/833727706 shutdown_connections 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa786510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77efc0 0x560eaa783a80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 --2- 172.21.15.103:0/833727706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa77ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 -- 172.21.15.103:0/833727706 >> 172.21.15.103:0/833727706 conn(0x560eaa779cf0 msgr2=0x560eaa77a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 -- 172.21.15.103:0/833727706 shutdown_connections 2024-08-19T01:38:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 -- 172.21.15.103:0/833727706 wait complete. 2024-08-19T01:38:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.966+0000 7f583b06e740 1 -- start start 2024-08-19T01:38:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 1 Processor -- start 2024-08-19T01:38:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa788830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa788d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa784050 0x560eaa78fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560eaa787020 con 0x560eaa77e680 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560eaa69d520 con 0x560eaa77efc0 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560eaa7871a0 con 0x560eaa784050 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f5835f76640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa788830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f5835f76640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa788830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44386/0 (socket says 172.21.15.103:44386) 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f5835f76640 1 -- 172.21.15.103:0/2441751888 learned_addr learned my addr 172.21.15.103:0/2441751888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f583b06e740 10 client.0 _ll_register_callbacks cb 0x560eaa3f4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f5835775640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa788d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f5836777640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa784050 0x560eaa78fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.967+0000 7f5835f76640 1 -- 172.21.15.103:0/2441751888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa784050 msgr2=0x560eaa78fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f5835f76640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa784050 0x560eaa78fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f5835f76640 1 -- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 msgr2=0x560eaa788d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f5835f76640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa788d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f5835f76640 1 -- 172.21.15.103:0/2441751888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eaa4c1620 con 0x560eaa77e680 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f5835775640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa788d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f5835f76640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa788830 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f582c000c00 tx=0x7f582c00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a0f70e2cf3e6c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f58197fa640 1 -- 172.21.15.103:0/2441751888 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c015070 con 0x560eaa77e680 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f58197fa640 1 -- 172.21.15.103:0/2441751888 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f582c01a460 con 0x560eaa77e680 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582c01a910 con 0x560eaa77e680 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f58197fa640 1 -- 172.21.15.103:0/2441751888 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f582c01a660 con 0x560eaa77e680 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f583b06e740 10 client.16092 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eaa792c10 con 0x560eaa77e680 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f58197fa640 1 -- 172.21.15.103:0/2441751888 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c01a910 con 0x560eaa77e680 2024-08-19T01:38:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.968+0000 7f58197fa640 1 -- 172.21.15.103:0/2441751888 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f582c0054b0 con 0x560eaa77e680 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f583b06e740 10 client.16092 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f583b06e740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f583b06e740 20 client.16092 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f583b06e740 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa794020 0x560eaa796480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f583b06e740 4 client.16092 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eaa6ef100 con 0x560eaa794020 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f5835775640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa794020 0x560eaa796480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f5835775640 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa794020 0x560eaa796480 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45fabb4c08535659 server_cookie=e95a6c8939b13c19 in_seq=0 out_seq=0 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.969+0000 7f58197fa640 10 client.16092 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.970+0000 7f58197fa640 1 -- 172.21.15.103:0/2441751888 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560eaa6ef100 con 0x560eaa794020 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.970+0000 7f58197fa640 10 client.16092 handle_command_reply: tid=0 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.970+0000 7f583b06e740 1 client.16092 shutdown 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.970+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 msgr2=0x560eaa788830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.970+0000 7f583b06e740 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa788830 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f582c000c00 tx=0x7f582c00bf70 comp rx=0 tx=0).stop 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 shutdown_connections 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa794020 0x560eaa796480 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa784050 0x560eaa78fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa788d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f58197fa640 0 client.16092 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 --2- 172.21.15.103:0/2441751888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77e680 0x560eaa788830 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 >> 172.21.15.103:0/2441751888 conn(0x560eaa779cf0 msgr2=0x560eaa77b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 shutdown_connections 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 1 -- 172.21.15.103:0/2441751888 wait complete. 2024-08-19T01:38:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.971+0000 7f583b06e740 20 client.16092 trim_cache size 0 max 16384 2024-08-19T01:38:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.989+0000 7f583b06e740 1 Processor -- start 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.989+0000 7f583b06e740 1 -- start start 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f583b06e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa7885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f583b06e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77efc0 0x560eaa78fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f583b06e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa788b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f583b06e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58301aade0 con 0x560eaa77efc0 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f583b06e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58301ac980 con 0x560eaa784050 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f583b06e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f58301ab4b0 con 0x560eaa77e680 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77efc0 0x560eaa78fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77efc0 0x560eaa78fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44400/0 (socket says 172.21.15.103:44400) 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 -- 172.21.15.103:0/2212228988 learned_addr learned my addr 172.21.15.103:0/2212228988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5836777640 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa788b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835f76640 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa7885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 -- 172.21.15.103:0/2212228988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 msgr2=0x560eaa7885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa7885e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 -- 172.21.15.103:0/2212228988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 msgr2=0x560eaa788b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa788b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 -- 172.21.15.103:0/2212228988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eaa774f90 con 0x560eaa77efc0 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835f76640 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa7885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5836777640 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa788b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.990+0000 7f5835775640 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77efc0 0x560eaa78fd70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f58200cc700 tx=0x7f58200cc5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa34340d09e7524 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f581b7fe640 1 -- 172.21.15.103:0/2212228988 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820020040 con 0x560eaa77efc0 2024-08-19T01:38:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f581b7fe640 1 -- 172.21.15.103:0/2212228988 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582009bb60 con 0x560eaa77efc0 2024-08-19T01:38:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f581b7fe640 1 -- 172.21.15.103:0/2212228988 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200a40d0 con 0x560eaa77efc0 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f583b06e740 1 -- 172.21.15.103:0/2212228988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77efc0 msgr2=0x560eaa78fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f583b06e740 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77efc0 0x560eaa78fd70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f58200cc700 tx=0x7f58200cc5d0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f583b06e740 1 -- 172.21.15.103:0/2212228988 shutdown_connections 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f583b06e740 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa784050 0x560eaa788b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f583b06e740 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa77efc0 0x560eaa78fd70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.991+0000 7f583b06e740 1 --2- 172.21.15.103:0/2212228988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa7885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 -- 172.21.15.103:0/2212228988 >> 172.21.15.103:0/2212228988 conn(0x560eaa779cf0 msgr2=0x560eaa780160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 -- 172.21.15.103:0/2212228988 shutdown_connections 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 -- 172.21.15.103:0/2212228988 wait complete. 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 -- start start 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 Processor -- start 2024-08-19T01:38:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa78d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa78d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa784050 0x560eaa78e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58301ac280 con 0x560eaa784050 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f58301abb80 con 0x560eaa77efc0 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f58301a8dc0 con 0x560eaa77e680 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.992+0000 7f583b06e740 10 client.0 _ll_register_callbacks cb 0x7f5828004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa784050 0x560eaa78e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa784050 0x560eaa78e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44406/0 (socket says 172.21.15.103:44406) 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 -- 172.21.15.103:0/2994331763 learned_addr learned my addr 172.21.15.103:0/2994331763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5835775640 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa78d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5835f76640 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa78d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 -- 172.21.15.103:0/2994331763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 msgr2=0x560eaa78d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa78d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 -- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 msgr2=0x560eaa78d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa78d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 -- 172.21.15.103:0/2994331763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eaa78edb0 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f5836777640 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa784050 0x560eaa78e7a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5828009ca0 tx=0x7f5828042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70024009076cd8bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f58197fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f58197fa640 1 -- 172.21.15.103:0/2994331763 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828003e40 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f58197fa640 1 -- 172.21.15.103:0/2994331763 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5828030700 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582009bb60 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f58197fa640 1 -- 172.21.15.103:0/2994331763 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f582805b070 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f583b06e740 10 client.16098 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.993+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eaa7900a0 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f58197fa640 1 -- 172.21.15.103:0/2994331763 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582009bb60 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f58197fa640 1 -- 172.21.15.103:0/2994331763 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5828053030 con 0x560eaa784050 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f583b06e740 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f583b06e740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f583b06e740 20 client.16098 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f583b06e740 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa7ff4d0 0x560eaa8018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f583b06e740 4 client.16098 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560eaa6e9450 con 0x560eaa7ff4d0 2024-08-19T01:38:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.994+0000 7f5835f76640 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa7ff4d0 0x560eaa8018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.995+0000 7f58197fa640 10 client.16098 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:18.995+0000 7f5835f76640 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa7ff4d0 0x560eaa8018f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=565d3f282e475a65 server_cookie=c8c5ee94d25052db in_seq=0 out_seq=0 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f58197fa640 1 -- 172.21.15.103:0/2994331763 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560eaa6e9450 con 0x560eaa7ff4d0 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f58197fa640 10 client.16098 handle_command_reply: tid=0 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 client.16098 shutdown 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa784050 msgr2=0x560eaa78e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa784050 0x560eaa78e7a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5828009ca0 tx=0x7f5828042000 comp rx=0 tx=0).stop 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 shutdown_connections 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560eaa7ff4d0 0x560eaa8018f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560eaa784050 0x560eaa78e7a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560eaa77efc0 0x560eaa78d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f58197fa640 0 client.16098 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 --2- 172.21.15.103:0/2994331763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560eaa77e680 0x560eaa78d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 >> 172.21.15.103:0/2994331763 conn(0x560eaa779cf0 msgr2=0x560eaa788550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 shutdown_connections 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 1 -- 172.21.15.103:0/2994331763 wait complete. 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.002+0000 7f583b06e740 20 client.16098 trim_cache size 0 max 16384 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:38:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:38:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f581c0a2cb0 msgr2=0x7f581c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f581c0a2cb0 0x7f581c0a5170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5828004af0 tx=0x7f58280095c0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830101960 msgr2=0x7f5830102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830101960 0x7f5830102d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f582000d950 tx=0x7f582000de10 comp rx=0 tx=0).stop 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 shutdown_connections 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f581c0a2cb0 0x7f581c0a5170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5830102320 0x7f5830103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5830101960 0x7f5830102d40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 --2- 172.21.15.103:0/777635051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5830100770 0x7f5830104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 >> 172.21.15.103:0/777635051 conn(0x7f58300fbf10 msgr2=0x7f58300fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 shutdown_connections 2024-08-19T01:38:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:19.003+0000 7f58381da640 1 -- 172.21.15.103:0/777635051 wait complete. 2024-08-19T01:38:20.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:19 smithi146 ceph-mon[22098]: pgmap v415: 97 pgs: 97 active+clean; 2.6 GiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:38:20.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:19 smithi103 ceph-mon[18436]: pgmap v415: 97 pgs: 97 active+clean; 2.6 GiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:38:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:19 smithi190 ceph-mon[22032]: pgmap v415: 97 pgs: 97 active+clean; 2.6 GiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:38:22.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:21 smithi146 ceph-mon[22098]: pgmap v416: 97 pgs: 97 active+clean; 2.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 33 op/s 2024-08-19T01:38:22.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:21 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:38:22.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:21 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:38:22.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:21 smithi103 ceph-mon[18436]: pgmap v416: 97 pgs: 97 active+clean; 2.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 33 op/s 2024-08-19T01:38:22.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:21 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:38:22.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:21 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:38:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:21 smithi190 ceph-mon[22032]: pgmap v416: 97 pgs: 97 active+clean; 2.7 GiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 33 op/s 2024-08-19T01:38:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:21 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:38:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:21 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:38:23.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:22 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:38:23.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:22 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:38:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:22 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:38:24.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:23 smithi146 ceph-mon[22098]: pgmap v417: 97 pgs: 97 active+clean; 2.8 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 29 op/s 2024-08-19T01:38:24.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:23 smithi103 ceph-mon[18436]: pgmap v417: 97 pgs: 97 active+clean; 2.8 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 29 op/s 2024-08-19T01:38:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:23 smithi190 ceph-mon[22032]: pgmap v417: 97 pgs: 97 active+clean; 2.8 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 29 op/s 2024-08-19T01:38:26.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:25 smithi146 ceph-mon[22098]: pgmap v418: 97 pgs: 97 active+clean; 2.8 GiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:26.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:25 smithi103 ceph-mon[18436]: pgmap v418: 97 pgs: 97 active+clean; 2.8 GiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:25 smithi190 ceph-mon[22032]: pgmap v418: 97 pgs: 97 active+clean; 2.8 GiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:28.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:27 smithi146 ceph-mon[22098]: pgmap v419: 97 pgs: 97 active+clean; 2.8 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:28.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:27 smithi103 ceph-mon[18436]: pgmap v419: 97 pgs: 97 active+clean; 2.8 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:27 smithi190 ceph-mon[22032]: pgmap v419: 97 pgs: 97 active+clean; 2.8 GiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:30.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:29 smithi146 ceph-mon[22098]: pgmap v420: 97 pgs: 97 active+clean; 2.9 GiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:30.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:29 smithi103 ceph-mon[18436]: pgmap v420: 97 pgs: 97 active+clean; 2.9 GiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:29 smithi190 ceph-mon[22032]: pgmap v420: 97 pgs: 97 active+clean; 2.9 GiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/3672988751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38101890 msgr2=0x7f0a38101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/3672988751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38101890 0x7f0a38101c90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009ca0 tx=0x7f0a3002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/3672988751 shutdown_connections 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/3672988751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38103440 0x7f0a38109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/3672988751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a38102a80 0x7f0a38102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/3672988751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38101890 0x7f0a38101c90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/3672988751 >> 172.21.15.103:0/3672988751 conn(0x7f0a380fd010 msgr2=0x7f0a380ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/3672988751 shutdown_connections 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.098+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/3672988751 wait complete. 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 Processor -- start 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 -- start start 2024-08-19T01:38:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a38101890 0x7f0a38197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38102a80 0x7f0a38198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38103440 0x7f0a3819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3810d4a0 con 0x7f0a38102a80 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3810d320 con 0x7f0a38101890 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3cbfc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3810d620 con 0x7f0a38103440 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38102a80 0x7f0a38198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38102a80 0x7f0a38198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41382/0 (socket says 172.21.15.103:41382) 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 -- 172.21.15.103:0/658051545 learned_addr learned my addr 172.21.15.103:0/658051545 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a3659c640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a38101890 0x7f0a38197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38103440 0x7f0a3819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 -- 172.21.15.103:0/658051545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38103440 msgr2=0x7f0a3819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38103440 0x7f0a3819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 -- 172.21.15.103:0/658051545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a38101890 msgr2=0x7f0a38197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a38101890 0x7f0a38197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 -- 172.21.15.103:0/658051545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3819f750 con 0x7f0a38102a80 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.099+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38102a80 0x7f0a38198070 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000b4a0 tx=0x7f0a2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.100+0000 7f0a2f7fe640 1 -- 172.21.15.103:0/658051545 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20014070 con 0x7f0a38102a80 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.100+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3819fa40 con 0x7f0a38102a80 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.100+0000 7f0a2f7fe640 1 -- 172.21.15.103:0/658051545 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a20010ac0 con 0x7f0a38102a80 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.100+0000 7f0a2f7fe640 1 -- 172.21.15.103:0/658051545 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20017c30 con 0x7f0a38102a80 2024-08-19T01:38:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.100+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3819ff90 con 0x7f0a38102a80 2024-08-19T01:38:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 Processor -- start 2024-08-19T01:38:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a2f7fe640 1 -- 172.21.15.103:0/658051545 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0a20017070 con 0x7f0a38102a80 2024-08-19T01:38:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 -- start start 2024-08-19T01:38:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54ba510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5626d54159d0 con 0x5626d54b2fc0 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5626d53bd820 con 0x5626d54b2680 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5626d5411990 con 0x5626d54b8050 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a36d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54b7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41386/0 (socket says 172.21.15.103:41386) 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.103+0000 7f0a3659c640 1 -- 172.21.15.103:0/1184031273 learned_addr learned my addr 172.21.15.103:0/1184031273 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a2f7fe640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0a240a2c30 0x7f0a240a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a3659c640 1 -- 172.21.15.103:0/1184031273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 msgr2=0x5626d54ba510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54ba510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a3659c640 1 -- 172.21.15.103:0/1184031273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 msgr2=0x5626d54b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a3659c640 1 -- 172.21.15.103:0/1184031273 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d5129210 con 0x5626d54b2fc0 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54ba510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a2f7fe640 1 -- 172.21.15.103:0/658051545 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0a200ce640 con 0x7f0a38102a80 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0a240a2c30 0x7f0a240a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54b7a80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0a381028e0 tx=0x7f0a30031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71c5ef4b2ce3ef6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a2cff9640 1 -- 172.21.15.103:0/1184031273 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30031b40 con 0x5626d54b2fc0 2024-08-19T01:38:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a2cff9640 1 -- 172.21.15.103:0/1184031273 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a30036030 con 0x5626d54b2fc0 2024-08-19T01:38:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.104+0000 7f0a2cff9640 1 -- 172.21.15.103:0/1184031273 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004660 con 0x5626d54b2fc0 2024-08-19T01:38:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1184031273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 msgr2=0x5626d54b7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0a240a2c30 0x7f0a240a50f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009ca0 tx=0x7f0a28008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54b7a80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0a381028e0 tx=0x7f0a30031450 comp rx=0 tx=0).stop 2024-08-19T01:38:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1184031273 shutdown_connections 2024-08-19T01:38:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54ba510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54b7a80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1184031273 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1184031273 >> 172.21.15.103:0/1184031273 conn(0x5626d54adcf0 msgr2=0x5626d54ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1184031273 shutdown_connections 2024-08-19T01:38:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1184031273 wait complete. 2024-08-19T01:38:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.105+0000 7f0a3f9bc740 1 -- start start 2024-08-19T01:38:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 1 Processor -- start 2024-08-19T01:38:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2fc0 0x5626d54bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5626d54bae40 con 0x5626d54b2680 2024-08-19T01:38:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5626d53d1520 con 0x5626d54b2fc0 2024-08-19T01:38:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5626d54bafc0 con 0x5626d54b8050 2024-08-19T01:38:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3f9bc740 10 client.0 _ll_register_callbacks cb 0x5626d5128a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54bc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41396/0 (socket says 172.21.15.103:41396) 2024-08-19T01:38:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 -- 172.21.15.103:0/27809287 learned_addr learned my addr 172.21.15.103:0/27809287 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2fc0 0x5626d54bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 -- 172.21.15.103:0/27809287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 msgr2=0x5626d54c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 -- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2fc0 msgr2=0x5626d54bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2fc0 0x5626d54bcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 -- 172.21.15.103:0/27809287 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d51f5620 con 0x5626d54b2680 2024-08-19T01:38:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a3659c640 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54bc5c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0a3002ade0 tx=0x7f0a30031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3800d1169b3bb0bf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.106+0000 7f0a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a1dffb640 1 -- 172.21.15.103:0/27809287 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3004a040 con 0x5626d54b2680 2024-08-19T01:38:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a1dffb640 1 -- 172.21.15.103:0/27809287 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0a300045e0 con 0x5626d54b2680 2024-08-19T01:38:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a30036030 con 0x5626d54b2680 2024-08-19T01:38:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a1dffb640 1 -- 172.21.15.103:0/27809287 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a30004800 con 0x5626d54b2680 2024-08-19T01:38:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626d54bae40 con 0x5626d54b2680 2024-08-19T01:38:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a1dffb640 1 -- 172.21.15.103:0/27809287 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a30036030 con 0x5626d54b2680 2024-08-19T01:38:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a1dffb640 1 -- 172.21.15.103:0/27809287 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0a3004d070 con 0x5626d54b2680 2024-08-19T01:38:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 20 client.16110 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d54c7c80 0x5626d54ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 4 client.16110 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.107+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626d5423100 con 0x5626d54c7c80 2024-08-19T01:38:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.108+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d54c7c80 0x5626d54ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.108+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d54c7c80 0x5626d54ca0e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa30ee643c4bcb4 server_cookie=23856f2a0c7547e0 in_seq=0 out_seq=0 2024-08-19T01:38:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.108+0000 7f0a1dffb640 10 client.16110 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a1dffb640 1 -- 172.21.15.103:0/27809287 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626d5423100 con 0x5626d54c7c80 2024-08-19T01:38:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a1dffb640 10 client.16110 handle_command_reply: tid=0 2024-08-19T01:38:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 client.16110 shutdown 2024-08-19T01:38:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 msgr2=0x5626d54bc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54bc5c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0a3002ade0 tx=0x7f0a30031850 comp rx=0 tx=0).stop 2024-08-19T01:38:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 shutdown_connections 2024-08-19T01:38:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d54c7c80 0x5626d54ca0e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2fc0 0x5626d54bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/27809287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54bc5c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 >> 172.21.15.103:0/27809287 conn(0x5626d54adcf0 msgr2=0x5626d54af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a1dffb640 0 client.16110 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 shutdown_connections 2024-08-19T01:38:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/27809287 wait complete. 2024-08-19T01:38:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.109+0000 7f0a3f9bc740 20 client.16110 trim_cache size 0 max 16384 2024-08-19T01:38:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.126+0000 7f0a3f9bc740 1 Processor -- start 2024-08-19T01:38:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3f9bc740 1 -- start start 2024-08-19T01:38:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381a8b30 con 0x5626d54b2fc0 2024-08-19T01:38:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381aa6d0 con 0x5626d54b2680 2024-08-19T01:38:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381a9200 con 0x5626d54b8050 2024-08-19T01:38:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41404/0 (socket says 172.21.15.103:41404) 2024-08-19T01:38:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 -- 172.21.15.103:0/1471401345 learned_addr learned my addr 172.21.15.103:0/1471401345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 -- 172.21.15.103:0/1471401345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 msgr2=0x5626d54c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 -- 172.21.15.103:0/1471401345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 msgr2=0x5626d54bcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54bcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 -- 172.21.15.103:0/1471401345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d54b73d0 con 0x5626d54b2fc0 2024-08-19T01:38:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54c9c90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0a2009af10 tx=0x7f0a20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40efb95e7127f860 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.127+0000 7f0a1ffff640 1 -- 172.21.15.103:0/1471401345 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2001e040 con 0x5626d54b2fc0 2024-08-19T01:38:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a1ffff640 1 -- 172.21.15.103:0/1471401345 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a2009af80 con 0x5626d54b2fc0 2024-08-19T01:38:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a1ffff640 1 -- 172.21.15.103:0/1471401345 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20010c70 con 0x5626d54b2fc0 2024-08-19T01:38:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1471401345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 msgr2=0x5626d54c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54c9c90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0a2009af10 tx=0x7f0a20000bc0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1471401345 shutdown_connections 2024-08-19T01:38:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b8050 0x5626d54c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2fc0 0x5626d54c9c90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1471401345 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b2680 0x5626d54bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1471401345 >> 172.21.15.103:0/1471401345 conn(0x5626d54adcf0 msgr2=0x5626d54ac680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1471401345 shutdown_connections 2024-08-19T01:38:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.128+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1471401345 wait complete. 2024-08-19T01:38:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 -- start start 2024-08-19T01:38:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 Processor -- start 2024-08-19T01:38:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b2fc0 0x5626d54c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b8050 0x5626d54c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381a9fd0 con 0x5626d54b2680 2024-08-19T01:38:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381a98d0 con 0x5626d54b8050 2024-08-19T01:38:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0a381a6b10 con 0x5626d54b2fc0 2024-08-19T01:38:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3f9bc740 10 client.0 _ll_register_callbacks cb 0x7f0a30031e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41420/0 (socket says 172.21.15.103:41420) 2024-08-19T01:38:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 -- 172.21.15.103:0/1693195794 learned_addr learned my addr 172.21.15.103:0/1693195794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b2fc0 0x5626d54c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b8050 0x5626d54c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 -- 172.21.15.103:0/1693195794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b2fc0 msgr2=0x5626d54c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b2fc0 0x5626d54c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 -- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b8050 msgr2=0x5626d54c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b8050 0x5626d54c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a3659c640 1 -- 172.21.15.103:0/1693195794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d54c2b60 con 0x5626d54b2680 2024-08-19T01:38:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.129+0000 7f0a36d9d640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b8050 0x5626d54c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3659c640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54c1000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0a30031970 tx=0x7f0a300314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7818ad45580aef1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a1dffb640 1 -- 172.21.15.103:0/1693195794 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30004d40 con 0x5626d54b2680 2024-08-19T01:38:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a1dffb640 1 -- 172.21.15.103:0/1693195794 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0a3002f910 con 0x5626d54b2680 2024-08-19T01:38:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a2009af80 con 0x5626d54b2680 2024-08-19T01:38:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a1dffb640 1 -- 172.21.15.103:0/1693195794 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a3002f4c0 con 0x5626d54b2680 2024-08-19T01:38:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626d54c3ee0 con 0x5626d54b2680 2024-08-19T01:38:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a1dffb640 1 -- 172.21.15.103:0/1693195794 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a2009af80 con 0x5626d54b2680 2024-08-19T01:38:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a1dffb640 1 -- 172.21.15.103:0/1693195794 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0a3004d040 con 0x5626d54b2680 2024-08-19T01:38:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 20 client.16116 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d55331c0 0x5626d55355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 4 client.16116 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.130+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626d541d450 con 0x5626d55331c0 2024-08-19T01:38:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.131+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d55331c0 0x5626d55355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.131+0000 7f0a1dffb640 10 client.16116 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.131+0000 7f0a35d9b640 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d55331c0 0x5626d55355e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a538fbb5eb6e88b server_cookie=75602b8ca4173cc1 in_seq=0 out_seq=0 2024-08-19T01:38:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a1dffb640 1 -- 172.21.15.103:0/1693195794 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626d541d450 con 0x5626d55331c0 2024-08-19T01:38:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a1dffb640 10 client.16116 handle_command_reply: tid=0 2024-08-19T01:38:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 client.16116 shutdown 2024-08-19T01:38:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 msgr2=0x5626d54c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54c1000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0a30031970 tx=0x7f0a300314d0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 shutdown_connections 2024-08-19T01:38:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5626d55331c0 0x5626d55355e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5626d54b8050 0x5626d54c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626d54b2fc0 0x5626d54c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a1dffb640 0 client.16116 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.138+0000 7f0a3f9bc740 1 --2- 172.21.15.103:0/1693195794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5626d54b2680 0x5626d54c1000 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 >> 172.21.15.103:0/1693195794 conn(0x5626d54adcf0 msgr2=0x5626d54acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 shutdown_connections 2024-08-19T01:38:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3f9bc740 1 -- 172.21.15.103:0/1693195794 wait complete. 2024-08-19T01:38:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3f9bc740 20 client.16116 trim_cache size 0 max 16384 2024-08-19T01:38:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:38:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:38:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:38:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0a240a2c30 msgr2=0x7f0a240a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0a240a2c30 0x7f0a240a50f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009ca0 tx=0x7f0a28008040 comp rx=0 tx=0).stop 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38102a80 msgr2=0x7f0a38198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38102a80 0x7f0a38198070 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000b4a0 tx=0x7f0a2000b960 comp rx=0 tx=0).stop 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 shutdown_connections 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0a240a2c30 0x7f0a240a50f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a38103440 0x7f0a3819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a38102a80 0x7f0a38198070 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 --2- 172.21.15.103:0/658051545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0a38101890 0x7f0a38197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 >> 172.21.15.103:0/658051545 conn(0x7f0a380fd010 msgr2=0x7f0a38105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 shutdown_connections 2024-08-19T01:38:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:32.139+0000 7f0a3cbfc640 1 -- 172.21.15.103:0/658051545 wait complete. 2024-08-19T01:38:32.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:31 smithi146 ceph-mon[22098]: pgmap v421: 97 pgs: 97 active+clean; 3.0 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:38:32.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:31 smithi103 ceph-mon[18436]: pgmap v421: 97 pgs: 97 active+clean; 3.0 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:38:32.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:31 smithi190 ceph-mon[22032]: pgmap v421: 97 pgs: 97 active+clean; 3.0 GiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:38:34.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:33 smithi146 ceph-mon[22098]: pgmap v422: 97 pgs: 97 active+clean; 3.1 GiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 26 op/s 2024-08-19T01:38:34.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:33 smithi103 ceph-mon[18436]: pgmap v422: 97 pgs: 97 active+clean; 3.1 GiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 26 op/s 2024-08-19T01:38:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:33 smithi190 ceph-mon[22032]: pgmap v422: 97 pgs: 97 active+clean; 3.1 GiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 35 MiB/s wr, 26 op/s 2024-08-19T01:38:35.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:34 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:38:35.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:34 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:38:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:34 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:38:36.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:35 smithi146 ceph-mon[22098]: pgmap v423: 97 pgs: 97 active+clean; 3.1 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-08-19T01:38:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:38:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:38:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:38:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:35 smithi103 ceph-mon[18436]: pgmap v423: 97 pgs: 97 active+clean; 3.1 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-08-19T01:38:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:38:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:38:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:38:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:35 smithi190 ceph-mon[22032]: pgmap v423: 97 pgs: 97 active+clean; 3.1 GiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-08-19T01:38:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:38:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:38:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:38:37.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:36 smithi146 ceph-mon[22098]: pgmap v424: 97 pgs: 97 active+clean; 3.1 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:38:37.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:36 smithi103 ceph-mon[18436]: pgmap v424: 97 pgs: 97 active+clean; 3.1 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:38:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:36 smithi190 ceph-mon[22032]: pgmap v424: 97 pgs: 97 active+clean; 3.1 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:38:40.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:39 smithi146 ceph-mon[22098]: pgmap v425: 97 pgs: 97 active+clean; 3.2 GiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:38:40.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:39 smithi103 ceph-mon[18436]: pgmap v425: 97 pgs: 97 active+clean; 3.2 GiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:38:40.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:39 smithi190 ceph-mon[22032]: pgmap v425: 97 pgs: 97 active+clean; 3.2 GiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:38:42.182 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:41 smithi190 ceph-mon[22032]: pgmap v426: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:38:42.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:41 smithi146 ceph-mon[22098]: pgmap v426: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:38:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:41 smithi103 ceph-mon[18436]: pgmap v426: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:38:44.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:43 smithi146 ceph-mon[22098]: pgmap v427: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:38:44.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:43 smithi103 ceph-mon[18436]: pgmap v427: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:38:44.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:43 smithi190 ceph-mon[22032]: pgmap v427: 97 pgs: 97 active+clean; 3.3 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.239+0000 7f678f7b9640 1 -- 172.21.15.103:0/4227282988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 msgr2=0x7f6788101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.239+0000 7f678f7b9640 1 --2- 172.21.15.103:0/4227282988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788101de0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6778009900 tx=0x7f677802a410 comp rx=0 tx=0).stop 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 -- 172.21.15.103:0/4227282988 shutdown_connections 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 --2- 172.21.15.103:0/4227282988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6788102320 0x7f6788108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 --2- 172.21.15.103:0/4227282988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788101de0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 --2- 172.21.15.103:0/4227282988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6788100770 0x7f6788100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 -- 172.21.15.103:0/4227282988 >> 172.21.15.103:0/4227282988 conn(0x7f67880fbf10 msgr2=0x7f67880fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 -- 172.21.15.103:0/4227282988 shutdown_connections 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 -- 172.21.15.103:0/4227282988 wait complete. 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 Processor -- start 2024-08-19T01:38:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.240+0000 7f678f7b9640 1 -- start start 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678f7b9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6788100770 0x7f6788104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678f7b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678f7b9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6788102320 0x7f6788103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678f7b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f678810c280 con 0x7f6788101960 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678f7b9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f678810c100 con 0x7f6788102320 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678f7b9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f678810c400 con 0x7f6788100770 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38630/0 (socket says 172.21.15.103:38630) 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 -- 172.21.15.103:0/1912348832 learned_addr learned my addr 172.21.15.103:0/1912348832 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678d555640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6788100770 0x7f6788104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678dd56640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6788102320 0x7f6788103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 -- 172.21.15.103:0/1912348832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6788100770 msgr2=0x7f6788104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6788100770 0x7f6788104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 -- 172.21.15.103:0/1912348832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6788102320 msgr2=0x7f6788103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6788102320 0x7f6788103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 -- 172.21.15.103:0/1912348832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6788103b10 con 0x7f6788101960 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678dd56640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6788102320 0x7f6788103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678cd54640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788102d40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6778004770 tx=0x7f67780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f677e7fc640 1 -- 172.21.15.103:0/1912348832 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677803f030 con 0x7f6788101960 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f677e7fc640 1 -- 172.21.15.103:0/1912348832 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778003cb0 con 0x7f6788101960 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.241+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a1b10 con 0x7f6788101960 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.242+0000 7f677e7fc640 1 -- 172.21.15.103:0/1912348832 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778035070 con 0x7f6788101960 2024-08-19T01:38:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.242+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a2060 con 0x7f6788101960 2024-08-19T01:38:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.244+0000 7f6792650740 1 Processor -- start 2024-08-19T01:38:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.244+0000 7f6792650740 1 -- start start 2024-08-19T01:38:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.244+0000 7f677e7fc640 1 -- 172.21.15.103:0/1912348832 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6778042030 con 0x7f6788101960 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.244+0000 7f6792650740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5680 0x55de5fbe5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.244+0000 7f6792650740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5fc0 0x55de5fbeaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.244+0000 7f6792650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f6792650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de5fb489d0 con 0x55de5fbeb050 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f6792650740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55de5faf0820 con 0x55de5fbe5680 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f6792650740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55de5fb44990 con 0x55de5fbe5fc0 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbed510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38644/0 (socket says 172.21.15.103:38644) 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 -- 172.21.15.103:0/1351807516 learned_addr learned my addr 172.21.15.103:0/1351807516 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678dd56640 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5680 0x55de5fbe5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678d555640 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5fc0 0x55de5fbeaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 -- 172.21.15.103:0/1351807516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5fc0 msgr2=0x55de5fbeaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5fc0 0x55de5fbeaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 -- 172.21.15.103:0/1351807516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5680 msgr2=0x55de5fbe5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5680 0x55de5fbe5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 -- 172.21.15.103:0/1351807516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5f85c210 con 0x55de5fbeb050 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678dd56640 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5680 0x55de5fbe5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f677e7fc640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67740a2c30 0x7f67740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678dd56640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67740a2c30 0x7f67740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f677e7fc640 1 -- 172.21.15.103:0/1912348832 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f67780ef9a0 con 0x7f6788101960 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.245+0000 7f678cd54640 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbed510 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f67780f5ba0 tx=0x7f67780f5c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e26c297acee6cf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f67737fe640 1 -- 172.21.15.103:0/1351807516 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778049070 con 0x55de5fbeb050 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f67737fe640 1 -- 172.21.15.103:0/1351807516 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67780efe50 con 0x55de5fbeb050 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f67737fe640 1 -- 172.21.15.103:0/1351807516 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780bc000 con 0x55de5fbeb050 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 -- 172.21.15.103:0/1351807516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 msgr2=0x55de5fbed510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f678dd56640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67740a2c30 0x7f67740a50f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6780005fd0 tx=0x7f6780009530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbed510 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f67780f5ba0 tx=0x7f67780f5c90 comp rx=0 tx=0).stop 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 -- 172.21.15.103:0/1351807516 shutdown_connections 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbed510 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5fc0 0x55de5fbeaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 --2- 172.21.15.103:0/1351807516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5680 0x55de5fbe5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 -- 172.21.15.103:0/1351807516 >> 172.21.15.103:0/1351807516 conn(0x55de5fbe0cf0 msgr2=0x55de5fbe10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.246+0000 7f6792650740 1 -- 172.21.15.103:0/1351807516 shutdown_connections 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 -- 172.21.15.103:0/1351807516 wait complete. 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 -- start start 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 Processor -- start 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbefa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbf6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de5fbedde0 con 0x55de5fbe5680 2024-08-19T01:38:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55de5fb04520 con 0x55de5fbe5fc0 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55de5fbedf60 con 0x55de5fbeb050 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f6792650740 10 client.0 _ll_register_callbacks cb 0x55de5f85ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.247+0000 7f678d555640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbef4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38648/0 (socket says 172.21.15.103:38648) 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 -- 172.21.15.103:0/250813890 learned_addr learned my addr 172.21.15.103:0/250813890 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678cd54640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbefa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678dd56640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbf6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 -- 172.21.15.103:0/250813890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 msgr2=0x55de5fbf6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbf6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 -- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 msgr2=0x55de5fbefa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbefa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 -- 172.21.15.103:0/250813890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5f928620 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678dd56640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f678d555640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbef4d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f678400ba70 tx=0x7f678400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957bf25722d8f706 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f6770ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f6770ff9640 1 -- 172.21.15.103:0/250813890 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784003e80 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67780efe50 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f6770ff9640 1 -- 172.21.15.103:0/250813890 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6784011440 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f6770ff9640 1 -- 172.21.15.103:0/250813890 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6784011640 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f6792650740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.248+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de5fbf9960 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6770ff9640 1 -- 172.21.15.103:0/250813890 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67780efe50 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6770ff9640 1 -- 172.21.15.103:0/250813890 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f67840269d0 con 0x55de5fbe5680 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6792650740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6792650740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6792650740 20 client.16128 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6792650740 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fbfb000 0x55de5fbfd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6792650740 4 client.16128 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de5fb56100 con 0x55de5fbfb000 2024-08-19T01:38:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.249+0000 7f678cd54640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fbfb000 0x55de5fbfd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.250+0000 7f6770ff9640 10 client.16128 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.250+0000 7f678cd54640 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fbfb000 0x55de5fbfd4c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc942b6ef54b2eb server_cookie=f7b28ed9f8baa93a in_seq=0 out_seq=0 2024-08-19T01:38:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.250+0000 7f6770ff9640 1 -- 172.21.15.103:0/250813890 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de5fb56100 con 0x55de5fbfb000 2024-08-19T01:38:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.250+0000 7f6770ff9640 10 client.16128 handle_command_reply: tid=0 2024-08-19T01:38:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.250+0000 7f6792650740 1 client.16128 shutdown 2024-08-19T01:38:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.250+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 msgr2=0x55de5fbef4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbef4d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f678400ba70 tx=0x7f678400bf30 comp rx=0 tx=0).stop 2024-08-19T01:38:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 shutdown_connections 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fbfb000 0x55de5fbfd4c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6770ff9640 0 client.16128 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbefa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 --2- 172.21.15.103:0/250813890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbef4d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 >> 172.21.15.103:0/250813890 conn(0x55de5fbe0cf0 msgr2=0x55de5fbe2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 shutdown_connections 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 1 -- 172.21.15.103:0/250813890 wait complete. 2024-08-19T01:38:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.251+0000 7f6792650740 20 client.16128 trim_cache size 0 max 16384 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 Processor -- start 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 -- start start 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbe10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbe1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67881aada0 con 0x55de5fbe5680 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f67881ac940 con 0x55de5fbe5fc0 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.268+0000 7f6792650740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67881ab470 con 0x55de5fbeb050 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbe10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbe10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38652/0 (socket says 172.21.15.103:38652) 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 -- 172.21.15.103:0/3405834153 learned_addr learned my addr 172.21.15.103:0/3405834153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 -- 172.21.15.103:0/3405834153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 msgr2=0x55de5fbe1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678dd56640 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbe1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678cd54640 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbe1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 -- 172.21.15.103:0/3405834153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 msgr2=0x55de5fbf6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 -- 172.21.15.103:0/3405834153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5fbdf220 con 0x55de5fbe5680 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678dd56640 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbe1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678cd54640 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f678d555640 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbe10d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6784026e10 tx=0x7f67840045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1086075d5ce0a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f6772ffd640 1 -- 172.21.15.103:0/3405834153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784003e80 con 0x55de5fbe5680 2024-08-19T01:38:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f6772ffd640 1 -- 172.21.15.103:0/3405834153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6784011440 con 0x55de5fbe5680 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.269+0000 7f6772ffd640 1 -- 172.21.15.103:0/3405834153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784026ba0 con 0x55de5fbe5680 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 -- 172.21.15.103:0/3405834153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 msgr2=0x55de5fbe10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbe10d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6784026e10 tx=0x7f67840045b0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 -- 172.21.15.103:0/3405834153 shutdown_connections 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbeb050 0x55de5fbe1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 --2- 172.21.15.103:0/3405834153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbe5680 0x55de5fbe10d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 -- 172.21.15.103:0/3405834153 >> 172.21.15.103:0/3405834153 conn(0x55de5fbe0cf0 msgr2=0x55de5fbdfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 -- 172.21.15.103:0/3405834153 shutdown_connections 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.270+0000 7f6792650740 1 -- 172.21.15.103:0/3405834153 wait complete. 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 -- start start 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 Processor -- start 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5680 0x55de5fbf3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.276 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbf5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67881ac240 con 0x55de5fbeb050 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f67881abb40 con 0x55de5fbe5fc0 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a8d80 con 0x55de5fbe5680 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f6792650740 10 client.0 _ll_register_callbacks cb 0x7f67840042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f678dd56640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbf5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f678dd56640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbf5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38664/0 (socket says 172.21.15.103:38664) 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f678dd56640 1 -- 172.21.15.103:0/4117554349 learned_addr learned my addr 172.21.15.103:0/4117554349 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f678cd54640 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.271+0000 7f678d555640 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5680 0x55de5fbf3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f678dd56640 1 -- 172.21.15.103:0/4117554349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5680 msgr2=0x55de5fbf3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f678dd56640 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5680 0x55de5fbf3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f678dd56640 1 -- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 msgr2=0x55de5fbf4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:45.277 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f678dd56640 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f678dd56640 1 -- 172.21.15.103:0/4117554349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5fbf5b20 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f678dd56640 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbf5510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f67800098b0 tx=0x7f6780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7adbfec696f940cb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6770ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6770ff9640 1 -- 172.21.15.103:0/4117554349 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678001ed40 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6770ff9640 1 -- 172.21.15.103:0/4117554349 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f678001f420 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6784011440 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6770ff9640 1 -- 172.21.15.103:0/4117554349 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f678001f620 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6792650740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de5fbf6e70 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.272+0000 7f6770ff9640 1 -- 172.21.15.103:0/4117554349 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6784011440 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6770ff9640 1 -- 172.21.15.103:0/4117554349 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6780033770 con 0x55de5fbeb050 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6792650740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6792650740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6792650740 20 client.16134 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6792650740 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fc665e0 0x55de5fc68a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6792650740 4 client.16134 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de5fb50450 con 0x55de5fc665e0 2024-08-19T01:38:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f678d555640 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fc665e0 0x55de5fc68a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f678d555640 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fc665e0 0x55de5fc68a00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3961d2832500e2ac server_cookie=db472059c820d96e in_seq=0 out_seq=0 2024-08-19T01:38:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.273+0000 7f6770ff9640 10 client.16134 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6770ff9640 1 -- 172.21.15.103:0/4117554349 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de5fb50450 con 0x55de5fc665e0 2024-08-19T01:38:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6770ff9640 10 client.16134 handle_command_reply: tid=0 2024-08-19T01:38:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 client.16134 shutdown 2024-08-19T01:38:45.282 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 msgr2=0x55de5fbf5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbf5510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f67800098b0 tx=0x7f6780004a40 comp rx=0 tx=0).stop 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 shutdown_connections 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55de5fc665e0 0x55de5fc68a00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de5fbeb050 0x55de5fbf5510 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55de5fbe5fc0 0x55de5fbf4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6770ff9640 0 client.16134 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 --2- 172.21.15.103:0/4117554349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de5fbe5680 0x55de5fbf3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 >> 172.21.15.103:0/4117554349 conn(0x55de5fbe0cf0 msgr2=0x55de5fbeea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 shutdown_connections 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 1 -- 172.21.15.103:0/4117554349 wait complete. 2024-08-19T01:38:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.281+0000 7f6792650740 20 client.16134 trim_cache size 0 max 16384 2024-08-19T01:38:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:38:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:38:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:38:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:38:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67740a2c30 msgr2=0x7f67740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67740a2c30 0x7f67740a50f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6780005fd0 tx=0x7f6780009530 comp rx=0 tx=0).stop 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 msgr2=0x7f6788102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788102d40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6778004770 tx=0x7f67780042f0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 shutdown_connections 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67740a2c30 0x7f67740a50f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6788102320 0x7f6788103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6788101960 0x7f6788102d40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 --2- 172.21.15.103:0/1912348832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6788100770 0x7f6788104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 >> 172.21.15.103:0/1912348832 conn(0x7f67880fbf10 msgr2=0x7f6788069600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 shutdown_connections 2024-08-19T01:38:45.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:45.282+0000 7f678f7b9640 1 -- 172.21.15.103:0/1912348832 wait complete. 2024-08-19T01:38:46.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:45 smithi146 ceph-mon[22098]: pgmap v428: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:46.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:45 smithi103 ceph-mon[18436]: pgmap v428: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:45 smithi190 ceph-mon[22032]: pgmap v428: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:48.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:47 smithi146 ceph-mon[22098]: pgmap v429: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:48.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:47 smithi103 ceph-mon[18436]: pgmap v429: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:48.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:47 smithi190 ceph-mon[22032]: pgmap v429: 97 pgs: 97 active+clean; 3.4 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:38:48.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:38:48.807 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:38:50.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:49 smithi146 ceph-mon[22098]: pgmap v430: 97 pgs: 97 active+clean; 3.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-08-19T01:38:50.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:49 smithi103 ceph-mon[18436]: pgmap v430: 97 pgs: 97 active+clean; 3.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-08-19T01:38:50.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:49 smithi190 ceph-mon[22032]: pgmap v430: 97 pgs: 97 active+clean; 3.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-08-19T01:38:50.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 -- 172.21.15.103:0/781336109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101067c0 msgr2=0x7fc910106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 --2- 172.21.15.103:0/781336109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101067c0 0x7fc910106ba0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc900009930 tx=0x7fc90002a420 comp rx=0 tx=0).stop 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 -- 172.21.15.103:0/781336109 shutdown_connections 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 --2- 172.21.15.103:0/781336109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910101160 0x7fc91010d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 --2- 172.21.15.103:0/781336109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9101007c0 0x7fc910100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 --2- 172.21.15.103:0/781336109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101067c0 0x7fc910106ba0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 -- 172.21.15.103:0/781336109 >> 172.21.15.103:0/781336109 conn(0x7fc9100fc500 msgr2=0x7fc9100fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 -- 172.21.15.103:0/781336109 shutdown_connections 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.696+0000 7fc915506640 1 -- 172.21.15.103:0/781336109 wait complete. 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.697+0000 7fc915506640 1 Processor -- start 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.697+0000 7fc915506640 1 -- start start 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.697+0000 7fc915506640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101007c0 0x7fc910198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.697+0000 7fc915506640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910101160 0x7fc910199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.697+0000 7fc915506640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9101067c0 0x7fc9101a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.697+0000 7fc915506640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc910110a70 con 0x7fc9101007c0 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc915506640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101108f0 con 0x7fc9101067c0 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc915506640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc910110bf0 con 0x7fc910101160 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101007c0 0x7fc910198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101007c0 0x7fc910198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38680/0 (socket says 172.21.15.103:38680) 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 -- 172.21.15.103:0/154404205 learned_addr learned my addr 172.21.15.103:0/154404205 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910101160 0x7fc910199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9101067c0 0x7fc9101a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 -- 172.21.15.103:0/154404205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910101160 msgr2=0x7fc910199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910101160 0x7fc910199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 -- 172.21.15.103:0/154404205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9101067c0 msgr2=0x7fc9101a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9101067c0 0x7fc9101a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 -- 172.21.15.103:0/154404205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9101a44a0 con 0x7fc9101007c0 2024-08-19T01:38:50.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9101067c0 0x7fc9101a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc90effd640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101007c0 0x7fc910198ee0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc900005940 tx=0x7fc9000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc8f7fff640 1 -- 172.21.15.103:0/154404205 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90003f030 con 0x7fc9101007c0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9101a4670 con 0x7fc9101007c0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc8f7fff640 1 -- 172.21.15.103:0/154404205 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc900003cb0 con 0x7fc9101007c0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc8f7fff640 1 -- 172.21.15.103:0/154404205 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900035070 con 0x7fc9101007c0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.698+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9101a4ad0 con 0x7fc9101007c0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.701+0000 7fc8f7fff640 1 -- 172.21.15.103:0/154404205 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc900042030 con 0x7fc9101007c0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.701+0000 7fc917904740 1 Processor -- start 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.701+0000 7fc917904740 1 -- start start 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.701+0000 7fc8f7fff640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc8f80a2b80 0x7fc8f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc8f7fff640 1 -- 172.21.15.103:0/154404205 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9000ef560 con 0x7fc9101007c0 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc917904740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fe45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc8f80a2b80 0x7fc8f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc917904740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401feeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc917904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560401ff17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc917904740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560401f49980 con 0x560401fe4b30 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc917904740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560401ee1a70 con 0x560401fe4210 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc917904740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560401f45940 con 0x560401fef3f0 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401feeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401feeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38684/0 (socket says 172.21.15.103:38684) 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1134826693 learned_addr learned my addr 172.21.15.103:0/1134826693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560401ff17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1134826693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 msgr2=0x560401ff17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560401ff17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc8f80a2b80 0x7fc8f80a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc908008c60 tx=0x7fc90800d710 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:50.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1134826693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 msgr2=0x560401fe45f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fe45f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.702+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1134826693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560401f446e0 con 0x560401fe4b30 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.703+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401feeeb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc90400aca0 tx=0x7fc90400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eac442cc4b08630 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.703+0000 7fc8f57fa640 1 -- 172.21.15.103:0/1134826693 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90401d070 con 0x560401fe4b30 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.703+0000 7fc8f57fa640 1 -- 172.21.15.103:0/1134826693 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc904018af0 con 0x560401fe4b30 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.703+0000 7fc8f57fa640 1 -- 172.21.15.103:0/1134826693 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904010ba0 con 0x560401fe4b30 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 -- 172.21.15.103:0/1134826693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 msgr2=0x560401feeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401feeeb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc90400aca0 tx=0x7fc90400f790 comp rx=0 tx=0).stop 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 -- 172.21.15.103:0/1134826693 shutdown_connections 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560401ff17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401feeeb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 --2- 172.21.15.103:0/1134826693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fe45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 -- 172.21.15.103:0/1134826693 >> 172.21.15.103:0/1134826693 conn(0x560401fe07a0 msgr2=0x560401fe0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:50.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 -- 172.21.15.103:0/1134826693 shutdown_connections 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 -- 172.21.15.103:0/1134826693 wait complete. 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 -- start start 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.704+0000 7fc917904740 1 Processor -- start 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc917904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fe4210 0x560401fec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc917904740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401fec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc917904740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fef3f0 0x560401fece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc917904740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560401ff1fc0 con 0x560401fe4b30 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc917904740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560401f04aa0 con 0x560401fef3f0 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc917904740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560401ff2140 con 0x560401fe4210 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc917904740 10 client.0 _ll_register_callbacks cb 0x560401ef3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:50.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401fec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.765 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:38:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:38:50.765 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:38:50.765 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:38:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401fec8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38686/0 (socket says 172.21.15.103:38686) 2024-08-19T01:38:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 -- 172.21.15.103:0/3196697816 learned_addr learned my addr 172.21.15.103:0/3196697816 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:50.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90effd640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fe4210 0x560401fec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fef3f0 0x560401fece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 -- 172.21.15.103:0/3196697816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fe4210 msgr2=0x560401fec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fe4210 0x560401fec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 -- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fef3f0 msgr2=0x560401fece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fef3f0 0x560401fece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90e7fc640 1 -- 172.21.15.103:0/3196697816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560401ee8650 con 0x560401fe4b30 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.705+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fef3f0 0x560401fece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401fec8f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc90400a580 tx=0x7fc904010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7f2f9690212a55 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc8f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/3196697816 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904027020 con 0x560401fe4b30 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/3196697816 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9040108a0 con 0x560401fe4b30 2024-08-19T01:38:50.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc904018af0 con 0x560401fe4b30 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/3196697816 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc90401d040 con 0x560401fe4b30 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560401cbb3c0 con 0x560401fe4b30 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/3196697816 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc904018af0 con 0x560401fe4b30 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/3196697816 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc9040046c0 con 0x560401fe4b30 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 20 client.16146 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560401ffeee0 0x5604020012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 4 client.16146 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.706+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560401f56ef0 con 0x560401ffeee0 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.707+0000 7fc90effd640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560401ffeee0 0x5604020012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.707+0000 7fc90effd640 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560401ffeee0 0x5604020012d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1047427d2b9f817b server_cookie=d2a344946f4d38fa in_seq=0 out_seq=0 2024-08-19T01:38:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.707+0000 7fc8f2ff5640 10 client.16146 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/3196697816 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560401f56ef0 con 0x560401ffeee0 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc8f2ff5640 10 client.16146 handle_command_reply: tid=0 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 client.16146 shutdown 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 msgr2=0x560401fec8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401fec8f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc90400a580 tx=0x7fc904010660 comp rx=0 tx=0).stop 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 shutdown_connections 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x560401ffeee0 0x5604020012d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fef3f0 0x560401fece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401fec8f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 --2- 172.21.15.103:0/3196697816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fe4210 0x560401fec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 >> 172.21.15.103:0/3196697816 conn(0x560401fe07a0 msgr2=0x560401fe64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc8f2ff5640 0 client.16146 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 shutdown_connections 2024-08-19T01:38:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 1 -- 172.21.15.103:0/3196697816 wait complete. 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.708+0000 7fc917904740 20 client.16146 trim_cache size 0 max 16384 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 Processor -- start 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 -- start start 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560402006900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560402006e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101ad090 con 0x560401fe4b30 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101aec30 con 0x560401fe4210 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.730+0000 7fc917904740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101ad760 con 0x560401fef3f0 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560402006900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560402006900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38696/0 (socket says 172.21.15.103:38696) 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1900005943 learned_addr learned my addr 172.21.15.103:0/1900005943 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560402006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90effd640 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1900005943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 msgr2=0x560402006e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560402006e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1900005943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 msgr2=0x560401fecc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fecc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 -- 172.21.15.103:0/1900005943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560401ff20b0 con 0x560401fe4b30 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90effd640 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fecc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560402006900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc904004c80 tx=0x7fc904001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fb8b801d1b9be2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.731+0000 7fc8f4ff9640 1 -- 172.21.15.103:0/1900005943 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904027070 con 0x560401fe4b30 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc8f4ff9640 1 -- 172.21.15.103:0/1900005943 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc904004660 con 0x560401fe4b30 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc8f4ff9640 1 -- 172.21.15.103:0/1900005943 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904010af0 con 0x560401fe4b30 2024-08-19T01:38:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 -- 172.21.15.103:0/1900005943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 msgr2=0x560402006900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560402006900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc904004c80 tx=0x7fc904001e50 comp rx=0 tx=0).stop 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 -- 172.21.15.103:0/1900005943 shutdown_connections 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x560402006e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560402006900 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 --2- 172.21.15.103:0/1900005943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401fecc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 -- 172.21.15.103:0/1900005943 >> 172.21.15.103:0/1900005943 conn(0x560401fe07a0 msgr2=0x560401feb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 -- 172.21.15.103:0/1900005943 shutdown_connections 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.732+0000 7fc917904740 1 -- 172.21.15.103:0/1900005943 wait complete. 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 -- start start 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 Processor -- start 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401ff2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401ff2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x5604020678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101ae530 con 0x560401fe4b30 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101ade30 con 0x560401fe4210 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101ab070 con 0x560401fef3f0 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc917904740 10 client.0 _ll_register_callbacks cb 0x7fc90401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401ff2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401ff2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38704/0 (socket says 172.21.15.103:38704) 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc90e7fc640 1 -- 172.21.15.103:0/952958078 learned_addr learned my addr 172.21.15.103:0/952958078 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc90effd640 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401ff2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.733+0000 7fc90f7fe640 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x5604020678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc90e7fc640 1 -- 172.21.15.103:0/952958078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 msgr2=0x5604020678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x5604020678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc90e7fc640 1 -- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 msgr2=0x560401ff2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401ff2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc90e7fc640 1 -- 172.21.15.103:0/952958078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560402067e00 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc90e7fc640 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401ff2990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc904005790 tx=0x7fc904010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c87f058ef4d0b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc8f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/952958078 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904041070 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/952958078 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9040189e0 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc904004660 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/952958078 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc90404d050 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc917904740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.734+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560402068f00 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/952958078 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc904004660 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/952958078 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc904035070 con 0x560401fe4b30 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc917904740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc917904740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc917904740 20 client.16152 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc917904740 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5604020695a0 0x56040206b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc917904740 4 client.16152 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560401f51400 con 0x5604020695a0 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc90effd640 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5604020695a0 0x56040206b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc90effd640 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5604020695a0 0x56040206b9c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cc46bdb7416aec server_cookie=775f7d6adec9c49f in_seq=0 out_seq=0 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.735+0000 7fc8f2ff5640 10 client.16152 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc8f2ff5640 1 -- 172.21.15.103:0/952958078 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560401f51400 con 0x5604020695a0 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc8f2ff5640 10 client.16152 handle_command_reply: tid=0 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 client.16152 shutdown 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 msgr2=0x560401ff2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401ff2990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc904005790 tx=0x7fc904010ef0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 shutdown_connections 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5604020695a0 0x56040206b9c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560401fef3f0 0x5604020678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560401fe4b30 0x560401ff2990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 --2- 172.21.15.103:0/952958078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560401fe4210 0x560401ff2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 >> 172.21.15.103:0/952958078 conn(0x560401fe07a0 msgr2=0x560401fecc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc8f2ff5640 0 client.16152 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 shutdown_connections 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 1 -- 172.21.15.103:0/952958078 wait complete. 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.736+0000 7fc917904740 20 client.16152 trim_cache size 0 max 16384 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc8f80a2b80 msgr2=0x7fc8f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc8f80a2b80 0x7fc8f80a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc908008c60 tx=0x7fc90800d710 comp rx=0 tx=0).stop 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101007c0 msgr2=0x7fc910198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101007c0 0x7fc910198ee0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc900005940 tx=0x7fc9000042f0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 shutdown_connections 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc8f80a2b80 0x7fc8f80a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9101067c0 0x7fc9101a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910101160 0x7fc910199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 --2- 172.21.15.103:0/154404205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101007c0 0x7fc910198ee0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 >> 172.21.15.103:0/154404205 conn(0x7fc9100fc500 msgr2=0x7fc9100fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 shutdown_connections 2024-08-19T01:38:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:38:49.737+0000 7fc915506640 1 -- 172.21.15.103:0/154404205 wait complete. 2024-08-19T01:38:52.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:51 smithi146 ceph-mon[22098]: pgmap v431: 97 pgs: 97 active+clean; 3.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:38:52.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:51 smithi103 ceph-mon[18436]: pgmap v431: 97 pgs: 97 active+clean; 3.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:38:52.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:51 smithi190 ceph-mon[22032]: pgmap v431: 97 pgs: 97 active+clean; 3.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:38:54.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:53 smithi146 ceph-mon[22098]: pgmap v432: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:54.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:53 smithi103 ceph-mon[18436]: pgmap v432: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:54.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:53 smithi190 ceph-mon[22032]: pgmap v432: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:38:56.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:55 smithi146 ceph-mon[22098]: pgmap v433: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:56.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:55 smithi103 ceph-mon[18436]: pgmap v433: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:56.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:55 smithi190 ceph-mon[22032]: pgmap v433: 97 pgs: 97 active+clean; 3.6 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:38:58.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:57 smithi146 ceph-mon[22098]: pgmap v434: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:38:58.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:57 smithi103 ceph-mon[18436]: pgmap v434: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:38:58.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:57 smithi190 ceph-mon[22032]: pgmap v434: 97 pgs: 97 active+clean; 3.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:38:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 -- 172.21.15.103:0/649054911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28101960 msgr2=0x7fba28101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/649054911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28101960 0x7fba28101de0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c02a410 comp rx=0 tx=0).stop 2024-08-19T01:38:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 -- 172.21.15.103:0/649054911 shutdown_connections 2024-08-19T01:38:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/649054911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba28102320 0x7fba28108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/649054911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28101960 0x7fba28101de0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/649054911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba28100770 0x7fba28100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 -- 172.21.15.103:0/649054911 >> 172.21.15.103:0/649054911 conn(0x7fba280fbf10 msgr2=0x7fba280fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 -- 172.21.15.103:0/649054911 shutdown_connections 2024-08-19T01:38:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.380+0000 7fba2cb3d640 1 -- 172.21.15.103:0/649054911 wait complete. 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 Processor -- start 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 -- start start 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba28100770 0x7fba28104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba28101960 0x7fba28102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28102320 0x7fba28103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810c350 con 0x7fba28102320 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810c1d0 con 0x7fba28101960 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2cb3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba2810c4d0 con 0x7fba28100770 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28102320 0x7fba28103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28102320 0x7fba28103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40880/0 (socket says 172.21.15.103:40880) 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba26d9d640 1 -- 172.21.15.103:0/699122819 learned_addr learned my addr 172.21.15.103:0/699122819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba25d9b640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba28101960 0x7fba28102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba2659c640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba28100770 0x7fba28104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.381+0000 7fba26d9d640 1 -- 172.21.15.103:0/699122819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba28100770 msgr2=0x7fba28104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba26d9d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba28100770 0x7fba28104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba26d9d640 1 -- 172.21.15.103:0/699122819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba28101960 msgr2=0x7fba28102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba26d9d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba28101960 0x7fba28102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba26d9d640 1 -- 172.21.15.103:0/699122819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba28103b10 con 0x7fba28102320 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba26d9d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28102320 0x7fba28103280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fba2000b970 tx=0x7fba2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba177fe640 1 -- 172.21.15.103:0/699122819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20019070 con 0x7fba28102320 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba177fe640 1 -- 172.21.15.103:0/699122819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba20014440 con 0x7fba28102320 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba281a1b90 con 0x7fba28102320 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba177fe640 1 -- 172.21.15.103:0/699122819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba200075d0 con 0x7fba28102320 2024-08-19T01:38:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.382+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba281a2060 con 0x7fba28102320 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.384+0000 7fba177fe640 1 -- 172.21.15.103:0/699122819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fba2001c070 con 0x7fba28102320 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 Processor -- start 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 -- start start 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba177fe640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba100a2c30 0x7fba100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9560a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f9565a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9566050 0x5611f9568510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5611f94c39d0 con 0x5611f9560fc0 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5611f946b820 con 0x5611f9566050 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5611f94bf990 con 0x5611f9560680 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba25d9b640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba100a2c30 0x7fba100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.385+0000 7fba25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f9565a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba177fe640 1 -- 172.21.15.103:0/699122819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fba200ce640 con 0x7fba28102320 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f9565a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40886/0 (socket says 172.21.15.103:40886) 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 -- 172.21.15.103:0/3919288011 learned_addr learned my addr 172.21.15.103:0/3919288011 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba26d9d640 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9566050 0x5611f9568510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba2659c640 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9560a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 -- 172.21.15.103:0/3919288011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 msgr2=0x5611f9560a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9560a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 -- 172.21.15.103:0/3919288011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9566050 msgr2=0x5611f9568510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9566050 0x5611f9568510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 -- 172.21.15.103:0/3919288011 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f91d7210 con 0x5611f9560fc0 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba2659c640 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9560a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba26d9d640 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9566050 0x5611f9568510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f9565a80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed21560c9ac287d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba25d9b640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba100a2c30 0x7fba100a50f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ec0 tx=0x7fba1c03a000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba14ff9640 1 -- 172.21.15.103:0/3919288011 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c04d070 con 0x5611f9560fc0 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba14ff9640 1 -- 172.21.15.103:0/3919288011 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba1c004450 con 0x5611f9560fc0 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.386+0000 7fba14ff9640 1 -- 172.21.15.103:0/3919288011 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c035040 con 0x5611f9560fc0 2024-08-19T01:38:58.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3919288011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 msgr2=0x5611f9565a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f9565a80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c003ab0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3919288011 shutdown_connections 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9566050 0x5611f9568510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f9565a80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3919288011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9560a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3919288011 >> 172.21.15.103:0/3919288011 conn(0x5611f955bcf0 msgr2=0x5611f955c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3919288011 shutdown_connections 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.387+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3919288011 wait complete. 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 -- start start 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 Processor -- start 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560680 0x5611f956a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560fc0 0x5611f956ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9566050 0x5611f9571de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5611f9568fc0 con 0x5611f9566050 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5611f947f520 con 0x5611f9560680 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5611f9569140 con 0x5611f9560fc0 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2f9d1740 10 client.0 _ll_register_callbacks cb 0x5611f91d6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9566050 0x5611f9571de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9566050 0x5611f9571de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40900/0 (socket says 172.21.15.103:40900) 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 -- 172.21.15.103:0/1751434602 learned_addr learned my addr 172.21.15.103:0/1751434602 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba25d9b640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560fc0 0x5611f956ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2659c640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560680 0x5611f956a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 -- 172.21.15.103:0/1751434602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560fc0 msgr2=0x5611f956ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560fc0 0x5611f956ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 -- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560680 msgr2=0x5611f956a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560680 0x5611f956a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba26d9d640 1 -- 172.21.15.103:0/1751434602 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f92a3620 con 0x5611f9566050 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.388+0000 7fba2659c640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560680 0x5611f956a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba26d9d640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9566050 0x5611f9571de0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fba2000b940 tx=0x7fba20049790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460391751078393c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba0dffb640 1 -- 172.21.15.103:0/1751434602 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2009f280 con 0x5611f9566050 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba0dffb640 1 -- 172.21.15.103:0/1751434602 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fba20049b50 con 0x5611f9566050 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1c004450 con 0x5611f9566050 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba0dffb640 1 -- 172.21.15.103:0/1751434602 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba200a3b40 con 0x5611f9566050 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5611f9574ca0 con 0x5611f9566050 2024-08-19T01:38:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba0dffb640 1 -- 172.21.15.103:0/1751434602 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba1c004450 con 0x5611f9566050 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba0dffb640 1 -- 172.21.15.103:0/1751434602 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fba2009a460 con 0x5611f9566050 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 20 client.16164 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9576350 0x5611f9578760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 4 client.16164 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.389+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611f94d1100 con 0x5611f9576350 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.390+0000 7fba2659c640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9576350 0x5611f9578760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.390+0000 7fba2659c640 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9576350 0x5611f9578760 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ad4bd185e8102 server_cookie=2bb04d9ad8ad6feb in_seq=0 out_seq=0 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.390+0000 7fba0dffb640 10 client.16164 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba0dffb640 1 -- 172.21.15.103:0/1751434602 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611f94d1100 con 0x5611f9576350 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba0dffb640 10 client.16164 handle_command_reply: tid=0 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 client.16164 shutdown 2024-08-19T01:38:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9566050 msgr2=0x5611f9571de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9566050 0x5611f9571de0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fba2000b940 tx=0x7fba20049790 comp rx=0 tx=0).stop 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 shutdown_connections 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9576350 0x5611f9578760 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9566050 0x5611f9571de0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560fc0 0x5611f956ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba0dffb640 0 client.16164 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/1751434602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560680 0x5611f956a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 >> 172.21.15.103:0/1751434602 conn(0x5611f955bcf0 msgr2=0x5611f955d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 shutdown_connections 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 1 -- 172.21.15.103:0/1751434602 wait complete. 2024-08-19T01:38:58.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.391+0000 7fba2f9d1740 20 client.16164 trim_cache size 0 max 16384 2024-08-19T01:38:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 Processor -- start 2024-08-19T01:38:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 -- start start 2024-08-19T01:38:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f955bcf0 0x5611f955cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9571da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f955d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba281aad10 con 0x5611f9560fc0 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fba281ac8b0 con 0x5611f955bcf0 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.405+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba281ab3e0 con 0x5611f9560680 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f955d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f955d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40914/0 (socket says 172.21.15.103:40914) 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 -- 172.21.15.103:0/3083960991 learned_addr learned my addr 172.21.15.103:0/3083960991 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba2659c640 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f955bcf0 0x5611f955cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba25d9b640 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9571da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 -- 172.21.15.103:0/3083960991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 msgr2=0x5611f9571da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9571da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 -- 172.21.15.103:0/3083960991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f955bcf0 msgr2=0x5611f955cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f955bcf0 0x5611f955cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.409 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 -- 172.21.15.103:0/3083960991 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f9578de0 con 0x5611f9560fc0 2024-08-19T01:38:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba2659c640 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f955bcf0 0x5611f955cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:38:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba26d9d640 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f955d0e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fba20097950 tx=0x7fba20099490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a230a0edc7e872 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba0ffff640 1 -- 172.21.15.103:0/3083960991 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2004e030 con 0x5611f9560fc0 2024-08-19T01:38:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba0ffff640 1 -- 172.21.15.103:0/3083960991 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba200b6070 con 0x5611f9560fc0 2024-08-19T01:38:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.406+0000 7fba0ffff640 1 -- 172.21.15.103:0/3083960991 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20009040 con 0x5611f9560fc0 2024-08-19T01:38:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3083960991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 msgr2=0x5611f955d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f955d0e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fba20097950 tx=0x7fba20099490 comp rx=0 tx=0).stop 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3083960991 shutdown_connections 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f9560fc0 0x5611f955d0e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f9571da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/3083960991 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f955bcf0 0x5611f955cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3083960991 >> 172.21.15.103:0/3083960991 conn(0x5611f9565be0 msgr2=0x5611f956a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3083960991 shutdown_connections 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 -- 172.21.15.103:0/3083960991 wait complete. 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 -- start start 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.407+0000 7fba2f9d1740 1 Processor -- start 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f955bcf0 0x5611f956f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f956fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560fc0 0x5611f9570c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba281ac1b0 con 0x5611f955bcf0 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fba281abab0 con 0x5611f9560fc0 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fba281a8cf0 con 0x5611f9560680 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 10 client.0 _ll_register_callbacks cb 0x7fba2009aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f955bcf0 0x5611f956f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f955bcf0 0x5611f956f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40916/0 (socket says 172.21.15.103:40916) 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 -- 172.21.15.103:0/4192385679 learned_addr learned my addr 172.21.15.103:0/4192385679 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba26d9d640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560fc0 0x5611f9570c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba25d9b640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f956fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 -- 172.21.15.103:0/4192385679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 msgr2=0x5611f956fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f956fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 -- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560fc0 msgr2=0x5611f9570c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560fc0 0x5611f9570c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 -- 172.21.15.103:0/4192385679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f9571230 con 0x5611f955bcf0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba25d9b640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f956fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2659c640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f955bcf0 0x5611f956f770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fba18005db0 tx=0x7fba18013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aa8657340d12bc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba0dffb640 1 -- 172.21.15.103:0/4192385679 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18004220 con 0x5611f955bcf0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba200b6070 con 0x5611f955bcf0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.408+0000 7fba0dffb640 1 -- 172.21.15.103:0/4192385679 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fba180043c0 con 0x5611f955bcf0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba0dffb640 1 -- 172.21.15.103:0/4192385679 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba180035f0 con 0x5611f955bcf0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5611f95e1370 con 0x5611f955bcf0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba0dffb640 1 -- 172.21.15.103:0/4192385679 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba200b6070 con 0x5611f955bcf0 2024-08-19T01:38:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba0dffb640 1 -- 172.21.15.103:0/4192385679 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fba18021070 con 0x5611f955bcf0 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 20 client.16170 populate_metadata read hostname 'smithi103' 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9566050 0x5611f95e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 4 client.16170 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611f94cb450 con 0x5611f9566050 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.409+0000 7fba25d9b640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9566050 0x5611f95e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.410+0000 7fba25d9b640 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9566050 0x5611f95e3dc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1feb309693d9b30b server_cookie=d5e82ce1241787f1 in_seq=0 out_seq=0 2024-08-19T01:38:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.410+0000 7fba0dffb640 10 client.16170 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba0dffb640 1 -- 172.21.15.103:0/4192385679 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611f94cb450 con 0x5611f9566050 2024-08-19T01:38:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba0dffb640 10 client.16170 handle_command_reply: tid=0 2024-08-19T01:38:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 client.16170 shutdown 2024-08-19T01:38:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f955bcf0 msgr2=0x5611f956f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f955bcf0 0x5611f956f770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fba18005db0 tx=0x7fba18013830 comp rx=0 tx=0).stop 2024-08-19T01:38:58.419 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 shutdown_connections 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611f9566050 0x5611f95e3dc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611f9560fc0 0x5611f9570c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611f9560680 0x5611f956fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba0dffb640 0 client.16170 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 --2- 172.21.15.103:0/4192385679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611f955bcf0 0x5611f956f770 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 >> 172.21.15.103:0/4192385679 conn(0x5611f9565be0 msgr2=0x5611f955a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 shutdown_connections 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 1 -- 172.21.15.103:0/4192385679 wait complete. 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.417+0000 7fba2f9d1740 20 client.16170 trim_cache size 0 max 16384 2024-08-19T01:38:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:38:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:38:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:38:58.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba100a2c30 msgr2=0x7fba100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba100a2c30 0x7fba100a50f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ec0 tx=0x7fba1c03a000 comp rx=0 tx=0).stop 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28102320 msgr2=0x7fba28103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28102320 0x7fba28103280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fba2000b970 tx=0x7fba2000be30 comp rx=0 tx=0).stop 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 shutdown_connections 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fba100a2c30 0x7fba100a50f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba28102320 0x7fba28103280 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba28101960 0x7fba28102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 --2- 172.21.15.103:0/699122819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba28100770 0x7fba28104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 >> 172.21.15.103:0/699122819 conn(0x7fba280fbf10 msgr2=0x7fba280fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 shutdown_connections 2024-08-19T01:38:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:38:58.418+0000 7fba2cb3d640 1 -- 172.21.15.103:0/699122819 wait complete. 2024-08-19T01:38:58.639 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:38:58.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:38:58.934 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:00.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:38:59 smithi146 ceph-mon[22098]: pgmap v435: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:39:00.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:38:59 smithi103 ceph-mon[18436]: pgmap v435: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:39:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:38:59 smithi190 ceph-mon[22032]: pgmap v435: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:39:00.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 -- 172.21.15.103:0/1821387825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740100b50 msgr2=0x7ff740100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:00.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 --2- 172.21.15.103:0/1821387825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740100b50 0x7ff740100f30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff730009900 tx=0x7ff73002a480 comp rx=0 tx=0).stop 2024-08-19T01:39:00.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 -- 172.21.15.103:0/1821387825 shutdown_connections 2024-08-19T01:39:00.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 --2- 172.21.15.103:0/1821387825 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff740102c20 0x7ff74010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 --2- 172.21.15.103:0/1821387825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7401022a0 0x7ff7401026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 --2- 172.21.15.103:0/1821387825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740100b50 0x7ff740100f30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 -- 172.21.15.103:0/1821387825 >> 172.21.15.103:0/1821387825 conn(0x7ff7400fc500 msgr2=0x7ff7400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 -- 172.21.15.103:0/1821387825 shutdown_connections 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.590+0000 7ff7461a7640 1 -- 172.21.15.103:0/1821387825 wait complete. 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 Processor -- start 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 -- start start 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff740100b50 0x7ff740103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7401022a0 0x7ff7401039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740102c20 0x7ff740103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff740110b60 con 0x7ff740102c20 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff7401109e0 con 0x7ff740100b50 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff7461a7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff740110ce0 con 0x7ff7401022a0 2024-08-19T01:39:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff73ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740102c20 0x7ff740103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff73ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740102c20 0x7ff740103f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40936/0 (socket says 172.21.15.103:40936) 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff73ffff640 1 -- 172.21.15.103:0/3684813617 learned_addr learned my addr 172.21.15.103:0/3684813617 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.591+0000 7ff73f7fe640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff740100b50 0x7ff740103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff737fff640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7401022a0 0x7ff7401039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73ffff640 1 -- 172.21.15.103:0/3684813617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7401022a0 msgr2=0x7ff7401039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73ffff640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7401022a0 0x7ff7401039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73ffff640 1 -- 172.21.15.103:0/3684813617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff740100b50 msgr2=0x7ff740103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73ffff640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff740100b50 0x7ff740103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73ffff640 1 -- 172.21.15.103:0/3684813617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7401a6610 con 0x7ff740102c20 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff737fff640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7401022a0 0x7ff7401039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73ffff640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740102c20 0x7ff740103f00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff73800cce0 tx=0x7ff73800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73d7fa640 1 -- 172.21.15.103:0/3684813617 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73800f450 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7401a6900 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73d7fa640 1 -- 172.21.15.103:0/3684813617 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff73800faa0 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff73d7fa640 1 -- 172.21.15.103:0/3684813617 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738018b70 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.592+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401a6df0 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.594+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff740100f30 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.595+0000 7ff73d7fa640 1 -- 172.21.15.103:0/3684813617 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff738018d10 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.595+0000 7ff73d7fa640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff72c0a2b80 0x7ff72c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.595+0000 7ff73f7fe640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff72c0a2b80 0x7ff72c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.595+0000 7ff73d7fa640 1 -- 172.21.15.103:0/3684813617 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff7380cf3e0 con 0x7ff740102c20 2024-08-19T01:39:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.596+0000 7ff73f7fe640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff72c0a2b80 0x7ff72c0a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff730009230 tx=0x7ff7300058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:00.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.601+0000 7ff73d7fa640 1 -- 172.21.15.103:0/3684813617 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff738014030 con 0x7ff740102c20 2024-08-19T01:39:00.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.838+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7401a70b0 con 0x7ff740102c20 2024-08-19T01:39:00.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.839+0000 7ff73d7fa640 1 -- 172.21.15.103:0/3684813617 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11991 (secure 0 0 0) 0x7ff73809c370 con 0x7ff740102c20 2024-08-19T01:39:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:39:00.841 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-08-19T01:37:36:616371+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:37:36.566253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:39:00.842 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-08-19T01:39:00.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff72c0a2b80 msgr2=0x7ff72c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:00.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff72c0a2b80 0x7ff72c0a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff730009230 tx=0x7ff7300058d0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740102c20 msgr2=0x7ff740103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740102c20 0x7ff740103f00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff73800cce0 tx=0x7ff73800a620 comp rx=0 tx=0).stop 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 shutdown_connections 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff72c0a2b80 0x7ff72c0a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff740102c20 0x7ff740103f00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7401022a0 0x7ff7401039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.841+0000 7ff7461a7640 1 --2- 172.21.15.103:0/3684813617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff740100b50 0x7ff740103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.842+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 >> 172.21.15.103:0/3684813617 conn(0x7ff7400fc500 msgr2=0x7ff740069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.842+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 shutdown_connections 2024-08-19T01:39:00.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:00.842+0000 7ff7461a7640 1 -- 172.21.15.103:0/3684813617 wait complete. 2024-08-19T01:39:01.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:00 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3684813617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:39:01.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:00 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3684813617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:39:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:00 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3684813617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:39:01.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:39:01.933 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:02.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:01 smithi146 ceph-mon[22098]: pgmap v436: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:39:02.203 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:01 smithi103 ceph-mon[18436]: pgmap v436: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:39:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:01 smithi190 ceph-mon[22032]: pgmap v436: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:39:02.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 -- 172.21.15.103:0/3085716754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1208102c20 msgr2=0x7f120810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 --2- 172.21.15.103:0/3085716754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1208102c20 0x7f120810d770 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1200009ca0 tx=0x7f120002a950 comp rx=0 tx=0).stop 2024-08-19T01:39:02.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 -- 172.21.15.103:0/3085716754 shutdown_connections 2024-08-19T01:39:02.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 --2- 172.21.15.103:0/3085716754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1208102c20 0x7f120810d770 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 --2- 172.21.15.103:0/3085716754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12081022a0 0x7f12081026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 --2- 172.21.15.103:0/3085716754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1208100b50 0x7f1208100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 -- 172.21.15.103:0/3085716754 >> 172.21.15.103:0/3085716754 conn(0x7f12080fc500 msgr2=0x7f12080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 -- 172.21.15.103:0/3085716754 shutdown_connections 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 -- 172.21.15.103:0/3085716754 wait complete. 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.442+0000 7f120f0bf640 1 Processor -- start 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120f0bf640 1 -- start start 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120f0bf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1208100b50 0x7f1208196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120f0bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12081022a0 0x7f1208197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120f0bf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1208102c20 0x7f12081a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120f0bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1208110bb0 con 0x7f12081022a0 2024-08-19T01:39:02.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120f0bf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1208110a30 con 0x7f1208102c20 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120f0bf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1208110d30 con 0x7f1208100b50 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12081022a0 0x7f1208197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12081022a0 0x7f1208197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41060/0 (socket says 172.21.15.103:41060) 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120ce5b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1208100b50 0x7f1208196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 -- 172.21.15.103:0/154230813 learned_addr learned my addr 172.21.15.103:0/154230813 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120d65c640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1208102c20 0x7f12081a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 -- 172.21.15.103:0/154230813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1208100b50 msgr2=0x7f1208196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1208100b50 0x7f1208196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 -- 172.21.15.103:0/154230813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1208102c20 msgr2=0x7f12081a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1208102c20 0x7f12081a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f1207fff640 1 -- 172.21.15.103:0/154230813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12081a65f0 con 0x7f12081022a0 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120ce5b640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1208100b50 0x7f1208196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.443+0000 7f120d65c640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1208102c20 0x7f12081a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:02.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.444+0000 7f1207fff640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12081022a0 0x7f1208197000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d770 tx=0x7f11f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.444+0000 7f1205ffb640 1 -- 172.21.15.103:0/154230813 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4017070 con 0x7f12081022a0 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.444+0000 7f1205ffb640 1 -- 172.21.15.103:0/154230813 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f40054d0 con 0x7f12081022a0 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.444+0000 7f1205ffb640 1 -- 172.21.15.103:0/154230813 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4004cc0 con 0x7f12081022a0 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.444+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12081a6880 con 0x7f12081022a0 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.444+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a6dc0 con 0x7f12081022a0 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.446+0000 7f1205ffb640 1 -- 172.21.15.103:0/154230813 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f11f4004e60 con 0x7f12081022a0 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 Processor -- start 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 -- start start 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e3f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e49c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e4c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0da4a40 con 0x55b5a0e3f910 2024-08-19T01:39:02.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0d3c9d0 con 0x55b5a0e3eff0 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f12114bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0da0a00 con 0x55b5a0e4a1d0 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f120ce5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e49c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f120ce5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e49c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41074/0 (socket says 172.21.15.103:41074) 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f120ce5b640 1 -- 172.21.15.103:0/2051032581 learned_addr learned my addr 172.21.15.103:0/2051032581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.447+0000 7f1207fff640 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e4c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f120ce5b640 1 -- 172.21.15.103:0/2051032581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e4a1d0 msgr2=0x55b5a0e4c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f120ce5b640 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e4c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f120ce5b640 1 -- 172.21.15.103:0/2051032581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e3eff0 msgr2=0x55b5a0e3f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f120ce5b640 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e3f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f1205ffb640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11f80a2c00 0x7f11f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f120ce5b640 1 -- 172.21.15.103:0/2051032581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a0d43650 con 0x55b5a0e3f910 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f1207fff640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11f80a2c00 0x7f11f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f1205ffb640 1 -- 172.21.15.103:0/154230813 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f11f40cea90 con 0x7f12081022a0 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f120ce5b640 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e49c90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00bd40 tx=0x7f11fc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52252a31c3572350 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f1207fff640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11f80a2c00 0x7f11f80a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f11f40cc930 tx=0x7f11f40bb040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.448+0000 7f11f2ffd640 1 -- 172.21.15.103:0/2051032581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc015070 con 0x55b5a0e3f910 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f11f2ffd640 1 -- 172.21.15.103:0/2051032581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11fc00be30 con 0x55b5a0e3f910 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f11f2ffd640 1 -- 172.21.15.103:0/2051032581 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc011990 con 0x55b5a0e3f910 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 -- 172.21.15.103:0/2051032581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 msgr2=0x55b5a0e49c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e49c90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00bd40 tx=0x7f11fc00f730 comp rx=0 tx=0).stop 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 -- 172.21.15.103:0/2051032581 shutdown_connections 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e4c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e49c90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 --2- 172.21.15.103:0/2051032581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e3f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 -- 172.21.15.103:0/2051032581 >> 172.21.15.103:0/2051032581 conn(0x55b5a0e3b580 msgr2=0x55b5a0e3b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 -- 172.21.15.103:0/2051032581 shutdown_connections 2024-08-19T01:39:02.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.449+0000 7f12114bd740 1 -- 172.21.15.103:0/2051032581 wait complete. 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 -- start start 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 Processor -- start 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e47b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0e4cda0 con 0x55b5a0e3f910 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0d5fb20 con 0x55b5a0e4a1d0 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b5a0e4cf20 con 0x55b5a0e3eff0 2024-08-19T01:39:02.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f12114bd740 10 client.0 _ll_register_callbacks cb 0x55b5a0d5dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.450+0000 7f120ce5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41088/0 (socket says 172.21.15.103:41088) 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 -- 172.21.15.103:0/2057133217 learned_addr learned my addr 172.21.15.103:0/2057133217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120d65c640 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e47b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120d65c640 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 -- 172.21.15.103:0/2057133217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 msgr2=0x55b5a0e47090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 -- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 msgr2=0x55b5a0e47b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e47b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 -- 172.21.15.103:0/2057133217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a0b163c0 con 0x55b5a0e3f910 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f120ce5b640 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e475d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f11fc010010 tx=0x7f11fc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522dc7feae9465f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f11e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:02.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f11e7fff640 1 -- 172.21.15.103:0/2057133217 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc015070 con 0x55b5a0e3f910 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f11e7fff640 1 -- 172.21.15.103:0/2057133217 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f11fc005390 con 0x55b5a0e3f910 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.451+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc00be30 con 0x55b5a0e3f910 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f11e7fff640 1 -- 172.21.15.103:0/2057133217 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11fc0055d0 con 0x55b5a0e3f910 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5a0e58980 con 0x55b5a0e3f910 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f11e7fff640 1 -- 172.21.15.103:0/2057133217 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11fc00be30 con 0x55b5a0e3f910 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f11e7fff640 1 -- 172.21.15.103:0/2057133217 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f11fc0046c0 con 0x55b5a0e3f910 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 20 client.16188 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0e59e40 0x55b5a0e5c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 4 client.16188 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:02.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.452+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5a0db1fb0 con 0x55b5a0e59e40 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.453+0000 7f120d65c640 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0e59e40 0x55b5a0e5c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.453+0000 7f11e7fff640 10 client.16188 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.453+0000 7f120d65c640 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0e59e40 0x55b5a0e5c230 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f187e5285bb6bbb server_cookie=ff4d106a3bb1164c in_seq=0 out_seq=0 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f11e7fff640 1 -- 172.21.15.103:0/2057133217 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5a0db1fb0 con 0x55b5a0e59e40 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f11e7fff640 10 client.16188 handle_command_reply: tid=0 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 client.16188 shutdown 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 msgr2=0x55b5a0e475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e475d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f11fc010010 tx=0x7f11fc00bab0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 shutdown_connections 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0e59e40 0x55b5a0e5c230 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e47b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e475d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 --2- 172.21.15.103:0/2057133217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 >> 172.21.15.103:0/2057133217 conn(0x55b5a0e3b580 msgr2=0x55b5a0e412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:02.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f11e7fff640 0 client.16188 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:02.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 shutdown_connections 2024-08-19T01:39:02.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 1 -- 172.21.15.103:0/2057133217 wait complete. 2024-08-19T01:39:02.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.454+0000 7f12114bd740 20 client.16188 trim_cache size 0 max 16384 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 Processor -- start 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 -- start start 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3f910 0x55b5a0e61860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12081af2c0 con 0x55b5a0e3eff0 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b0e60 con 0x55b5a0e4a1d0 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f12114bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12081af990 con 0x55b5a0e3f910 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f120d65c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f120d65c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41090/0 (socket says 172.21.15.103:41090) 2024-08-19T01:39:02.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.469+0000 7f120d65c640 1 -- 172.21.15.103:0/1542278866 learned_addr learned my addr 172.21.15.103:0/1542278866 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120d65c640 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120ce5b640 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3f910 0x55b5a0e61860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120d65c640 1 -- 172.21.15.103:0/1542278866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3f910 msgr2=0x55b5a0e61860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120d65c640 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3f910 0x55b5a0e61860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120d65c640 1 -- 172.21.15.103:0/1542278866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 msgr2=0x55b5a0e61da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120d65c640 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e61da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120d65c640 1 -- 172.21.15.103:0/1542278866 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a0e3c630 con 0x55b5a0e3eff0 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120ce5b640 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3f910 0x55b5a0e61860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f120d65c640 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47960 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f120002ae70 tx=0x7f1200009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35953d49db814fa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f11f27fc640 1 -- 172.21.15.103:0/1542278866 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120004e070 con 0x55b5a0e3eff0 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f11f27fc640 1 -- 172.21.15.103:0/1542278866 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f120002fd70 con 0x55b5a0e3eff0 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.470+0000 7f11f27fc640 1 -- 172.21.15.103:0/1542278866 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200034040 con 0x55b5a0e3eff0 2024-08-19T01:39:02.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 -- 172.21.15.103:0/1542278866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3eff0 msgr2=0x55b5a0e47960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47960 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f120002ae70 tx=0x7f1200009860 comp rx=0 tx=0).stop 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 -- 172.21.15.103:0/1542278866 shutdown_connections 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e61da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3f910 0x55b5a0e61860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 --2- 172.21.15.103:0/1542278866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e47960 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 -- 172.21.15.103:0/1542278866 >> 172.21.15.103:0/1542278866 conn(0x55b5a0e3b580 msgr2=0x55b5a0e3a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 -- 172.21.15.103:0/1542278866 shutdown_connections 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 -- 172.21.15.103:0/1542278866 wait complete. 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.471+0000 7f12114bd740 1 -- start start 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 1 Processor -- start 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e44d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e52170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b0760 con 0x55b5a0e3f910 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f12081b0060 con 0x55b5a0e4a1d0 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad2c0 con 0x55b5a0e3eff0 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f12114bd740 10 client.0 _ll_register_callbacks cb 0x7f11fc00bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f120ce5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e52170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f120ce5b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e52170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41104/0 (socket says 172.21.15.103:41104) 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f120ce5b640 1 -- 172.21.15.103:0/1986183028 learned_addr learned my addr 172.21.15.103:0/1986183028 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f120d65c640 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.472+0000 7f120d65c640 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e44d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f120ce5b640 1 -- 172.21.15.103:0/1986183028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 msgr2=0x55b5a0e44d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f120ce5b640 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e44d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f120ce5b640 1 -- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 msgr2=0x55b5a0e52900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f120ce5b640 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e52900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f120ce5b640 1 -- 172.21.15.103:0/1986183028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a0e43400 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f120ce5b640 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e52170 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006010 tx=0x7f11fc00b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b0c3aa873c97cf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f11e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f11e7fff640 1 -- 172.21.15.103:0/1986183028 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc015040 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f11e7fff640 1 -- 172.21.15.103:0/1986183028 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f11fc004d00 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120002fd70 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f11e7fff640 1 -- 172.21.15.103:0/1986183028 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11fc011e60 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f12114bd740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5a0ec3c20 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.473+0000 7f11e7fff640 1 -- 172.21.15.103:0/1986183028 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f120002fd70 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f11e7fff640 1 -- 172.21.15.103:0/1986183028 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f11fc01a4f0 con 0x55b5a0e3f910 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f12114bd740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:39:02.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f12114bd740 10 client.16194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:39:02.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f12114bd740 20 client.16194 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:02.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f12114bd740 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0ec42e0 0x55b5a0ec6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:02.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f12114bd740 4 client.16194 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:02.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5a0dac4c0 con 0x55b5a0ec42e0 2024-08-19T01:39:02.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f120d65c640 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0ec42e0 0x55b5a0ec6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f120d65c640 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0ec42e0 0x55b5a0ec6700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cab257f1da91a25c server_cookie=f056542c260aa161 in_seq=0 out_seq=0 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.474+0000 7f11e7fff640 10 client.16194 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f11e7fff640 1 -- 172.21.15.103:0/1986183028 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5a0dac4c0 con 0x55b5a0ec42e0 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f11e7fff640 10 client.16194 handle_command_reply: tid=0 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 client.16194 shutdown 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 msgr2=0x55b5a0e52170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e52170 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006010 tx=0x7f11fc00b5e0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 shutdown_connections 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55b5a0ec42e0 0x55b5a0ec6700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b5a0e4a1d0 0x55b5a0e52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f11e7fff640 0 client.16194 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5a0e3f910 0x55b5a0e52170 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 --2- 172.21.15.103:0/1986183028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b5a0e3eff0 0x55b5a0e44d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 >> 172.21.15.103:0/1986183028 conn(0x55b5a0e3b580 msgr2=0x55b5a0e48700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 shutdown_connections 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 1 -- 172.21.15.103:0/1986183028 wait complete. 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.475+0000 7f12114bd740 20 client.16194 trim_cache size 0 max 16384 2024-08-19T01:39:02.483 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:02.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11f80a2c00 msgr2=0x7f11f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11f80a2c00 0x7f11f80a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f11f40cc930 tx=0x7f11f40bb040 comp rx=0 tx=0).stop 2024-08-19T01:39:02.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12081022a0 msgr2=0x7f1208197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:02.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12081022a0 0x7f1208197000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d770 tx=0x7f11f400dc30 comp rx=0 tx=0).stop 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 shutdown_connections 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f11f80a2c00 0x7f11f80a50c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1208102c20 0x7f12081a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12081022a0 0x7f1208197000 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 --2- 172.21.15.103:0/154230813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1208100b50 0x7f1208196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 >> 172.21.15.103:0/154230813 conn(0x7f12080fc500 msgr2=0x7f12080fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 shutdown_connections 2024-08-19T01:39:02.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:02.476+0000 7f120f0bf640 1 -- 172.21.15.103:0/154230813 wait complete. 2024-08-19T01:39:03.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:02 smithi103 ceph-mon[18436]: pgmap v437: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:39:03.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:02 smithi146 ceph-mon[22098]: pgmap v437: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:39:03.292 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:39:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:02 smithi190 ceph-mon[22032]: pgmap v437: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:39:03.581 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:04.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 -- 172.21.15.103:0/1079496890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab54100f70 msgr2=0x7fab5410db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1079496890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab54100f70 0x7fab5410db60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fab40009900 tx=0x7fab4002a410 comp rx=0 tx=0).stop 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 -- 172.21.15.103:0/1079496890 shutdown_connections 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1079496890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab5410e130 0x7fab54110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1079496890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab54100f70 0x7fab5410db60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1079496890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab54100650 0x7fab54100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 -- 172.21.15.103:0/1079496890 >> 172.21.15.103:0/1079496890 conn(0x7fab54075ca0 msgr2=0x7fab540760b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 -- 172.21.15.103:0/1079496890 shutdown_connections 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.131+0000 7fab5aa96640 1 -- 172.21.15.103:0/1079496890 wait complete. 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 Processor -- start 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 -- start start 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab54100650 0x7fab5419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab54100f70 0x7fab5419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab5410e130 0x7fab5419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab54113d50 con 0x7fab5410e130 2024-08-19T01:39:04.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab54113bd0 con 0x7fab54100650 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab5aa96640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fab54113ed0 con 0x7fab54100f70 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab59033640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab5410e130 0x7fab5419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.132+0000 7fab59033640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab5410e130 0x7fab5419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41134/0 (socket says 172.21.15.103:41134) 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab59033640 1 -- 172.21.15.103:0/1714791114 learned_addr learned my addr 172.21.15.103:0/1714791114 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab58832640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab54100650 0x7fab5419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab53fff640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab54100f70 0x7fab5419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab59033640 1 -- 172.21.15.103:0/1714791114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab54100f70 msgr2=0x7fab5419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab59033640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab54100f70 0x7fab5419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab59033640 1 -- 172.21.15.103:0/1714791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab54100650 msgr2=0x7fab5419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab59033640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab54100650 0x7fab5419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab59033640 1 -- 172.21.15.103:0/1714791114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5419f2c0 con 0x7fab5410e130 2024-08-19T01:39:04.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab59033640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab5410e130 0x7fab5419ebe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00eb40 tx=0x7fab4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab51ffb640 1 -- 172.21.15.103:0/1714791114 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c004b20 con 0x7fab5410e130 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5419f5b0 con 0x7fab5410e130 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab51ffb640 1 -- 172.21.15.103:0/1714791114 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab4c004cc0 con 0x7fab5410e130 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.133+0000 7fab51ffb640 1 -- 172.21.15.103:0/1714791114 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0107b0 con 0x7fab5410e130 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.134+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab541a6e90 con 0x7fab5410e130 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.136+0000 7fab51ffb640 1 -- 172.21.15.103:0/1714791114 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fab4c005020 con 0x7fab5410e130 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 Processor -- start 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 -- start start 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb587063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb58710c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab51ffb640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fab440a2b80 0x7fab440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb5866ba40 con 0x55cb58706910 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cb586039d0 con 0x55cb587111d0 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab5ce94740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cb58667a00 con 0x55cb58705ff0 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.137+0000 7fab58832640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb58710c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab59033640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fab440a2b80 0x7fab440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb58710c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41142/0 (socket says 172.21.15.103:41142) 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 -- 172.21.15.103:0/3730847727 learned_addr learned my addr 172.21.15.103:0/3730847727 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab51ffb640 1 -- 172.21.15.103:0/1714791114 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fab4c0cf7b0 con 0x7fab5410e130 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab53fff640 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 -- 172.21.15.103:0/3730847727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 msgr2=0x55cb587063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb587063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 -- 172.21.15.103:0/3730847727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 msgr2=0x55cb587135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 -- 172.21.15.103:0/3730847727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5860a650 con 0x55cb58706910 2024-08-19T01:39:04.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab53fff640 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab58832640 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb58710c90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fab4800ac20 tx=0x7fab4800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1872c541cdb311 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.138+0000 7fab59033640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fab440a2b80 0x7fab440a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fab4c046550 tx=0x7fab4c066c10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.139+0000 7fab3effd640 1 -- 172.21.15.103:0/3730847727 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4801d070 con 0x55cb58706910 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.139+0000 7fab3effd640 1 -- 172.21.15.103:0/3730847727 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab48018af0 con 0x55cb58706910 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.139+0000 7fab3effd640 1 -- 172.21.15.103:0/3730847727 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48010af0 con 0x55cb58706910 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 -- 172.21.15.103:0/3730847727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 msgr2=0x55cb58710c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb58710c90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fab4800ac20 tx=0x7fab4800f790 comp rx=0 tx=0).stop 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 -- 172.21.15.103:0/3730847727 shutdown_connections 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb58710c90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 --2- 172.21.15.103:0/3730847727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb587063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 -- 172.21.15.103:0/3730847727 >> 172.21.15.103:0/3730847727 conn(0x55cb58702580 msgr2=0x55cb58702960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:04.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 -- 172.21.15.103:0/3730847727 shutdown_connections 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 -- 172.21.15.103:0/3730847727 wait complete. 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.140+0000 7fab5ce94740 1 -- start start 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.141+0000 7fab5ce94740 1 Processor -- start 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.141+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb5870e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.141+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb5870e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb5870e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb5870e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41150/0 (socket says 172.21.15.103:41150) 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 -- 172.21.15.103:0/1044455593 learned_addr learned my addr 172.21.15.103:0/1044455593 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb5870eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb58713da0 con 0x55cb58706910 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cb58626b20 con 0x55cb587111d0 2024-08-19T01:39:04.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cb58713f20 con 0x55cb58705ff0 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab53fff640 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb5870e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab53fff640 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb5870eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 -- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 msgr2=0x55cb5870e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb5870e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 -- 172.21.15.103:0/1044455593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 msgr2=0x55cb5870eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb5870eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.142+0000 7fab58832640 1 -- 172.21.15.103:0/1044455593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb583dd3c0 con 0x55cb58706910 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab58832640 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb5870e600 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fab480105c0 tx=0x7fab48018960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2fb1dcebcee91e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab33fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab5ce94740 10 client.0 _ll_register_callbacks cb 0x55cb58624bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab48018af0 con 0x55cb58706910 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab33fff640 1 -- 172.21.15.103:0/1044455593 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48027020 con 0x55cb58706910 2024-08-19T01:39:04.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab33fff640 1 -- 172.21.15.103:0/1044455593 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fab48005240 con 0x55cb58706910 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab33fff640 1 -- 172.21.15.103:0/1044455593 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab4801d040 con 0x55cb58706910 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab5ce94740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.143+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55cb5871f720 con 0x55cb58706910 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab33fff640 1 -- 172.21.15.103:0/1044455593 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fab48005480 con 0x55cb58706910 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab5ce94740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab5ce94740 10 client.16206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab5ce94740 20 client.16206 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb58720ad0 0x55cb58722ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab5ce94740 4 client.16206 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb58678fb0 con 0x55cb58720ad0 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.144+0000 7fab53fff640 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb58720ad0 0x55cb58722ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.145+0000 7fab33fff640 10 client.16206 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.145+0000 7fab53fff640 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb58720ad0 0x55cb58722ec0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=baa89f923e68e481 server_cookie=c16dbc7d91300bad in_seq=0 out_seq=0 2024-08-19T01:39:04.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab33fff640 1 -- 172.21.15.103:0/1044455593 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb58678fb0 con 0x55cb58720ad0 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab33fff640 10 client.16206 handle_command_reply: tid=0 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 client.16206 shutdown 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 msgr2=0x55cb5870e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb5870e600 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fab480105c0 tx=0x7fab48018960 comp rx=0 tx=0).stop 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 shutdown_connections 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb58720ad0 0x55cb58722ec0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb5870eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb5870e600 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1044455593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb5870e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 >> 172.21.15.103:0/1044455593 conn(0x55cb58702580 msgr2=0x55cb58708260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab33fff640 0 client.16206 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 shutdown_connections 2024-08-19T01:39:04.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 1 -- 172.21.15.103:0/1044455593 wait complete. 2024-08-19T01:39:04.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.146+0000 7fab5ce94740 20 client.16206 trim_cache size 0 max 16384 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 Processor -- start 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 -- start start 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb58705ff0 0x55cb587089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58706910 0x55cb5870e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb587111d0 0x55cb5870ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab541aaf40 con 0x55cb587111d0 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab541acae0 con 0x55cb58705ff0 2024-08-19T01:39:04.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.159+0000 7fab5ce94740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fab541ab610 con 0x55cb58706910 2024-08-19T01:39:04.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb587111d0 0x55cb5870ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb587111d0 0x55cb5870ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41164/0 (socket says 172.21.15.103:41164) 2024-08-19T01:39:04.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 -- 172.21.15.103:0/1472520022 learned_addr learned my addr 172.21.15.103:0/1472520022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:04.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab58832640 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58706910 0x55cb5870e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb58705ff0 0x55cb587089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 -- 172.21.15.103:0/1472520022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58706910 msgr2=0x55cb5870e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58706910 0x55cb5870e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 -- 172.21.15.103:0/1472520022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb58705ff0 msgr2=0x55cb587089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb58705ff0 0x55cb587089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 -- 172.21.15.103:0/1472520022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb58710b70 con 0x55cb587111d0 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab53fff640 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb587111d0 0x55cb5870ff20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fab40009900 tx=0x7fab40031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67984cf2e168081a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab3e7fc640 1 -- 172.21.15.103:0/1472520022 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40035040 con 0x55cb587111d0 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab3e7fc640 1 -- 172.21.15.103:0/1472520022 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab40004020 con 0x55cb587111d0 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.160+0000 7fab3e7fc640 1 -- 172.21.15.103:0/1472520022 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4004be10 con 0x55cb587111d0 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 -- 172.21.15.103:0/1472520022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb587111d0 msgr2=0x55cb5870ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb587111d0 0x55cb5870ff20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fab40009900 tx=0x7fab40031400 comp rx=0 tx=0).stop 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 -- 172.21.15.103:0/1472520022 shutdown_connections 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb587111d0 0x55cb5870ff20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58706910 0x55cb5870e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 --2- 172.21.15.103:0/1472520022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb58705ff0 0x55cb587089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 -- 172.21.15.103:0/1472520022 >> 172.21.15.103:0/1472520022 conn(0x55cb58702580 msgr2=0x55cb5870f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:04.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 -- 172.21.15.103:0/1472520022 shutdown_connections 2024-08-19T01:39:04.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.161+0000 7fab5ce94740 1 -- 172.21.15.103:0/1472520022 wait complete. 2024-08-19T01:39:04.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 -- start start 2024-08-19T01:39:04.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 Processor -- start 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb58719490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb587199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab541ac3e0 con 0x55cb58706910 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fab541abce0 con 0x55cb587111d0 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fab541a8f20 con 0x55cb58705ff0 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab5ce94740 10 client.0 _ll_register_callbacks cb 0x7fab4801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab58832640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb587199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab58832640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb587199d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41178/0 (socket says 172.21.15.103:41178) 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab58832640 1 -- 172.21.15.103:0/2899723286 learned_addr learned my addr 172.21.15.103:0/2899723286 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.162+0000 7fab53fff640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb58719490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab53fff640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab58832640 1 -- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 msgr2=0x55cb58719490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab58832640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb58719490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab58832640 1 -- 172.21.15.103:0/2899723286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 msgr2=0x55cb587148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab58832640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab58832640 1 -- 172.21.15.103:0/2899723286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb58715110 con 0x55cb58706910 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab53fff640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb58719490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab58832640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb587199d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fab4800a520 tx=0x7fab48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decc64087e48b8fb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab33fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:04.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab33fff640 1 -- 172.21.15.103:0/2899723286 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48027040 con 0x55cb58706910 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab33fff640 1 -- 172.21.15.103:0/2899723286 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fab48010b90 con 0x55cb58706910 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab40004020 con 0x55cb58706910 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab33fff640 1 -- 172.21.15.103:0/2899723286 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab48010d90 con 0x55cb58706910 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab5ce94740 10 client.16212 fetch_fsmap learned FSMap version 29 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.163+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb5878b1b0 con 0x55cb58706910 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab33fff640 1 -- 172.21.15.103:0/2899723286 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab40004020 con 0x55cb58706910 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab33fff640 1 -- 172.21.15.103:0/2899723286 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fab48005040 con 0x55cb58706910 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab5ce94740 10 client.16212 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab5ce94740 10 client.16212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab5ce94740 20 client.16212 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab5ce94740 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb5878b860 0x55cb5878dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab5ce94740 4 client.16212 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb586734c0 con 0x55cb5878b860 2024-08-19T01:39:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.164+0000 7fab53fff640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb5878b860 0x55cb5878dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab53fff640 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb5878b860 0x55cb5878dc50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be160d60bb3b5136 server_cookie=341dd616d52e6068 in_seq=0 out_seq=0 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab33fff640 10 client.16212 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab33fff640 1 -- 172.21.15.103:0/2899723286 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb586734c0 con 0x55cb5878b860 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab33fff640 10 client.16212 handle_command_reply: tid=0 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 client.16212 shutdown 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 msgr2=0x55cb587199d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb587199d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fab4800a520 tx=0x7fab48000bc0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 shutdown_connections 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55cb5878b860 0x55cb5878dc50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cb587111d0 0x55cb587148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb58706910 0x55cb587199d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 --2- 172.21.15.103:0/2899723286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb58705ff0 0x55cb58719490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 >> 172.21.15.103:0/2899723286 conn(0x55cb58702580 msgr2=0x55cb587015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:04.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab33fff640 0 client.16212 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:04.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 shutdown_connections 2024-08-19T01:39:04.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 1 -- 172.21.15.103:0/2899723286 wait complete. 2024-08-19T01:39:04.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.165+0000 7fab5ce94740 20 client.16212 trim_cache size 0 max 16384 2024-08-19T01:39:04.172 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fab440a2b80 msgr2=0x7fab440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fab440a2b80 0x7fab440a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fab4c046550 tx=0x7fab4c066c10 comp rx=0 tx=0).stop 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab5410e130 msgr2=0x7fab5419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab5410e130 0x7fab5419ebe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00eb40 tx=0x7fab4c00c2d0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 shutdown_connections 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fab440a2b80 0x7fab440a5040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab5410e130 0x7fab5419ebe0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab54100f70 0x7fab5419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 --2- 172.21.15.103:0/1714791114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fab54100650 0x7fab5419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 >> 172.21.15.103:0/1714791114 conn(0x7fab54075ca0 msgr2=0x7fab5410e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 shutdown_connections 2024-08-19T01:39:04.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:04.166+0000 7fab5aa96640 1 -- 172.21.15.103:0/1714791114 wait complete. 2024-08-19T01:39:05.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:39:05.418 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:05.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 -- 172.21.15.103:0/27345742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4104460 msgr2=0x7f39c4110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/27345742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4104460 0x7f39c4110990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009cd0 tx=0x7f39bc02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 -- 172.21.15.103:0/27345742 shutdown_connections 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/27345742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4104460 0x7f39c4110990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/27345742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c4103ac0 0x7f39c4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/27345742 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39c4109ac0 0x7f39c4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 -- 172.21.15.103:0/27345742 >> 172.21.15.103:0/27345742 conn(0x7f39c40ff7e0 msgr2=0x7f39c4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 -- 172.21.15.103:0/27345742 shutdown_connections 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.915+0000 7f39cbcfd640 1 -- 172.21.15.103:0/27345742 wait complete. 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 Processor -- start 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 -- start start 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4103ac0 0x7f39c419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39c4104460 0x7f39c419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c4109ac0 0x7f39c419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4113d70 con 0x7f39c4103ac0 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4113bf0 con 0x7f39c4104460 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39cbcfd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4113ef0 con 0x7f39c4109ac0 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.916+0000 7f39c9a99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4103ac0 0x7f39c419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4103ac0 0x7f39c419a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41208/0 (socket says 172.21.15.103:41208) 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 -- 172.21.15.103:0/3470543486 learned_addr learned my addr 172.21.15.103:0/3470543486 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9298640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39c4104460 0x7f39c419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39ca29a640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c4109ac0 0x7f39c419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 -- 172.21.15.103:0/3470543486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c4109ac0 msgr2=0x7f39c419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c4109ac0 0x7f39c419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 -- 172.21.15.103:0/3470543486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39c4104460 msgr2=0x7f39c419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39c4104460 0x7f39c419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 -- 172.21.15.103:0/3470543486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c419f220 con 0x7f39c4103ac0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9298640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39c4104460 0x7f39c419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39ca29a640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c4109ac0 0x7f39c419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39c9a99640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4103ac0 0x7f39c419a270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f39c000d950 tx=0x7f39c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39baffd640 1 -- 172.21.15.103:0/3470543486 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0017030 con 0x7f39c4103ac0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39baffd640 1 -- 172.21.15.103:0/3470543486 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39c0004020 con 0x7f39c4103ac0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c419f510 con 0x7f39c4103ac0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39baffd640 1 -- 172.21.15.103:0/3470543486 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00047a0 con 0x7f39c4103ac0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.917+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c41a6df0 con 0x7f39c4103ac0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.920+0000 7f39baffd640 1 -- 172.21.15.103:0/3470543486 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f39c00041d0 con 0x7f39c4103ac0 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.920+0000 7f39ce0fb740 1 Processor -- start 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.920+0000 7f39baffd640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f39b00a2b80 0x7f39b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ca29a640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f39b00a2b80 0x7f39b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ce0fb740 1 -- start start 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39baffd640 1 -- 172.21.15.103:0/3470543486 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f39c00ceaa0 con 0x7f39c4103ac0 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a05ff0 0x557ca3a063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a06910 0x557ca3a10c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ca396ba40 con 0x557ca3a06910 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557ca39039d0 con 0x557ca3a05ff0 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557ca3967a00 con 0x557ca3a111d0 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39c9298640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a06910 0x557ca3a10c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39c9298640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a06910 0x557ca3a10c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41210/0 (socket says 172.21.15.103:41210) 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39c9298640 1 -- 172.21.15.103:0/825053951 learned_addr learned my addr 172.21.15.103:0/825053951 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39c9a99640 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a05ff0 0x557ca3a063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ca29a640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f39b00a2b80 0x7f39b00a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009ca0 tx=0x7f39bc005f80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39ca29a640 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.921+0000 7f39c9298640 1 -- 172.21.15.103:0/825053951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 msgr2=0x557ca3a135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39c9298640 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39c9298640 1 -- 172.21.15.103:0/825053951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a05ff0 msgr2=0x557ca3a063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39c9298640 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a05ff0 0x557ca3a063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39c9298640 1 -- 172.21.15.103:0/825053951 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca390a650 con 0x557ca3a06910 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39ca29a640 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39c9a99640 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a05ff0 0x557ca3a063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39c9298640 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a06910 0x557ca3a10c90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009c00 tx=0x7f39b4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2368f840f6f3a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39affff640 1 -- 172.21.15.103:0/825053951 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4004030 con 0x557ca3a06910 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39affff640 1 -- 172.21.15.103:0/825053951 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39b40035f0 con 0x557ca3a06910 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.922+0000 7f39affff640 1 -- 172.21.15.103:0/825053951 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b40126b0 con 0x557ca3a06910 2024-08-19T01:39:05.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 -- 172.21.15.103:0/825053951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a06910 msgr2=0x557ca3a10c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a06910 0x557ca3a10c90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009c00 tx=0x7f39b4010740 comp rx=0 tx=0).stop 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 -- 172.21.15.103:0/825053951 shutdown_connections 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a06910 0x557ca3a10c90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/825053951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a05ff0 0x557ca3a063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 -- 172.21.15.103:0/825053951 >> 172.21.15.103:0/825053951 conn(0x557ca3a02580 msgr2=0x557ca3a02960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 -- 172.21.15.103:0/825053951 shutdown_connections 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 -- 172.21.15.103:0/825053951 wait complete. 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 -- start start 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.923+0000 7f39ce0fb740 1 Processor -- start 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a0dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ca3a13da0 con 0x557ca3a111d0 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557ca3926b20 con 0x557ca3a06910 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557ca3a13f20 con 0x557ca3a05ff0 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ce0fb740 10 client.0 _ll_register_callbacks cb 0x557ca3924bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39ca29a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39c9298640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39c9298640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a0e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:55952/0 (socket says 172.21.15.103:55952) 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39c9298640 1 -- 172.21.15.103:0/1541968068 learned_addr learned my addr 172.21.15.103:0/1541968068 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39c9a99640 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a0dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39c9298640 1 -- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 msgr2=0x557ca3a0dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39c9298640 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a0dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.924+0000 7f39c9298640 1 -- 172.21.15.103:0/1541968068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 msgr2=0x557ca3a0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39c9298640 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39c9298640 1 -- 172.21.15.103:0/1541968068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca36dd3c0 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ca29a640 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39c9298640 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a0e530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39b400a010 tx=0x7f39b4009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3084812bd49f773 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1541968068 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b402b060 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1541968068 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f39b40039b0 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b40035f0 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1541968068 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39b4003f90 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ce0fb740 10 client.25664 fetch_fsmap learned FSMap version 29 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.925+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ca3a1f860 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1541968068 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39b4012450 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1541968068 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f39b400f040 con 0x557ca3a06910 2024-08-19T01:39:05.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ce0fb740 10 client.25664 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ce0fb740 10 client.25664 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ce0fb740 20 client.25664 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a23000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ce0fb740 4 client.25664 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39b40035f0 con 0x557ca3a20c10 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39c9a99640 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a23000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.926+0000 7f39c9a99640 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a23000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=beb69fd29495fccf server_cookie=e5aca8c8be4b212e in_seq=0 out_seq=0 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ad7fa640 10 client.25664 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1541968068 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f39b40035f0 con 0x557ca3a20c10 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ad7fa640 10 client.25664 handle_command_reply: tid=0 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 client.25664 shutdown 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 msgr2=0x557ca3a0e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a0e530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39b400a010 tx=0x7f39b4009940 comp rx=0 tx=0).stop 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 shutdown_connections 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a23000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a0e530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ad7fa640 0 client.25664 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1541968068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a0dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.927+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 >> 172.21.15.103:0/1541968068 conn(0x557ca3a02580 msgr2=0x557ca3a082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.928+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 shutdown_connections 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.928+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1541968068 wait complete. 2024-08-19T01:39:05.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.928+0000 7f39ce0fb740 20 client.25664 trim_cache size 0 max 16384 2024-08-19T01:39:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 Processor -- start 2024-08-19T01:39:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 -- start start 2024-08-19T01:39:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a05ff0 0x557ca3a08a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a28660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a28ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41af2d0 con 0x557ca3a05ff0 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b0e70 con 0x557ca3a06910 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41af9a0 con 0x557ca3a111d0 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a05ff0 0x557ca3a08a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a05ff0 0x557ca3a08a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41226/0 (socket says 172.21.15.103:41226) 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 -- 172.21.15.103:0/3368922832 learned_addr learned my addr 172.21.15.103:0/3368922832 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9298640 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a28660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39ca29a640 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a28ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 -- 172.21.15.103:0/3368922832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 msgr2=0x557ca3a28ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a28ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 -- 172.21.15.103:0/3368922832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 msgr2=0x557ca3a28660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a28660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.941+0000 7f39c9a99640 1 -- 172.21.15.103:0/3368922832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca3a1d000 con 0x557ca3a05ff0 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.942+0000 7f39c9a99640 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a05ff0 0x557ca3a08a50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f39c00cc5d0 tx=0x7f39c00bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9c1b4ddd1c404d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:05.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.942+0000 7f39af7fe640 1 -- 172.21.15.103:0/3368922832 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00a3fa0 con 0x557ca3a05ff0 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.942+0000 7f39af7fe640 1 -- 172.21.15.103:0/3368922832 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39c00993d0 con 0x557ca3a05ff0 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.942+0000 7f39af7fe640 1 -- 172.21.15.103:0/3368922832 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c009f190 con 0x557ca3a05ff0 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 -- 172.21.15.103:0/3368922832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a05ff0 msgr2=0x557ca3a08a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a05ff0 0x557ca3a08a50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f39c00cc5d0 tx=0x7f39c00bb040 comp rx=0 tx=0).stop 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 -- 172.21.15.103:0/3368922832 shutdown_connections 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a111d0 0x557ca3a28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a28660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/3368922832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a05ff0 0x557ca3a08a50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 -- 172.21.15.103:0/3368922832 >> 172.21.15.103:0/3368922832 conn(0x557ca3a02580 msgr2=0x557ca3a00d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 -- 172.21.15.103:0/3368922832 shutdown_connections 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 -- 172.21.15.103:0/3368922832 wait complete. 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 -- start start 2024-08-19T01:39:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.943+0000 7f39ce0fb740 1 Processor -- start 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a19350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a19890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ce0fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b0770 con 0x557ca3a111d0 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b0070 con 0x557ca3a06910 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ce0fb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41ad2b0 con 0x557ca3a05ff0 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9a99640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a19350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9a99640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a19350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:40366/0 (socket says 172.21.15.103:40366) 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9a99640 1 -- 172.21.15.103:0/1838890111 learned_addr learned my addr 172.21.15.103:0/1838890111 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ce0fb740 10 client.0 _ll_register_callbacks cb 0x7f39c0005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39ca29a640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9298640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a19890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9a99640 1 -- 172.21.15.103:0/1838890111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 msgr2=0x557ca3a19890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9a99640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a19890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9a99640 1 -- 172.21.15.103:0/1838890111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 msgr2=0x557ca3a0a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.944+0000 7f39c9a99640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39c9a99640 1 -- 172.21.15.103:0/1838890111 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca3a0b000 con 0x557ca3a05ff0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ca29a640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39c9a99640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a19350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f39c009b2c0 tx=0x7f39c009bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6941ef7daa9e26fd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1838890111 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00b9060 con 0x557ca3a05ff0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1838890111 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f39c00a3b10 con 0x557ca3a05ff0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c00993d0 con 0x557ca3a05ff0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1838890111 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39c00a3d50 con 0x557ca3a05ff0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ce0fb740 10 client.25645 fetch_fsmap learned FSMap version 29 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ca3a8b0b0 con 0x557ca3a05ff0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.945+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1838890111 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39c00993d0 con 0x557ca3a05ff0 2024-08-19T01:39:05.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1838890111 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f39c0097080 con 0x557ca3a05ff0 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ce0fb740 10 client.25645 fetch_fsmap finished waiting for FSMap version 29 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ce0fb740 10 client.25645 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ce0fb740 20 client.25645 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a8d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ce0fb740 4 client.25645 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ca3978fb0 con 0x557ca3a20c10 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39c9298640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a8d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.946+0000 7f39ad7fa640 10 client.25645 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39c9298640 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a8d5a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eaa1df95a837e486 server_cookie=c9929032442aa3b4 in_seq=0 out_seq=0 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ad7fa640 1 -- 172.21.15.103:0/1838890111 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ca3978fb0 con 0x557ca3a20c10 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ad7fa640 10 client.25645 handle_command_reply: tid=0 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 client.25645 shutdown 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 msgr2=0x557ca3a19350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a19350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f39c009b2c0 tx=0x7f39c009bb50 comp rx=0 tx=0).stop 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 shutdown_connections 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x557ca3a20c10 0x557ca3a8d5a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ca3a111d0 0x557ca3a0a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557ca3a06910 0x557ca3a19890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ad7fa640 0 client.25645 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 --2- 172.21.15.103:0/1838890111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ca3a05ff0 0x557ca3a19350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 >> 172.21.15.103:0/1838890111 conn(0x557ca3a02580 msgr2=0x557ca3a0f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 shutdown_connections 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.947+0000 7f39ce0fb740 1 -- 172.21.15.103:0/1838890111 wait complete. 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39ce0fb740 20 client.25645 trim_cache size 0 max 16384 2024-08-19T01:39:05.956 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f39b00a2b80 msgr2=0x7f39b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f39b00a2b80 0x7f39b00a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009ca0 tx=0x7f39bc005f80 comp rx=0 tx=0).stop 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4103ac0 msgr2=0x7f39c419a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4103ac0 0x7f39c419a270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f39c000d950 tx=0x7f39c000de10 comp rx=0 tx=0).stop 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 shutdown_connections 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f39b00a2b80 0x7f39b00a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39c4109ac0 0x7f39c419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39c4104460 0x7f39c419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 --2- 172.21.15.103:0/3470543486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39c4103ac0 0x7f39c419a270 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:05.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 >> 172.21.15.103:0/3470543486 conn(0x7f39c40ff7e0 msgr2=0x7f39c40fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:05.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 shutdown_connections 2024-08-19T01:39:05.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:05.948+0000 7f39cbcfd640 1 -- 172.21.15.103:0/3470543486 wait complete. 2024-08-19T01:39:06.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:05 smithi146 ceph-mon[22098]: pgmap v438: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-08-19T01:39:06.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:05 smithi103 ceph-mon[18436]: pgmap v438: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-08-19T01:39:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:05 smithi190 ceph-mon[22032]: pgmap v438: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-08-19T01:39:06.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:39:07.147 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:07.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/2552556029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc41022a0 msgr2=0x7f3fc41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/2552556029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc41022a0 0x7f3fc41026e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009900 tx=0x7f3fb402a480 comp rx=0 tx=0).stop 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/2552556029 shutdown_connections 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/2552556029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc4102c20 0x7f3fc410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/2552556029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc41022a0 0x7f3fc41026e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/2552556029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fc4100b50 0x7f3fc4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/2552556029 >> 172.21.15.103:0/2552556029 conn(0x7f3fc40fc500 msgr2=0x7f3fc40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/2552556029 shutdown_connections 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.690+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/2552556029 wait complete. 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 Processor -- start 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 -- start start 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc4100b50 0x7f3fc4198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc41022a0 0x7f3fc41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fc4102c20 0x7f3fc41a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4110b60 con 0x7f3fc4100b50 2024-08-19T01:39:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41109e0 con 0x7f3fc4102c20 2024-08-19T01:39:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fcb6c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4110ce0 con 0x7f3fc41022a0 2024-08-19T01:39:07.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc4100b50 0x7f3fc4198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc4100b50 0x7f3fc4198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41248/0 (socket says 172.21.15.103:41248) 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 -- 172.21.15.103:0/3631391184 learned_addr learned my addr 172.21.15.103:0/3631391184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc41022a0 0x7f3fc41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fc4102c20 0x7f3fc41a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 -- 172.21.15.103:0/3631391184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc41022a0 msgr2=0x7f3fc41994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc41022a0 0x7f3fc41994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 -- 172.21.15.103:0/3631391184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fc4102c20 msgr2=0x7f3fc41a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fc4102c20 0x7f3fc41a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.691+0000 7f3fc9464640 1 -- 172.21.15.103:0/3631391184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41a4550 con 0x7f3fc4100b50 2024-08-19T01:39:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.692+0000 7f3fc9464640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc4100b50 0x7f3fc4198f90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9f0 tx=0x7f3fc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.692+0000 7f3fba7fc640 1 -- 172.21.15.103:0/3631391184 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000cd80 con 0x7f3fc4100b50 2024-08-19T01:39:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.692+0000 7f3fba7fc640 1 -- 172.21.15.103:0/3631391184 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fc00108e0 con 0x7f3fc4100b50 2024-08-19T01:39:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.692+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc41a4720 con 0x7f3fc4100b50 2024-08-19T01:39:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.692+0000 7f3fba7fc640 1 -- 172.21.15.103:0/3631391184 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0018920 con 0x7f3fc4100b50 2024-08-19T01:39:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.692+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41a4c30 con 0x7f3fc4100b50 2024-08-19T01:39:07.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fba7fc640 1 -- 172.21.15.103:0/3631391184 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3fc0010a90 con 0x7f3fc4100b50 2024-08-19T01:39:07.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 Processor -- start 2024-08-19T01:39:07.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 -- start start 2024-08-19T01:39:07.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a69ff0 0x559b56a6a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a6a910 0x559b56a74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b569cfa40 con 0x559b56a69ff0 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559b569679d0 con 0x559b56a6a910 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.695+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b569cba00 con 0x559b56a751d0 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fba7fc640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3fb00a2c00 0x7f3fb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3fb00a2c00 0x7f3fb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a6a910 0x559b56a74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fba7fc640 1 -- 172.21.15.103:0/3631391184 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3fc00cf2f0 con 0x7f3fc4100b50 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a6a910 0x559b56a74c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:55988/0 (socket says 172.21.15.103:55988) 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 -- 172.21.15.103:0/1127815997 learned_addr learned my addr 172.21.15.103:0/1127815997 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a69ff0 0x559b56a6a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 -- 172.21.15.103:0/1127815997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 msgr2=0x559b56a775c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a775c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 -- 172.21.15.103:0/1127815997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a69ff0 msgr2=0x559b56a6a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a69ff0 0x559b56a6a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 -- 172.21.15.103:0/1127815997 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5696e650 con 0x559b56a6a910 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a69ff0 0x559b56a6a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.696+0000 7f3fc9464640 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a6a910 0x559b56a74c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9c0 tx=0x7f3fc009b5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f591312e4ad0f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.697+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3fb00a2c00 0x7f3fb00a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009fd0 tx=0x7f3fbc009840 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.697+0000 7f3faf7fe640 1 -- 172.21.15.103:0/1127815997 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0026040 con 0x559b56a6a910 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.697+0000 7f3faf7fe640 1 -- 172.21.15.103:0/1127815997 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fc009ce50 con 0x559b56a6a910 2024-08-19T01:39:07.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.697+0000 7f3faf7fe640 1 -- 172.21.15.103:0/1127815997 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc009b280 con 0x559b56a6a910 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.697+0000 7f3fcdac6740 1 -- 172.21.15.103:0/1127815997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a6a910 msgr2=0x559b56a74c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.697+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a6a910 0x559b56a74c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9c0 tx=0x7f3fc009b5f0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 -- 172.21.15.103:0/1127815997 shutdown_connections 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a6a910 0x559b56a74c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/1127815997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a69ff0 0x559b56a6a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 -- 172.21.15.103:0/1127815997 >> 172.21.15.103:0/1127815997 conn(0x559b56a66580 msgr2=0x559b56a66960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 -- 172.21.15.103:0/1127815997 shutdown_connections 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 -- 172.21.15.103:0/1127815997 wait complete. 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 -- start start 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.698+0000 7f3fcdac6740 1 Processor -- start 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a6a910 0x559b56a725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a751d0 0x559b56a72b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b56a77da0 con 0x559b56a751d0 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559b5698ab20 con 0x559b56a69ff0 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559b56a77f20 con 0x559b56a6a910 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fcdac6740 10 client.0 _ll_register_callbacks cb 0x559b56988bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc9c65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a751d0 0x559b56a72b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc9c65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a751d0 0x559b56a72b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41268/0 (socket says 172.21.15.103:41268) 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc9c65640 1 -- 172.21.15.103:0/2745959401 learned_addr learned my addr 172.21.15.103:0/2745959401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc9464640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a6a910 0x559b56a725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc8c63640 1 -- 172.21.15.103:0/2745959401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 msgr2=0x559b56a720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc8c63640 1 -- 172.21.15.103:0/2745959401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a751d0 msgr2=0x559b56a72b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a751d0 0x559b56a72b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc8c63640 1 -- 172.21.15.103:0/2745959401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b567413c0 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.699+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a751d0 0x559b56a72b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a6a910 0x559b56a725f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005940 tx=0x7f3fb4031e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b31453032a63f7a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3facff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3facff9640 1 -- 172.21.15.103:0/2745959401 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4035040 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fc009ce50 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3facff9640 1 -- 172.21.15.103:0/2745959401 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3fb4031630 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3facff9640 1 -- 172.21.15.103:0/2745959401 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4031830 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3fcdac6740 10 client.25657 fetch_fsmap learned FSMap version 30 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b56a839f0 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.700+0000 7f3facff9640 1 -- 172.21.15.103:0/2745959401 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fc009ce50 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3facff9640 1 -- 172.21.15.103:0/2745959401 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3fb40047a0 con 0x559b56a6a910 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fcdac6740 10 client.25657 fetch_fsmap finished waiting for FSMap version 30 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fcdac6740 10 client.25657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fcdac6740 20 client.25657 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:07.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56a84e00 0x559b56a87260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fcdac6740 4 client.25657 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b569dcfb0 con 0x559b56a84e00 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fc9464640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56a84e00 0x559b56a87260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3facff9640 10 client.25657 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.701+0000 7f3fc9464640 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56a84e00 0x559b56a87260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d025ee3f93e4c2da server_cookie=1c022a40b43fd82c in_seq=0 out_seq=0 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3facff9640 1 -- 172.21.15.103:0/2745959401 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b569dcfb0 con 0x559b56a84e00 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3facff9640 10 client.25657 handle_command_reply: tid=0 2024-08-19T01:39:07.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 client.25657 shutdown 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a6a910 msgr2=0x559b56a725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a6a910 0x559b56a725f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005940 tx=0x7f3fb4031e10 comp rx=0 tx=0).stop 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 shutdown_connections 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56a84e00 0x559b56a87260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a751d0 0x559b56a72b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a6a910 0x559b56a725f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2745959401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 >> 172.21.15.103:0/2745959401 conn(0x559b56a66580 msgr2=0x559b56a6c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.702+0000 7f3facff9640 0 client.25657 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.703+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 shutdown_connections 2024-08-19T01:39:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.703+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2745959401 wait complete. 2024-08-19T01:39:07.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.703+0000 7f3fcdac6740 20 client.25657 trim_cache size 0 max 16384 2024-08-19T01:39:07.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 Processor -- start 2024-08-19T01:39:07.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 -- start start 2024-08-19T01:39:07.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a71970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a73d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41aaee0 con 0x559b56a6a910 2024-08-19T01:39:07.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41acab0 con 0x559b56a69ff0 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.718+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41ab5b0 con 0x559b56a751d0 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a73800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41274/0 (socket says 172.21.15.103:41274) 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 -- 172.21.15.103:0/3420953515 learned_addr learned my addr 172.21.15.103:0/3420953515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc9464640 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a71970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a73d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 -- 172.21.15.103:0/3420953515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 msgr2=0x559b56a73d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a73d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 -- 172.21.15.103:0/3420953515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 msgr2=0x559b56a71970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a71970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 -- 172.21.15.103:0/3420953515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b56a70600 con 0x559b56a6a910 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a73800 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009850 tx=0x7f3fb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbec9084e05d82c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3faeffd640 1 -- 172.21.15.103:0/3420953515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4035070 con 0x559b56a6a910 2024-08-19T01:39:07.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3faeffd640 1 -- 172.21.15.103:0/3420953515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb4004950 con 0x559b56a6a910 2024-08-19T01:39:07.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.719+0000 7f3faeffd640 1 -- 172.21.15.103:0/3420953515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4045480 con 0x559b56a6a910 2024-08-19T01:39:07.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 -- 172.21.15.103:0/3420953515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 msgr2=0x559b56a73800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a73800 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009850 tx=0x7f3fb4000bc0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 -- 172.21.15.103:0/3420953515 shutdown_connections 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a751d0 0x559b56a73d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a73800 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/3420953515 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a69ff0 0x559b56a71970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 -- 172.21.15.103:0/3420953515 >> 172.21.15.103:0/3420953515 conn(0x559b56a66580 msgr2=0x559b56a66960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 -- 172.21.15.103:0/3420953515 shutdown_connections 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.720+0000 7f3fcdac6740 1 -- 172.21.15.103:0/3420953515 wait complete. 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 -- start start 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 Processor -- start 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a69ff0 0x559b56a7d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a78130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a751d0 0x559b56a78670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41ac380 con 0x559b56a6a910 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41abc80 con 0x559b56a751d0 2024-08-19T01:39:07.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41a8ee0 con 0x559b56a69ff0 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fcdac6740 10 client.0 _ll_register_callbacks cb 0x7f3fb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.721+0000 7f3fc8c63640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a78130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a78130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41282/0 (socket says 172.21.15.103:41282) 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 -- 172.21.15.103:0/2409622833 learned_addr learned my addr 172.21.15.103:0/2409622833 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc9c65640 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a751d0 0x559b56a78670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc9464640 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a69ff0 0x559b56a7d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 -- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a69ff0 msgr2=0x559b56a7d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a69ff0 0x559b56a7d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 -- 172.21.15.103:0/2409622833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a751d0 msgr2=0x559b56a78670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a751d0 0x559b56a78670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 -- 172.21.15.103:0/2409622833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b56a78ea0 con 0x559b56a6a910 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fc8c63640 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a78130 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40096c0 tx=0x7f3fb4005010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f6c4ccf60a7379 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3facff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:07.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3facff9640 1 -- 172.21.15.103:0/2409622833 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4035070 con 0x559b56a6a910 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3facff9640 1 -- 172.21.15.103:0/2409622833 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3fb4005410 con 0x559b56a6a910 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.722+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb4004950 con 0x559b56a6a910 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3facff9640 1 -- 172.21.15.103:0/2409622833 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4005850 con 0x559b56a6a910 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 10 client.16248 fetch_fsmap learned FSMap version 30 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b56aeea60 con 0x559b56a6a910 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3facff9640 1 -- 172.21.15.103:0/2409622833 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb4004950 con 0x559b56a6a910 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3facff9640 1 -- 172.21.15.103:0/2409622833 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3fb40634f0 con 0x559b56a6a910 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 10 client.16248 fetch_fsmap finished waiting for FSMap version 30 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 10 client.16248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 20 client.16248 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56aef090 0x559b56af14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 4 client.16248 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.723+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b569d74c0 con 0x559b56aef090 2024-08-19T01:39:07.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fc9464640 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56aef090 0x559b56af14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:07.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fc9464640 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56aef090 0x559b56af14b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bb6ec2f3dbd02b9 server_cookie=c3624213c23a2b7 in_seq=0 out_seq=0 2024-08-19T01:39:07.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3facff9640 10 client.16248 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3facff9640 1 -- 172.21.15.103:0/2409622833 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b569d74c0 con 0x559b56aef090 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3facff9640 10 client.16248 handle_command_reply: tid=0 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 client.16248 shutdown 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 msgr2=0x559b56a78130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a78130 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40096c0 tx=0x7f3fb4005010 comp rx=0 tx=0).stop 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 shutdown_connections 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559b56aef090 0x559b56af14b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559b56a751d0 0x559b56a78670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b56a6a910 0x559b56a78130 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 --2- 172.21.15.103:0/2409622833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b56a69ff0 0x559b56a7d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3facff9640 0 client.16248 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.724+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 >> 172.21.15.103:0/2409622833 conn(0x559b56a66580 msgr2=0x559b56a72070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 shutdown_connections 2024-08-19T01:39:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcdac6740 1 -- 172.21.15.103:0/2409622833 wait complete. 2024-08-19T01:39:07.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcdac6740 20 client.16248 trim_cache size 0 max 16384 2024-08-19T01:39:07.731 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3fb00a2c00 msgr2=0x7f3fb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3fb00a2c00 0x7f3fb00a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009fd0 tx=0x7f3fbc009840 comp rx=0 tx=0).stop 2024-08-19T01:39:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc4100b50 msgr2=0x7f3fc4198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc4100b50 0x7f3fc4198f90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9f0 tx=0x7f3fc000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 shutdown_connections 2024-08-19T01:39:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3fb00a2c00 0x7f3fb00a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3fc4102c20 0x7f3fc41a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fc41022a0 0x7f3fc41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 --2- 172.21.15.103:0/3631391184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fc4100b50 0x7f3fc4198f90 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.725+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 >> 172.21.15.103:0/3631391184 conn(0x7f3fc40fc500 msgr2=0x7f3fc40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.726+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 shutdown_connections 2024-08-19T01:39:07.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:07.726+0000 7f3fcb6c8640 1 -- 172.21.15.103:0/3631391184 wait complete. 2024-08-19T01:39:08.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:07 smithi146 ceph-mon[22098]: pgmap v439: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:39:08.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:07 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:08.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:07 smithi103 ceph-mon[18436]: pgmap v439: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:39:08.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:07 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:07 smithi190 ceph-mon[22032]: pgmap v439: 97 pgs: 97 active+clean; 3.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:39:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:07 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:08.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:39:08.860 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:09.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.369+0000 7f4c4f392640 1 -- 172.21.15.103:0/3958828285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481018b0 msgr2=0x7f4c48101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.369+0000 7f4c4f392640 1 --2- 172.21.15.103:0/3958828285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481018b0 0x7f4c48101d10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009930 tx=0x7f4c3802a420 comp rx=0 tx=0).stop 2024-08-19T01:39:09.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 -- 172.21.15.103:0/3958828285 shutdown_connections 2024-08-19T01:39:09.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 --2- 172.21.15.103:0/3958828285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c48102250 0x7f4c4810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 --2- 172.21.15.103:0/3958828285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481018b0 0x7f4c48101d10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 --2- 172.21.15.103:0/3958828285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c481078b0 0x7f4c48107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 -- 172.21.15.103:0/3958828285 >> 172.21.15.103:0/3958828285 conn(0x7f4c480fd610 msgr2=0x7f4c480ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:09.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 -- 172.21.15.103:0/3958828285 shutdown_connections 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 -- 172.21.15.103:0/3958828285 wait complete. 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.370+0000 7f4c4f392640 1 Processor -- start 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4f392640 1 -- start start 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4f392640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c481018b0 0x7f4c48198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4f392640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c48102250 0x7f4c48198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4f392640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481078b0 0x7f4c4819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4f392640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48111b60 con 0x7f4c481078b0 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4f392640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481119e0 con 0x7f4c48102250 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4f392640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48111ce0 con 0x7f4c481018b0 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481078b0 0x7f4c4819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481078b0 0x7f4c4819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41296/0 (socket says 172.21.15.103:41296) 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 -- 172.21.15.103:0/1101656056 learned_addr learned my addr 172.21.15.103:0/1101656056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:09.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c481018b0 0x7f4c48198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c48102250 0x7f4c48198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 -- 172.21.15.103:0/1101656056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c481018b0 msgr2=0x7f4c48198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c481018b0 0x7f4c48198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 -- 172.21.15.103:0/1101656056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c48102250 msgr2=0x7f4c48198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c48102250 0x7f4c48198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.371+0000 7f4c4d92f640 1 -- 172.21.15.103:0/1101656056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4819d0b0 con 0x7f4c481078b0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.372+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481078b0 0x7f4c4819c9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000eb10 tx=0x7f4c4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.372+0000 7f4c3e7fc640 1 -- 172.21.15.103:0/1101656056 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40004b20 con 0x7f4c481078b0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.372+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4819d3a0 con 0x7f4c481078b0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.372+0000 7f4c3e7fc640 1 -- 172.21.15.103:0/1101656056 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c40004cc0 con 0x7f4c481078b0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.372+0000 7f4c3e7fc640 1 -- 172.21.15.103:0/1101656056 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40010690 con 0x7f4c481078b0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.372+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c481a4c80 con 0x7f4c481078b0 2024-08-19T01:39:09.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.374+0000 7f4c3e7fc640 1 -- 172.21.15.103:0/1101656056 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4c40005020 con 0x7f4c481078b0 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c3e7fc640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4c340a2b80 0x7f4c340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 Processor -- start 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 -- start start 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4c340a2b80 0x7f4c340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c3e7fc640 1 -- 172.21.15.103:0/1101656056 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4c400cf5a0 con 0x7f4c481078b0 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762edf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762eec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559762e44a40 con 0x559762edeff0 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559762ddc9d0 con 0x559762edf910 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c51790740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559762e40a00 con 0x559762eea1d0 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c4d12e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762edf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c4d12e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762edf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41302/0 (socket says 172.21.15.103:41302) 2024-08-19T01:39:09.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.375+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1954881315 learned_addr learned my addr 172.21.15.103:0/1954881315 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762eec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1954881315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 msgr2=0x559762eec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762eec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1954881315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 msgr2=0x559762ee9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1954881315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559762de3650 con 0x559762edeff0 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762eec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4c340a2b80 0x7f4c340a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009fd0 tx=0x7f4c44009b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762edf3d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009df0 tx=0x7f4c4401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7205e69b7ab7f8a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c337fe640 1 -- 172.21.15.103:0/1954881315 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44026070 con 0x559762edeff0 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c337fe640 1 -- 172.21.15.103:0/1954881315 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44013170 con 0x559762edeff0 2024-08-19T01:39:09.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.376+0000 7f4c337fe640 1 -- 172.21.15.103:0/1954881315 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44010650 con 0x559762edeff0 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 -- 172.21.15.103:0/1954881315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 msgr2=0x559762edf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762edf3d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009df0 tx=0x7f4c4401d000 comp rx=0 tx=0).stop 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 -- 172.21.15.103:0/1954881315 shutdown_connections 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762eec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 --2- 172.21.15.103:0/1954881315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762edf3d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 -- 172.21.15.103:0/1954881315 >> 172.21.15.103:0/1954881315 conn(0x559762edb580 msgr2=0x559762edb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 -- 172.21.15.103:0/1954881315 shutdown_connections 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.377+0000 7f4c51790740 1 -- 172.21.15.103:0/1954881315 wait complete. 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 -- start start 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 Processor -- start 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762ee7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762ee7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559762ef11b0 con 0x559762edeff0 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559762dffb20 con 0x559762edf910 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559762ef1330 con 0x559762eea1d0 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c51790740 10 client.0 _ll_register_callbacks cb 0x559762dfdbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c4d12e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762ee7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c4d12e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762ee7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41304/0 (socket says 172.21.15.103:41304) 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.378+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1303113001 learned_addr learned my addr 172.21.15.103:0/1303113001 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762ee7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1303113001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 msgr2=0x559762ee7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762ee7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 msgr2=0x559762ee7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4d12e640 1 -- 172.21.15.103:0/1303113001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559762bb63c0 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762ee7110 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4c44024040 tx=0x7f4c44005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c450ce881cc91d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c30ff9640 1 -- 172.21.15.103:0/1303113001 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44026070 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c30ff9640 1 -- 172.21.15.103:0/1303113001 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4c44009040 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.379+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c44013170 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c30ff9640 1 -- 172.21.15.103:0/1303113001 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4c4400e990 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 10 client.16260 fetch_fsmap learned FSMap version 30 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559762efce10 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c30ff9640 1 -- 172.21.15.103:0/1303113001 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44013170 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c30ff9640 1 -- 172.21.15.103:0/1303113001 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4c44020050 con 0x559762edeff0 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 10 client.16260 fetch_fsmap finished waiting for FSMap version 30 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 20 client.16260 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:09.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762efe270 0x559762f00660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 4 client.16260 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559762e51fb0 con 0x559762efe270 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.380+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762efe270 0x559762f00660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.381+0000 7f4c30ff9640 10 client.16260 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.381+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762efe270 0x559762f00660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f195817f56a2dad server_cookie=8fa25e199b189ebf in_seq=0 out_seq=0 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c30ff9640 1 -- 172.21.15.103:0/1303113001 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559762e51fb0 con 0x559762efe270 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c30ff9640 10 client.16260 handle_command_reply: tid=0 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 client.16260 shutdown 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 msgr2=0x559762ee7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762ee7110 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4c44024040 tx=0x7f4c44005c50 comp rx=0 tx=0).stop 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 shutdown_connections 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762efe270 0x559762f00660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762eea1d0 0x559762ee7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762ee7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 --2- 172.21.15.103:0/1303113001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edeff0 0x559762ee7110 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c30ff9640 0 client.16260 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:09.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 >> 172.21.15.103:0/1303113001 conn(0x559762edb580 msgr2=0x559762ee1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:09.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 shutdown_connections 2024-08-19T01:39:09.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 1 -- 172.21.15.103:0/1303113001 wait complete. 2024-08-19T01:39:09.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.382+0000 7f4c51790740 20 client.16260 trim_cache size 0 max 16384 2024-08-19T01:39:09.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.397+0000 7f4c51790740 1 Processor -- start 2024-08-19T01:39:09.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.397+0000 7f4c51790740 1 -- start start 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c51790740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ee6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c51790740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762f05e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c51790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762eea1d0 0x559762f063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c51790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481aaf20 con 0x559762eea1d0 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c51790740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481acaf0 con 0x559762edf910 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c51790740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481ab5f0 con 0x559762edeff0 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d92f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762eea1d0 0x559762f063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d92f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762eea1d0 0x559762f063d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41316/0 (socket says 172.21.15.103:41316) 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d92f640 1 -- 172.21.15.103:0/952429879 learned_addr learned my addr 172.21.15.103:0/952429879 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ee6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762f05e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d12e640 1 -- 172.21.15.103:0/952429879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 msgr2=0x559762f05e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762f05e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d12e640 1 -- 172.21.15.103:0/952429879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762eea1d0 msgr2=0x559762f063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762eea1d0 0x559762f063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d12e640 1 -- 172.21.15.103:0/952429879 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559762ee9bc0 con 0x559762edeff0 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762eea1d0 0x559762f063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.398+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ee6a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4c44004aa0 tx=0x7f4c44009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9fae4cba59d94a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.399+0000 7f4c32ffd640 1 -- 172.21.15.103:0/952429879 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44026070 con 0x559762edeff0 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.399+0000 7f4c32ffd640 1 -- 172.21.15.103:0/952429879 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c4402e070 con 0x559762edeff0 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.399+0000 7f4c32ffd640 1 -- 172.21.15.103:0/952429879 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44004d30 con 0x559762edeff0 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.399+0000 7f4c51790740 1 -- 172.21.15.103:0/952429879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 msgr2=0x559762ee6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.399+0000 7f4c51790740 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ee6a10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4c44004aa0 tx=0x7f4c44009620 comp rx=0 tx=0).stop 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.399+0000 7f4c51790740 1 -- 172.21.15.103:0/952429879 shutdown_connections 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.399+0000 7f4c51790740 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762eea1d0 0x559762f063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762edf910 0x559762f05e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 --2- 172.21.15.103:0/952429879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ee6a10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 -- 172.21.15.103:0/952429879 >> 172.21.15.103:0/952429879 conn(0x559762edb580 msgr2=0x559762ee9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 -- 172.21.15.103:0/952429879 shutdown_connections 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 -- 172.21.15.103:0/952429879 wait complete. 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 -- start start 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 Processor -- start 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ef6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.400+0000 7f4c51790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edf910 0x559762ee3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c51790740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762eea1d0 0x559762ee4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c51790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481ac3c0 con 0x559762edf910 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c51790740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481abcc0 con 0x559762eea1d0 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c51790740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481a8f20 con 0x559762edeff0 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c51790740 10 client.0 _ll_register_callbacks cb 0x7f4c44024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4c92d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edf910 0x559762ee3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4c92d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edf910 0x559762ee3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41318/0 (socket says 172.21.15.103:41318) 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4c92d640 1 -- 172.21.15.103:0/169137682 learned_addr learned my addr 172.21.15.103:0/169137682 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ef6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d92f640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762eea1d0 0x559762ee4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d12e640 1 -- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762eea1d0 msgr2=0x559762ee4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762eea1d0 0x559762ee4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d12e640 1 -- 172.21.15.103:0/169137682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edf910 msgr2=0x559762ee3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edf910 0x559762ee3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d12e640 1 -- 172.21.15.103:0/169137682 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559762ee4590 con 0x559762edeff0 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edf910 0x559762ee3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c4d12e640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ef6980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c44022000 tx=0x7f4c4400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46782721dafd7c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.401+0000 7f4c30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:39:09.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.402+0000 7f4c30ff9640 1 -- 172.21.15.103:0/169137682 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44026070 con 0x559762edeff0 2024-08-19T01:39:09.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.402+0000 7f4c30ff9640 1 -- 172.21.15.103:0/169137682 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4c44013460 con 0x559762edeff0 2024-08-19T01:39:09.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.402+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4402e070 con 0x559762edeff0 2024-08-19T01:39:09.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.402+0000 7f4c30ff9640 1 -- 172.21.15.103:0/169137682 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4c44033450 con 0x559762edeff0 2024-08-19T01:39:09.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.402+0000 7f4c51790740 10 client.25678 fetch_fsmap learned FSMap version 30 2024-08-19T01:39:09.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.402+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559762f68af0 con 0x559762edeff0 2024-08-19T01:39:09.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.402+0000 7f4c30ff9640 1 -- 172.21.15.103:0/169137682 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c4402e070 con 0x559762edeff0 2024-08-19T01:39:09.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c30ff9640 1 -- 172.21.15.103:0/169137682 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4c4403b030 con 0x559762edeff0 2024-08-19T01:39:09.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c51790740 10 client.25678 fetch_fsmap finished waiting for FSMap version 30 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c51790740 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c51790740 20 client.25678 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c51790740 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762f69170 0x559762f6b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c51790740 4 client.25678 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559762e4c4c0 con 0x559762f69170 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762f69170 0x559762f6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c4c92d640 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762f69170 0x559762f6b590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=563d6e125d16a7e8 server_cookie=6b2025406a5c7c46 in_seq=0 out_seq=0 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.403+0000 7f4c30ff9640 10 client.25678 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c30ff9640 1 -- 172.21.15.103:0/169137682 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559762e4c4c0 con 0x559762f69170 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c30ff9640 10 client.25678 handle_command_reply: tid=0 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 client.25678 shutdown 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 msgr2=0x559762ef6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ef6980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c44022000 tx=0x7f4c4400f790 comp rx=0 tx=0).stop 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 shutdown_connections 2024-08-19T01:39:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559762f69170 0x559762f6b590 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559762eea1d0 0x559762ee4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559762edf910 0x559762ee3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c30ff9640 0 client.25678 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 --2- 172.21.15.103:0/169137682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559762edeff0 0x559762ef6980 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 >> 172.21.15.103:0/169137682 conn(0x559762edb580 msgr2=0x559762ee70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 shutdown_connections 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 1 -- 172.21.15.103:0/169137682 wait complete. 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.404+0000 7f4c51790740 20 client.25678 trim_cache size 0 max 16384 2024-08-19T01:39:09.410 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4c340a2b80 msgr2=0x7f4c340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4c340a2b80 0x7f4c340a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009fd0 tx=0x7f4c44009b40 comp rx=0 tx=0).stop 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481078b0 msgr2=0x7f4c4819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481078b0 0x7f4c4819c9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000eb10 tx=0x7f4c4000c2d0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 shutdown_connections 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4c340a2b80 0x7f4c340a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c481078b0 0x7f4c4819c9d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c48102250 0x7f4c48198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 --2- 172.21.15.103:0/1101656056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c481018b0 0x7f4c48198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:09.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 >> 172.21.15.103:0/1101656056 conn(0x7f4c480fd610 msgr2=0x7f4c480fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:09.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 shutdown_connections 2024-08-19T01:39:09.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:09.405+0000 7f4c4f392640 1 -- 172.21.15.103:0/1101656056 wait complete. 2024-08-19T01:39:10.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:09 smithi103 ceph-mon[18436]: pgmap v440: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:39:10.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:09 smithi146 ceph-mon[22098]: pgmap v440: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:39:10.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:09 smithi190 ceph-mon[22032]: pgmap v440: 97 pgs: 97 active+clean; 4.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:39:11.263 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:39:11.264 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:39:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 -- 172.21.15.103:0/3438022292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2290102bd0 msgr2=0x7f2290103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 --2- 172.21.15.103:0/3438022292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2290102bd0 0x7f2290103050 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2284009900 tx=0x7f228402a410 comp rx=0 tx=0).stop 2024-08-19T01:39:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 -- 172.21.15.103:0/3438022292 shutdown_connections 2024-08-19T01:39:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 --2- 172.21.15.103:0/3438022292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290103590 0x7f2290109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 --2- 172.21.15.103:0/3438022292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2290102bd0 0x7f2290103050 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 --2- 172.21.15.103:0/3438022292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22900ffb60 0x7f22900fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 -- 172.21.15.103:0/3438022292 >> 172.21.15.103:0/3438022292 conn(0x7f22900fb660 msgr2=0x7f22900fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 -- 172.21.15.103:0/3438022292 shutdown_connections 2024-08-19T01:39:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.517+0000 7f22948df640 1 -- 172.21.15.103:0/3438022292 wait complete. 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 Processor -- start 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 -- start start 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22900ffb60 0x7f2290199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290102bd0 0x7f229019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2290103590 0x7f22901a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f229010d3c0 con 0x7f22900ffb60 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f229010d240 con 0x7f2290103590 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f22948df640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f229010d540 con 0x7f2290102bd0 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.518+0000 7f228e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22900ffb60 0x7f2290199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22900ffb60 0x7f2290199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41328/0 (socket says 172.21.15.103:41328) 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 -- 172.21.15.103:0/845143569 learned_addr learned my addr 172.21.15.103:0/845143569 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 -- 172.21.15.103:0/845143569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290102bd0 msgr2=0x7f229019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290102bd0 0x7f229019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 -- 172.21.15.103:0/845143569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2290103590 msgr2=0x7f22901a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2290103590 0x7f22901a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 -- 172.21.15.103:0/845143569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22901a1840 con 0x7f22900ffb60 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f228e59c640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22900ffb60 0x7f2290199c30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f228000d950 tx=0x7f228000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f227f7fe640 1 -- 172.21.15.103:0/845143569 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280017020 con 0x7f22900ffb60 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.519+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22901a1b30 con 0x7f22900ffb60 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.520+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22901a20d0 con 0x7f22900ffb60 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.520+0000 7f227f7fe640 1 -- 172.21.15.103:0/845143569 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2280009c30 con 0x7f22900ffb60 2024-08-19T01:39:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.520+0000 7f227f7fe640 1 -- 172.21.15.103:0/845143569 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228000f6b0 con 0x7f22900ffb60 2024-08-19T01:39:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.522+0000 7f227f7fe640 1 -- 172.21.15.103:0/845143569 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f228001a020 con 0x7f22900ffb60 2024-08-19T01:39:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 Processor -- start 2024-08-19T01:39:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 -- start start 2024-08-19T01:39:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895b680 0x5611d895ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895bfc0 0x5611d8960a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d8961050 0x5611d8963510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5611d88be9d0 con 0x5611d8961050 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5611d8866820 con 0x5611d895b680 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f2297698740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5611d88ba990 con 0x5611d895bfc0 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.523+0000 7f228dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d8961050 0x5611d8963510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d8961050 0x5611d8963510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41336/0 (socket says 172.21.15.103:41336) 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 -- 172.21.15.103:0/3614516635 learned_addr learned my addr 172.21.15.103:0/3614516635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f227f7fe640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f22780a2c30 0x7f22780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228e59c640 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895bfc0 0x5611d8960a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 -- 172.21.15.103:0/3614516635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895bfc0 msgr2=0x5611d8960a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228ed9d640 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895b680 0x5611d895ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895bfc0 0x5611d8960a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 -- 172.21.15.103:0/3614516635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895b680 msgr2=0x5611d895ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895b680 0x5611d895ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 -- 172.21.15.103:0/3614516635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d85d2210 con 0x5611d8961050 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228ed9d640 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895b680 0x5611d895ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f227f7fe640 1 -- 172.21.15.103:0/845143569 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f22800cee20 con 0x7f22900ffb60 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228ed9d640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f22780a2c30 0x7f22780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.524+0000 7f228dd9b640 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d8961050 0x5611d8963510 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2284032c30 tx=0x7f2284000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba776cdb033c54c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.525+0000 7f227cff9640 1 -- 172.21.15.103:0/3614516635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284040070 con 0x5611d8961050 2024-08-19T01:39:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.525+0000 7f227cff9640 1 -- 172.21.15.103:0/3614516635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2284032e10 con 0x5611d8961050 2024-08-19T01:39:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.525+0000 7f227cff9640 1 -- 172.21.15.103:0/3614516635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284035040 con 0x5611d8961050 2024-08-19T01:39:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.525+0000 7f228ed9d640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f22780a2c30 0x7f22780a50f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2288005c40 tx=0x7f2288004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.526+0000 7f2297698740 1 -- 172.21.15.103:0/3614516635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d8961050 msgr2=0x5611d8963510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.526+0000 7f2297698740 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d8961050 0x5611d8963510 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2284032c30 tx=0x7f2284000bc0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.526+0000 7f2297698740 1 -- 172.21.15.103:0/3614516635 shutdown_connections 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.526+0000 7f2297698740 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d8961050 0x5611d8963510 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.526+0000 7f2297698740 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895bfc0 0x5611d8960a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.526+0000 7f2297698740 1 --2- 172.21.15.103:0/3614516635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895b680 0x5611d895ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.526+0000 7f2297698740 1 -- 172.21.15.103:0/3614516635 >> 172.21.15.103:0/3614516635 conn(0x5611d8956cf0 msgr2=0x5611d89570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 -- 172.21.15.103:0/3614516635 shutdown_connections 2024-08-19T01:39:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 -- 172.21.15.103:0/3614516635 wait complete. 2024-08-19T01:39:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 -- start start 2024-08-19T01:39:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 Processor -- start 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895b680 0x5611d89659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8965f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d896af60 0x5611d896d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5611d8963fc0 con 0x5611d895b680 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5611d887a520 con 0x5611d895bfc0 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.527+0000 7f2297698740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5611d8964140 con 0x5611d896af60 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f2297698740 10 client.0 _ll_register_callbacks cb 0x5611d85d1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8965f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8965f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:56098/0 (socket says 172.21.15.103:56098) 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 -- 172.21.15.103:0/551524855 learned_addr learned my addr 172.21.15.103:0/551524855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228ed9d640 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d896af60 0x5611d896d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 -- 172.21.15.103:0/551524855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d896af60 msgr2=0x5611d896d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d896af60 0x5611d896d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 -- 172.21.15.103:0/551524855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895b680 msgr2=0x5611d89659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895b680 0x5611d89659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.528+0000 7f228dd9b640 1 -- 172.21.15.103:0/551524855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d869e620 con 0x5611d895bfc0 2024-08-19T01:39:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f228dd9b640 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8965f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2284009900 tx=0x7f22840317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a111057844c3ea7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:39:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2275ffb640 1 -- 172.21.15.103:0/551524855 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284052060 con 0x5611d895bfc0 2024-08-19T01:39:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2284032e10 con 0x5611d895bfc0 2024-08-19T01:39:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2275ffb640 1 -- 172.21.15.103:0/551524855 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f2284045860 con 0x5611d895bfc0 2024-08-19T01:39:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2275ffb640 1 -- 172.21.15.103:0/551524855 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2284035040 con 0x5611d895bfc0 2024-08-19T01:39:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2297698740 10 client.25712 fetch_fsmap learned FSMap version 30 2024-08-19T01:39:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611d89700b0 con 0x5611d895bfc0 2024-08-19T01:39:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.529+0000 7f2275ffb640 1 -- 172.21.15.103:0/551524855 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2284032e10 con 0x5611d895bfc0 2024-08-19T01:39:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2275ffb640 1 -- 172.21.15.103:0/551524855 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f228403a960 con 0x5611d895bfc0 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2297698740 10 client.25712 fetch_fsmap finished waiting for FSMap version 30 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2297698740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2297698740 20 client.25712 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2297698740 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d8971400 0x5611d89737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2297698740 4 client.25712 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611d88cc100 con 0x5611d8971400 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f228e59c640 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d8971400 0x5611d89737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f2275ffb640 10 client.25712 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.530+0000 7f228e59c640 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d8971400 0x5611d89737f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0dd6d07cecf0cdd server_cookie=3b1f6e5a2c8ef59b in_seq=0 out_seq=0 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2275ffb640 1 -- 172.21.15.103:0/551524855 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611d88cc100 con 0x5611d8971400 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2275ffb640 10 client.25712 handle_command_reply: tid=0 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 client.25712 shutdown 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 msgr2=0x5611d8965f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8965f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2284009900 tx=0x7f22840317b0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 shutdown_connections 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d8971400 0x5611d89737f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d896af60 0x5611d896d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8965f30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2275ffb640 0 client.25712 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 --2- 172.21.15.103:0/551524855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895b680 0x5611d89659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 >> 172.21.15.103:0/551524855 conn(0x5611d8956cf0 msgr2=0x5611d8958500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 shutdown_connections 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.531+0000 7f2297698740 1 -- 172.21.15.103:0/551524855 wait complete. 2024-08-19T01:39:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.532+0000 7f2297698740 20 client.25712 trim_cache size 0 max 16384 2024-08-19T01:39:11.555 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 Processor -- start 2024-08-19T01:39:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 -- start start 2024-08-19T01:39:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d895c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895bfc0 0x5611d8955d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d896af60 0x5611d89580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22901aad60 con 0x5611d895bfc0 2024-08-19T01:39:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22901ac930 con 0x5611d896af60 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f2297698740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22901ab430 con 0x5611d895b680 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f228dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895bfc0 0x5611d8955d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f228dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895bfc0 0x5611d8955d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41362/0 (socket says 172.21.15.103:41362) 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f228dd9b640 1 -- 172.21.15.103:0/4209918533 learned_addr learned my addr 172.21.15.103:0/4209918533 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f228e59c640 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d895c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.554+0000 7f228ed9d640 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d896af60 0x5611d89580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f228dd9b640 1 -- 172.21.15.103:0/4209918533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 msgr2=0x5611d895c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f228dd9b640 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d895c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f228dd9b640 1 -- 172.21.15.103:0/4209918533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d896af60 msgr2=0x5611d89580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f228dd9b640 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d896af60 0x5611d89580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f228dd9b640 1 -- 172.21.15.103:0/4209918533 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d8973b60 con 0x5611d895bfc0 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f228dd9b640 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895bfc0 0x5611d8955d90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f22840314e0 tx=0x7f2284000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c209f9bdb1b180cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f2277fff640 1 -- 172.21.15.103:0/4209918533 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228405d030 con 0x5611d895bfc0 2024-08-19T01:39:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f2277fff640 1 -- 172.21.15.103:0/4209918533 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2284032e10 con 0x5611d895bfc0 2024-08-19T01:39:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.555+0000 7f2277fff640 1 -- 172.21.15.103:0/4209918533 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284052070 con 0x5611d895bfc0 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 -- 172.21.15.103:0/4209918533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895bfc0 msgr2=0x5611d8955d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895bfc0 0x5611d8955d90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f22840314e0 tx=0x7f2284000bc0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 -- 172.21.15.103:0/4209918533 shutdown_connections 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d896af60 0x5611d89580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d895bfc0 0x5611d8955d90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 --2- 172.21.15.103:0/4209918533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d895c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 -- 172.21.15.103:0/4209918533 >> 172.21.15.103:0/4209918533 conn(0x5611d8956cf0 msgr2=0x5611d8955680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 -- 172.21.15.103:0/4209918533 shutdown_connections 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 -- 172.21.15.103:0/4209918533 wait complete. 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.556+0000 7f2297698740 1 -- start start 2024-08-19T01:39:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 1 Processor -- start 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d89826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8982c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d896af60 0x5611d89d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22901ac200 con 0x5611d896af60 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22901abb00 con 0x5611d895bfc0 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f22901a8d40 con 0x5611d895b680 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d896af60 0x5611d89d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d896af60 0x5611d89d9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41374/0 (socket says 172.21.15.103:41374) 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 -- 172.21.15.103:0/1116445257 learned_addr learned my addr 172.21.15.103:0/1116445257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228e59c640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d89826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f2297698740 10 client.0 _ll_register_callbacks cb 0x7f228009b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228dd9b640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8982c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 -- 172.21.15.103:0/1116445257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 msgr2=0x5611d89826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d89826f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 -- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 msgr2=0x5611d8982c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8982c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228ed9d640 1 -- 172.21.15.103:0/1116445257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d89da4e0 con 0x5611d896af60 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.557+0000 7f228dd9b640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8982c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f228ed9d640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d896af60 0x5611d89d9ed0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f228801e7f0 tx=0x7f2288009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64377de0dd75f36f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2275ffb640 1 -- 172.21.15.103:0/1116445257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288011840 con 0x5611d896af60 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2275ffb640 1 -- 172.21.15.103:0/1116445257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f22880119e0 con 0x5611d896af60 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2284032e10 con 0x5611d896af60 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2275ffb640 1 -- 172.21.15.103:0/1116445257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2288011be0 con 0x5611d896af60 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2297698740 10 client.16281 fetch_fsmap learned FSMap version 30 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611d89db8c0 con 0x5611d896af60 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2275ffb640 1 -- 172.21.15.103:0/1116445257 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2284032e10 con 0x5611d896af60 2024-08-19T01:39:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2275ffb640 1 -- 172.21.15.103:0/1116445257 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f2288028420 con 0x5611d896af60 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2297698740 10 client.16281 fetch_fsmap finished waiting for FSMap version 30 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.558+0000 7f2297698740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.559+0000 7f2297698740 20 client.16281 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.559+0000 7f2297698740 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d89dc320 0x5611d89de740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.559+0000 7f2297698740 4 client.16281 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.559+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611d88c6450 con 0x5611d89dc320 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.559+0000 7f228e59c640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d89dc320 0x5611d89de740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.559+0000 7f228e59c640 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d89dc320 0x5611d89de740 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09c47c185e88b60 server_cookie=1bdeef19a2cbc9c2 in_seq=0 out_seq=0 2024-08-19T01:39:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.559+0000 7f2275ffb640 10 client.16281 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.566+0000 7f2275ffb640 1 -- 172.21.15.103:0/1116445257 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611d88c6450 con 0x5611d89dc320 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.566+0000 7f2275ffb640 10 client.16281 handle_command_reply: tid=0 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.566+0000 7f2297698740 1 client.16281 shutdown 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d896af60 msgr2=0x5611d89d9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d896af60 0x5611d89d9ed0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f228801e7f0 tx=0x7f2288009260 comp rx=0 tx=0).stop 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 shutdown_connections 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5611d89dc320 0x5611d89de740 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5611d896af60 0x5611d89d9ed0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5611d895bfc0 0x5611d8982c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2275ffb640 0 client.16281 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 --2- 172.21.15.103:0/1116445257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5611d895b680 0x5611d89826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 >> 172.21.15.103:0/1116445257 conn(0x5611d8956cf0 msgr2=0x5611d8955ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 shutdown_connections 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 1 -- 172.21.15.103:0/1116445257 wait complete. 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.567+0000 7f2297698740 20 client.16281 trim_cache size 0 max 16384 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:39:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:39:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f22780a2c30 msgr2=0x7f22780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f22780a2c30 0x7f22780a50f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2288005c40 tx=0x7f2288004a40 comp rx=0 tx=0).stop 2024-08-19T01:39:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22900ffb60 msgr2=0x7f2290199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22900ffb60 0x7f2290199c30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f228000d950 tx=0x7f228000de10 comp rx=0 tx=0).stop 2024-08-19T01:39:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 shutdown_connections 2024-08-19T01:39:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f22780a2c30 0x7f22780a50f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2290103590 0x7f22901a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290102bd0 0x7f229019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 --2- 172.21.15.103:0/845143569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22900ffb60 0x7f2290199c30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 >> 172.21.15.103:0/845143569 conn(0x7f22900fb660 msgr2=0x7f22901009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 shutdown_connections 2024-08-19T01:39:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:11.568+0000 7f22948df640 1 -- 172.21.15.103:0/845143569 wait complete. 2024-08-19T01:39:12.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:11 smithi103 ceph-mon[18436]: pgmap v441: 97 pgs: 97 active+clean; 4.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:39:12.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 -- 172.21.15.103:0/3276725378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b810c020 msgr2=0x7fb3b810e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/3276725378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b810c020 0x7fb3b810e4c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009ca0 tx=0x7fb3b002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 -- 172.21.15.103:0/3276725378 shutdown_connections 2024-08-19T01:39:12.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/3276725378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b810c020 0x7fb3b810e4c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/3276725378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3b81034d0 0x7fb3b810b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/3276725378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b8102b20 0x7fb3b8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 -- 172.21.15.103:0/3276725378 >> 172.21.15.103:0/3276725378 conn(0x7fb3b80fc500 msgr2=0x7fb3b80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 -- 172.21.15.103:0/3276725378 shutdown_connections 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 -- 172.21.15.103:0/3276725378 wait complete. 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.114+0000 7fb3bca6a640 1 Processor -- start 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3bca6a640 1 -- start start 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3bca6a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b8102b20 0x7fb3b81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3bca6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b81034d0 0x7fb3b8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3bca6a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3b810c020 0x7fb3b819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3bca6a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8111cd0 con 0x7fb3b81034d0 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3bca6a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8111b50 con 0x7fb3b8102b20 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3bca6a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8111e50 con 0x7fb3b810c020 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b81034d0 0x7fb3b8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b81034d0 0x7fb3b8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53004/0 (socket says 172.21.15.103:53004) 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 -- 172.21.15.103:0/2074277547 learned_addr learned my addr 172.21.15.103:0/2074277547 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:12.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3b810c020 0x7fb3b819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3b659c640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b8102b20 0x7fb3b81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 -- 172.21.15.103:0/2074277547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3b810c020 msgr2=0x7fb3b819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3b810c020 0x7fb3b819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 -- 172.21.15.103:0/2074277547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b8102b20 msgr2=0x7fb3b81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b8102b20 0x7fb3b81981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3affff640 1 -- 172.21.15.103:0/2074277547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b819d180 con 0x7fb3b81034d0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.115+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3b810c020 0x7fb3b819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.116+0000 7fb3affff640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b81034d0 0x7fb3b8198710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e9f0 tx=0x7fb3a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.116+0000 7fb3af7fe640 1 -- 172.21.15.103:0/2074277547 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000ccc0 con 0x7fb3b81034d0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.116+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b819d470 con 0x7fb3b81034d0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.116+0000 7fb3af7fe640 1 -- 172.21.15.103:0/2074277547 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a00108e0 con 0x7fb3b81034d0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.116+0000 7fb3af7fe640 1 -- 172.21.15.103:0/2074277547 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00187e0 con 0x7fb3b81034d0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.116+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b81a4d50 con 0x7fb3b81034d0 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 Processor -- start 2024-08-19T01:39:12.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 -- start start 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3af7fe640 1 -- 172.21.15.103:0/2074277547 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb3a0010a90 con 0x7fb3b81034d0 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561cc15be40 con 0x5561cc1a33e0 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5561cc0fd7f0 con 0x5561cc198200 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.119+0000 7fb3bee68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561cc0a4fb0 con 0x5561cc198b20 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1a2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:48660/0 (socket says 172.21.15.103:48660) 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 -- 172.21.15.103:0/3094212843 learned_addr learned my addr 172.21.15.103:0/3094212843 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 -- 172.21.15.103:0/3094212843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 msgr2=0x5561cc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 -- 172.21.15.103:0/3094212843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 msgr2=0x5561cc1a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 -- 172.21.15.103:0/3094212843 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cbe6f6a0 con 0x5561cc198b20 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3affff640 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3af7fe640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb3a40a2c00 0x7fb3a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b659c640 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1a2ea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800bd10 tx=0x7fb3a80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3159dd5b404084df server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3af7fe640 1 -- 172.21.15.103:0/2074277547 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fb3a00cf230 con 0x7fb3b81034d0 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb3a40a2c00 0x7fb3a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.120+0000 7fb3acff9640 1 -- 172.21.15.103:0/3094212843 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8004030 con 0x5561cc198b20 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3acff9640 1 -- 172.21.15.103:0/3094212843 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a80035f0 con 0x5561cc198b20 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3acff9640 1 -- 172.21.15.103:0/3094212843 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80117b0 con 0x5561cc198b20 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3bee68740 1 -- 172.21.15.103:0/3094212843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 msgr2=0x5561cc1a2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb3a40a2c00 0x7fb3a40a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0004af0 tx=0x7fb3b0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1a2ea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800bd10 tx=0x7fb3a80107a0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3bee68740 1 -- 172.21.15.103:0/3094212843 shutdown_connections 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1a2ea0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3094212843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.121+0000 7fb3bee68740 1 -- 172.21.15.103:0/3094212843 >> 172.21.15.103:0/3094212843 conn(0x5561cc194790 msgr2=0x5561cc194b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 -- 172.21.15.103:0/3094212843 shutdown_connections 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 -- 172.21.15.103:0/3094212843 wait complete. 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 -- start start 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 Processor -- start 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc1a33e0 0x5561cc1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561cc0b8cb0 con 0x5561cc198b20 2024-08-19T01:39:12.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5561cc0f97b0 con 0x5561cc198200 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5561cc1a5fb0 con 0x5561cc1a33e0 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3bee68740 10 client.0 _ll_register_callbacks cb 0x5561cbe0cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53020/0 (socket says 172.21.15.103:53020) 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.122+0000 7fb3affff640 1 -- 172.21.15.103:0/3156638789 learned_addr learned my addr 172.21.15.103:0/3156638789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3b659c640 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc1a33e0 0x5561cc1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3affff640 1 -- 172.21.15.103:0/3156638789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc1a33e0 msgr2=0x5561cc1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3affff640 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc1a33e0 0x5561cc1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3affff640 1 -- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 msgr2=0x5561cc1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3affff640 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3affff640 1 -- 172.21.15.103:0/3156638789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cc0b7bc0 con 0x5561cc198b20 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3affff640 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a0850 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000c6a0 tx=0x7fb3a0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef3936776d7ea00 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:12.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb39dffb640 1 -- 172.21.15.103:0/3156638789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a009b4d0 con 0x5561cc198b20 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb39dffb640 1 -- 172.21.15.103:0/3156638789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fb3a0010630 con 0x5561cc198b20 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a80035f0 con 0x5561cc198b20 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb39dffb640 1 -- 172.21.15.103:0/3156638789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3a0010de0 con 0x5561cc198b20 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3bee68740 10 client.16293 fetch_fsmap learned FSMap version 31 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.123+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561cc1b1f10 con 0x5561cc198b20 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb39dffb640 1 -- 172.21.15.103:0/3156638789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a80035f0 con 0x5561cc198b20 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb39dffb640 1 -- 172.21.15.103:0/3156638789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb3a0097a60 con 0x5561cc198b20 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb3bee68740 10 client.16293 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb3bee68740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb3bee68740 20 client.16293 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc1b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb3bee68740 4 client.16293 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:12.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561cc10ad60 con 0x5561cc1b3420 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.124+0000 7fb3b659c640 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc1b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.125+0000 7fb3b659c640 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc1b58e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8508086990d5857 server_cookie=199f2d265c1e1c90 in_seq=0 out_seq=0 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.125+0000 7fb39dffb640 10 client.16293 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.125+0000 7fb39dffb640 1 -- 172.21.15.103:0/3156638789 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561cc10ad60 con 0x5561cc1b3420 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.125+0000 7fb39dffb640 10 client.16293 handle_command_reply: tid=0 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 client.16293 shutdown 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 msgr2=0x5561cc1a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a0850 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000c6a0 tx=0x7fb3a0009310 comp rx=0 tx=0).stop 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 shutdown_connections 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc1b58e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc1a33e0 0x5561cc1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb39dffb640 0 client.16293 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:12.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a0850 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 --2- 172.21.15.103:0/3156638789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 >> 172.21.15.103:0/3156638789 conn(0x5561cc194790 msgr2=0x5561cc19a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 shutdown_connections 2024-08-19T01:39:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 1 -- 172.21.15.103:0/3156638789 wait complete. 2024-08-19T01:39:12.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.126+0000 7fb3bee68740 20 client.16293 trim_cache size 0 max 16384 2024-08-19T01:39:12.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 Processor -- start 2024-08-19T01:39:12.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 -- start start 2024-08-19T01:39:12.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81ad110 con 0x5561cc1a33e0 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81aed40 con 0x5561cc198200 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3bee68740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81ad7e0 con 0x5561cc198b20 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3b6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3b6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53022/0 (socket says 172.21.15.103:53022) 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3b659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3b6d9d640 1 -- 172.21.15.103:0/2418065207 learned_addr learned my addr 172.21.15.103:0/2418065207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.145+0000 7fb3affff640 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3b659c640 1 -- 172.21.15.103:0/2418065207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 msgr2=0x5561cc1bb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3b659c640 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1bb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3b659c640 1 -- 172.21.15.103:0/2418065207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 msgr2=0x5561cc1930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3b659c640 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3b659c640 1 -- 172.21.15.103:0/2418065207 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cc199710 con 0x5561cc198200 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3affff640 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1bb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb3b659c640 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1bb1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800bce0 tx=0x7fb3a8003dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7c8e3f82cd65d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb39ffff640 1 -- 172.21.15.103:0/2418065207 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8005240 con 0x5561cc198200 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb39ffff640 1 -- 172.21.15.103:0/2418065207 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a80035f0 con 0x5561cc198200 2024-08-19T01:39:12.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.146+0000 7fb39ffff640 1 -- 172.21.15.103:0/2418065207 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80266a0 con 0x5561cc198200 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 -- 172.21.15.103:0/2418065207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 msgr2=0x5561cc1bb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1bb1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800bce0 tx=0x7fb3a8003dc0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 -- 172.21.15.103:0/2418065207 shutdown_connections 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc1a33e0 0x5561cc1930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198b20 0x5561cc1bb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 --2- 172.21.15.103:0/2418065207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc198200 0x5561cc1bb1b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 -- 172.21.15.103:0/2418065207 >> 172.21.15.103:0/2418065207 conn(0x5561cc194790 msgr2=0x5561cc19ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 -- 172.21.15.103:0/2418065207 shutdown_connections 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 -- 172.21.15.103:0/2418065207 wait complete. 2024-08-19T01:39:12.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 -- start start 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.147+0000 7fb3bee68740 1 Processor -- start 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198200 0x5561cc1a6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3bee68740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc1a33e0 0x5561cc21c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3bee68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81ae5b0 con 0x5561cc198b20 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53036/0 (socket says 172.21.15.103:53036) 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 -- 172.21.15.103:0/575056379 learned_addr learned my addr 172.21.15.103:0/575056379 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3b659c640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198200 0x5561cc1a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3b6d9d640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc1a33e0 0x5561cc21c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3bee68740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81adeb0 con 0x5561cc1a33e0 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81ab0f0 con 0x5561cc198200 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3bee68740 10 client.0 _ll_register_callbacks cb 0x7fb3a00096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:12.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 -- 172.21.15.103:0/575056379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198200 msgr2=0x5561cc1a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198200 0x5561cc1a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 -- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc1a33e0 msgr2=0x5561cc21c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc1a33e0 0x5561cc21c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3affff640 1 -- 172.21.15.103:0/575056379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cc21c6b0 con 0x5561cc198b20 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.148+0000 7fb3b659c640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198200 0x5561cc1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb3affff640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a6a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb3a009bb00 tx=0x7fb3a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59619b0cc8cce36 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb39dffb640 1 -- 172.21.15.103:0/575056379 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00d2a00 con 0x5561cc198b20 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb39dffb640 1 -- 172.21.15.103:0/575056379 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fb3a009afc0 con 0x5561cc198b20 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a80035f0 con 0x5561cc198b20 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb39dffb640 1 -- 172.21.15.103:0/575056379 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3a009bb70 con 0x5561cc198b20 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb3bee68740 10 client.16299 fetch_fsmap learned FSMap version 31 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5561cc21d870 con 0x5561cc198b20 2024-08-19T01:39:12.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb39dffb640 1 -- 172.21.15.103:0/575056379 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a80035f0 con 0x5561cc198b20 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb39dffb640 1 -- 172.21.15.103:0/575056379 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb3a00a1050 con 0x5561cc198b20 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb3bee68740 10 client.16299 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb3bee68740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.149+0000 7fb3bee68740 20 client.16299 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.150+0000 7fb3bee68740 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc21fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.150+0000 7fb3bee68740 4 client.16299 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.150+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5561cc105270 con 0x5561cc1b3420 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.150+0000 7fb3b659c640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc21fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.150+0000 7fb3b659c640 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc21fe90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64cfb1b06bf81c86 server_cookie=a480a0135f1ab46b in_seq=0 out_seq=0 2024-08-19T01:39:12.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.150+0000 7fb39dffb640 10 client.16299 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb39dffb640 1 -- 172.21.15.103:0/575056379 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5561cc105270 con 0x5561cc1b3420 2024-08-19T01:39:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb39dffb640 10 client.16299 handle_command_reply: tid=0 2024-08-19T01:39:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 client.16299 shutdown 2024-08-19T01:39:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 msgr2=0x5561cc1a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a6a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb3a009bb00 tx=0x7fb3a0000bc0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 shutdown_connections 2024-08-19T01:39:12.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5561cc1b3420 0x5561cc21fe90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5561cc1a33e0 0x5561cc21c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561cc198b20 0x5561cc1a6a80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 --2- 172.21.15.103:0/575056379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5561cc198200 0x5561cc1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 >> 172.21.15.103:0/575056379 conn(0x5561cc194790 msgr2=0x5561cc1a1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb39dffb640 0 client.16299 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.151+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 shutdown_connections 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bee68740 1 -- 172.21.15.103:0/575056379 wait complete. 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bee68740 20 client.16299 trim_cache size 0 max 16384 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:39:12.157 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "83157a57-78f4-4978-9160-eae1e686f027", 2024-08-19T01:39:12.158 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:39:12.158 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb3a40a2c00 msgr2=0x7fb3a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb3a40a2c00 0x7fb3a40a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0004af0 tx=0x7fb3b0004a40 comp rx=0 tx=0).stop 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b81034d0 msgr2=0x7fb3b8198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b81034d0 0x7fb3b8198710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e9f0 tx=0x7fb3a000eeb0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 shutdown_connections 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb3a40a2c00 0x7fb3a40a50c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3b810c020 0x7fb3b819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3b81034d0 0x7fb3b8198710 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 --2- 172.21.15.103:0/2074277547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3b8102b20 0x7fb3b81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.152+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 >> 172.21.15.103:0/2074277547 conn(0x7fb3b80fc500 msgr2=0x7fb3b80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.153+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 shutdown_connections 2024-08-19T01:39:12.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:12.153+0000 7fb3bca6a640 1 -- 172.21.15.103:0/2074277547 wait complete. 2024-08-19T01:39:12.182 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:11 smithi190 ceph-mon[22032]: pgmap v441: 97 pgs: 97 active+clean; 4.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:39:12.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:11 smithi146 ceph-mon[22098]: pgmap v441: 97 pgs: 97 active+clean; 4.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:39:12.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:12.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:39:12.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:12.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:39:12.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:12 smithi103 ceph-mon[18436]: pgmap v442: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 32 op/s 2024-08-19T01:39:13.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:39:13.294 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:39:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:39:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:39:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:12 smithi190 ceph-mon[22032]: pgmap v442: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 32 op/s 2024-08-19T01:39:13.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:13.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:39:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:12 smithi146 ceph-mon[22098]: pgmap v442: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 32 op/s 2024-08-19T01:39:13.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 -- 172.21.15.103:0/3385177875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c301022a0 msgr2=0x7f8c301026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 --2- 172.21.15.103:0/3385177875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c301022a0 0x7f8c301026e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009900 tx=0x7f8c2402a4d0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 -- 172.21.15.103:0/3385177875 shutdown_connections 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 --2- 172.21.15.103:0/3385177875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c30102c20 0x7f8c3010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 --2- 172.21.15.103:0/3385177875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c301022a0 0x7f8c301026e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 --2- 172.21.15.103:0/3385177875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c30100b50 0x7f8c30100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 -- 172.21.15.103:0/3385177875 >> 172.21.15.103:0/3385177875 conn(0x7f8c300fc500 msgr2=0x7f8c300fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 -- 172.21.15.103:0/3385177875 shutdown_connections 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.791+0000 7f8c3482a640 1 -- 172.21.15.103:0/3385177875 wait complete. 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 Processor -- start 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 -- start start 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c30100b50 0x7f8c301982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c301022a0 0x7f8c30198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c30102c20 0x7f8c3019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30110c40 con 0x7f8c30100b50 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30110ac0 con 0x7f8c30102c20 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c3482a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30110dc0 con 0x7f8c301022a0 2024-08-19T01:39:13.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c30100b50 0x7f8c301982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c30100b50 0x7f8c301982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53058/0 (socket says 172.21.15.103:53058) 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.792+0000 7f8c2e59c640 1 -- 172.21.15.103:0/844478581 learned_addr learned my addr 172.21.15.103:0/844478581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c301022a0 0x7f8c30198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c30102c20 0x7f8c3019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2e59c640 1 -- 172.21.15.103:0/844478581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c301022a0 msgr2=0x7f8c30198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c301022a0 0x7f8c30198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2e59c640 1 -- 172.21.15.103:0/844478581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c30102c20 msgr2=0x7f8c3019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c30102c20 0x7f8c3019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2e59c640 1 -- 172.21.15.103:0/844478581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3019d2a0 con 0x7f8c30100b50 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c30100b50 0x7f8c301982f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b450 tx=0x7f8c2000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c1f7fe640 1 -- 172.21.15.103:0/844478581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20015020 con 0x7f8c30100b50 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.793+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3019d590 con 0x7f8c30100b50 2024-08-19T01:39:13.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.794+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301a4e70 con 0x7f8c30100b50 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.794+0000 7f8c1f7fe640 1 -- 172.21.15.103:0/844478581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c20011d40 con 0x7f8c30100b50 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.794+0000 7f8c1f7fe640 1 -- 172.21.15.103:0/844478581 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000f340 con 0x7f8c30100b50 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 Processor -- start 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 -- start start 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56293580a980 con 0x5629358a59f0 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5629357a2a70 con 0x5629358a50d0 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c36c28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562935806940 con 0x5629358b4800 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c1f7fe640 1 -- 172.21.15.103:0/844478581 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8c20018020 con 0x7f8c30100b50 2024-08-19T01:39:13.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53072/0 (socket says 172.21.15.103:53072) 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.797+0000 7f8c2e59c640 1 -- 172.21.15.103:0/2469552586 learned_addr learned my addr 172.21.15.103:0/2469552586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2e59c640 1 -- 172.21.15.103:0/2469552586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 msgr2=0x5629358b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2e59c640 1 -- 172.21.15.103:0/2469552586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 msgr2=0x5629358a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2e59c640 1 -- 172.21.15.103:0/2469552586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629358056e0 con 0x5629358a59f0 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.798+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b42c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8c200d2b80 tx=0x7f8c200d7540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1d918698059a7f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.799+0000 7f8c1cff9640 1 -- 172.21.15.103:0/2469552586 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c200d8040 con 0x5629358a59f0 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.799+0000 7f8c1cff9640 1 -- 172.21.15.103:0/2469552586 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c200d7c50 con 0x5629358a59f0 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.799+0000 7f8c1f7fe640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c180a2c00 0x7f8c180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.799+0000 7f8c1cff9640 1 -- 172.21.15.103:0/2469552586 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c200db5f0 con 0x5629358a59f0 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.799+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c180a2c00 0x7f8c180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.799+0000 7f8c1f7fe640 1 -- 172.21.15.103:0/844478581 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8c200cea70 con 0x7f8c30100b50 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.800+0000 7f8c36c28740 1 -- 172.21.15.103:0/2469552586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 msgr2=0x5629358b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.800+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c180a2c00 0x7f8c180a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009ee0 tx=0x7f8c280094d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.800+0000 7f8c36c28740 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b42c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8c200d2b80 tx=0x7f8c200d7540 comp rx=0 tx=0).stop 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 -- 172.21.15.103:0/2469552586 shutdown_connections 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b42c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 --2- 172.21.15.103:0/2469552586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 -- 172.21.15.103:0/2469552586 >> 172.21.15.103:0/2469552586 conn(0x5629358a17a0 msgr2=0x5629358a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 -- 172.21.15.103:0/2469552586 shutdown_connections 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 -- 172.21.15.103:0/2469552586 wait complete. 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.801+0000 7f8c36c28740 1 -- start start 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 1 Processor -- start 2024-08-19T01:39:13.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a50d0 0x5629358b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358b4800 0x5629358b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629358b73d0 con 0x5629358a59f0 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5629357c5aa0 con 0x5629358b4800 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5629358b7550 con 0x5629358a50d0 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c36c28740 10 client.0 _ll_register_callbacks cb 0x5629357b4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c2dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358b4800 0x5629358b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c2dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53078/0 (socket says 172.21.15.103:53078) 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c2dd9b640 1 -- 172.21.15.103:0/3959171704 learned_addr learned my addr 172.21.15.103:0/3959171704 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.802+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a50d0 0x5629358b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c2ed9d640 1 -- 172.21.15.103:0/3959171704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a50d0 msgr2=0x5629358b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a50d0 0x5629358b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c2ed9d640 1 -- 172.21.15.103:0/3959171704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 msgr2=0x5629358b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c2ed9d640 1 -- 172.21.15.103:0/3959171704 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629357a9650 con 0x5629358b4800 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358b4800 0x5629358b2200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c28004b10 tx=0x7f8c28004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e5d63931111a4d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c15ffb640 1 -- 172.21.15.103:0/3959171704 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28027060 con 0x5629358b4800 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c15ffb640 1 -- 172.21.15.103:0/3959171704 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8c28012030 con 0x5629358b4800 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c200d7c50 con 0x5629358b4800 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c15ffb640 1 -- 172.21.15.103:0/3959171704 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c28016560 con 0x5629358b4800 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c36c28740 10 client.25742 fetch_fsmap learned FSMap version 31 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.803+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56293557c3c0 con 0x5629358b4800 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c15ffb640 1 -- 172.21.15.103:0/3959171704 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c200d7c50 con 0x5629358b4800 2024-08-19T01:39:13.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c15ffb640 1 -- 172.21.15.103:0/3959171704 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8c2802d870 con 0x5629358b4800 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c36c28740 10 client.25742 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c36c28740 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c36c28740 20 client.25742 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c36c28740 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629358aad90 0x5629358ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c36c28740 4 client.25742 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562935817ef0 con 0x5629358aad90 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.804+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629358aad90 0x5629358ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.805+0000 7f8c15ffb640 10 client.25742 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.805+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629358aad90 0x5629358ab1a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b29155cdfd60df server_cookie=a954ad94c74fe54e in_seq=0 out_seq=0 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.805+0000 7f8c15ffb640 1 -- 172.21.15.103:0/3959171704 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562935817ef0 con 0x5629358aad90 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.805+0000 7f8c15ffb640 10 client.25742 handle_command_reply: tid=0 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 client.25742 shutdown 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358b4800 msgr2=0x5629358b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358b4800 0x5629358b2200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c28004b10 tx=0x7f8c28004a60 comp rx=0 tx=0).stop 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 shutdown_connections 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629358aad90 0x5629358ab1a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358b4800 0x5629358b2200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a59f0 0x5629358b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 --2- 172.21.15.103:0/3959171704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a50d0 0x5629358b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 >> 172.21.15.103:0/3959171704 conn(0x5629358a17a0 msgr2=0x5629358a7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c15ffb640 0 client.25742 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 shutdown_connections 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 1 -- 172.21.15.103:0/3959171704 wait complete. 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.806+0000 7f8c36c28740 20 client.25742 trim_cache size 0 max 16384 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.825+0000 7f8c36c28740 1 Processor -- start 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.825+0000 7f8c36c28740 1 -- start start 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a59f0 0x5629358aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358b4800 0x5629358aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c36c28740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301ab080 con 0x5629358b4800 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c36c28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301accb0 con 0x5629358a50d0 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c36c28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301ab750 con 0x5629358a59f0 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358b4800 0x5629358aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358b4800 0x5629358aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53092/0 (socket says 172.21.15.103:53092) 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 -- 172.21.15.103:0/1631148600 learned_addr learned my addr 172.21.15.103:0/1631148600 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a59f0 0x5629358aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 -- 172.21.15.103:0/1631148600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a59f0 msgr2=0x5629358aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a59f0 0x5629358aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 -- 172.21.15.103:0/1631148600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 msgr2=0x5629358b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 -- 172.21.15.103:0/1631148600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629358a9580 con 0x5629358b4800 2024-08-19T01:39:13.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.826+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358b4800 0x5629358aef40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cc80 tx=0x7f8c280163a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a480ddb17ecc791 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c17fff640 1 -- 172.21.15.103:0/1631148600 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28032070 con 0x5629358b4800 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c17fff640 1 -- 172.21.15.103:0/1631148600 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c2802c360 con 0x5629358b4800 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c17fff640 1 -- 172.21.15.103:0/1631148600 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28016700 con 0x5629358b4800 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 -- 172.21.15.103:0/1631148600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358b4800 msgr2=0x5629358aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358b4800 0x5629358aef40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800cc80 tx=0x7f8c280163a0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 -- 172.21.15.103:0/1631148600 shutdown_connections 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358b4800 0x5629358aef40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358a59f0 0x5629358aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1631148600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a50d0 0x5629358b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 -- 172.21.15.103:0/1631148600 >> 172.21.15.103:0/1631148600 conn(0x5629358a17a0 msgr2=0x5629358abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:13.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 -- 172.21.15.103:0/1631148600 shutdown_connections 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.827+0000 7f8c36c28740 1 -- 172.21.15.103:0/1631148600 wait complete. 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 -- start start 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 Processor -- start 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a50d0 0x5629358cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a59f0 0x5629358cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301ac520 con 0x5629358a50d0 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301abe20 con 0x5629358a59f0 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301a9080 con 0x5629358b4800 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c36c28740 10 client.0 _ll_register_callbacks cb 0x7f8c2800ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a50d0 0x5629358cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c2e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a50d0 0x5629358cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53098/0 (socket says 172.21.15.103:53098) 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c2e59c640 1 -- 172.21.15.103:0/1787620660 learned_addr learned my addr 172.21.15.103:0/1787620660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:13.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.828+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a59f0 0x5629358cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2e59c640 1 -- 172.21.15.103:0/1787620660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 msgr2=0x5629358ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2ed9d640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2e59c640 1 -- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a59f0 msgr2=0x5629358cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a59f0 0x5629358cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2e59c640 1 -- 172.21.15.103:0/1787620660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629359321c0 con 0x5629358a50d0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a59f0 0x5629358cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c2e59c640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a50d0 0x5629358cc070 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009fd0 tx=0x7f8c200dca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eca4a998aa9e40 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c15ffb640 1 -- 172.21.15.103:0/1787620660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20080e80 con 0x5629358a50d0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c15ffb640 1 -- 172.21.15.103:0/1787620660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8c200d2060 con 0x5629358a50d0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c2802c360 con 0x5629358a50d0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c15ffb640 1 -- 172.21.15.103:0/1787620660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c200975a0 con 0x5629358a50d0 2024-08-19T01:39:13.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c36c28740 10 client.16317 fetch_fsmap learned FSMap version 31 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5629359334b0 con 0x5629358a50d0 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.829+0000 7f8c15ffb640 1 -- 172.21.15.103:0/1787620660 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c2802c360 con 0x5629358a50d0 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c15ffb640 1 -- 172.21.15.103:0/1787620660 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8c200db070 con 0x5629358a50d0 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c36c28740 10 client.16317 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c36c28740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c36c28740 20 client.16317 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629359337e0 0x562935935c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c36c28740 4 client.16317 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562935812400 con 0x5629359337e0 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.830+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629359337e0 0x562935935c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c2dd9b640 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629359337e0 0x562935935c00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de19b3725a44eae8 server_cookie=755f9a2d59899a4b in_seq=0 out_seq=0 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c15ffb640 10 client.16317 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c15ffb640 1 -- 172.21.15.103:0/1787620660 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562935812400 con 0x5629359337e0 2024-08-19T01:39:13.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c15ffb640 10 client.16317 handle_command_reply: tid=0 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 client.16317 shutdown 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a50d0 msgr2=0x5629358cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a50d0 0x5629358cc070 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009fd0 tx=0x7f8c200dca30 comp rx=0 tx=0).stop 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 shutdown_connections 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5629359337e0 0x562935935c00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:39:13.874 INFO:teuthology.orchestra.run.smithi103.stdout: "83157a57-78f4-4978-9160-eae1e686f027": { 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "83157a57-78f4-4978-9160-eae1e686f027", 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5629358b4800 0x5629358ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5629358a59f0 0x5629358cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 --2- 172.21.15.103:0/1787620660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629358a50d0 0x5629358cc070 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 >> 172.21.15.103:0/1787620660 conn(0x5629358a17a0 msgr2=0x5629358bf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c15ffb640 0 client.16317 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 shutdown_connections 2024-08-19T01:39:13.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.831+0000 7f8c36c28740 1 -- 172.21.15.103:0/1787620660 wait complete. 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c36c28740 20 client.16317 trim_cache size 0 max 16384 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c180a2c00 msgr2=0x7f8c180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c180a2c00 0x7f8c180a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009ee0 tx=0x7f8c280094d0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c30100b50 msgr2=0x7f8c301982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c30100b50 0x7f8c301982f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000b450 tx=0x7f8c2000b910 comp rx=0 tx=0).stop 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 shutdown_connections 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8c180a2c00 0x7f8c180a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c30102c20 0x7f8c3019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c301022a0 0x7f8c30198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 --2- 172.21.15.103:0/844478581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c30100b50 0x7f8c301982f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 >> 172.21.15.103:0/844478581 conn(0x7f8c300fc500 msgr2=0x7f8c3006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 shutdown_connections 2024-08-19T01:39:13.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:13.832+0000 7f8c3482a640 1 -- 172.21.15.103:0/844478581 wait complete. 2024-08-19T01:39:14.662 INFO:tasks.cephfs.filesystem:scrub status for tag:83157a57-78f4-4978-9160-eae1e686f027 - {'path': '/', 'tag': '83157a57-78f4-4978-9160-eae1e686f027', 'options': 'recursive,force'} 2024-08-19T01:39:16.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:15 smithi146 ceph-mon[22098]: pgmap v443: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 MiB/s wr, 32 op/s 2024-08-19T01:39:16.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:15 smithi103 ceph-mon[18436]: pgmap v443: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 MiB/s wr, 32 op/s 2024-08-19T01:39:16.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:15 smithi190 ceph-mon[22032]: pgmap v443: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 MiB/s wr, 32 op/s 2024-08-19T01:39:18.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:17 smithi146 ceph-mon[22098]: pgmap v444: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 41 op/s 2024-08-19T01:39:18.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:17 smithi103 ceph-mon[18436]: pgmap v444: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 41 op/s 2024-08-19T01:39:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:17 smithi190 ceph-mon[22032]: pgmap v444: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 41 op/s 2024-08-19T01:39:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:19 smithi146 ceph-mon[22098]: pgmap v445: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 MiB/s wr, 50 op/s 2024-08-19T01:39:20.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:19 smithi103 ceph-mon[18436]: pgmap v445: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 MiB/s wr, 50 op/s 2024-08-19T01:39:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:19 smithi190 ceph-mon[22032]: pgmap v445: 97 pgs: 97 active+clean; 4.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 MiB/s wr, 50 op/s 2024-08-19T01:39:22.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:21 smithi146 ceph-mon[22098]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 48 op/s 2024-08-19T01:39:22.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:21 smithi103 ceph-mon[18436]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 48 op/s 2024-08-19T01:39:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:21 smithi190 ceph-mon[22032]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 48 op/s 2024-08-19T01:39:24.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:23 smithi146 ceph-mon[22098]: pgmap v447: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 32 MiB/s wr, 51 op/s 2024-08-19T01:39:24.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:23 smithi103 ceph-mon[18436]: pgmap v447: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 32 MiB/s wr, 51 op/s 2024-08-19T01:39:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:23 smithi190 ceph-mon[22032]: pgmap v447: 97 pgs: 97 active+clean; 4.4 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 32 MiB/s wr, 51 op/s 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 -- 172.21.15.103:0/3487163154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c100780 msgr2=0x7fdc4c100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 --2- 172.21.15.103:0/3487163154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c100780 0x7fdc4c100b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009900 tx=0x7fdc4802a410 comp rx=0 tx=0).stop 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 -- 172.21.15.103:0/3487163154 shutdown_connections 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 --2- 172.21.15.103:0/3487163154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc4c102330 0x7fdc4c108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 --2- 172.21.15.103:0/3487163154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc4c101970 0x7fdc4c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 --2- 172.21.15.103:0/3487163154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c100780 0x7fdc4c100b80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 -- 172.21.15.103:0/3487163154 >> 172.21.15.103:0/3487163154 conn(0x7fdc4c0fbee0 msgr2=0x7fdc4c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 -- 172.21.15.103:0/3487163154 shutdown_connections 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 -- 172.21.15.103:0/3487163154 wait complete. 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 Processor -- start 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.669+0000 7fdc539d1640 1 -- start start 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc539d1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc4c100780 0x7fdc4c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc539d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c101970 0x7fdc4c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc539d1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc4c102330 0x7fdc4c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc539d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c10c290 con 0x7fdc4c101970 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc539d1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c10c110 con 0x7fdc4c102330 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc539d1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c10c410 con 0x7fdc4c100780 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c101970 0x7fdc4c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c101970 0x7fdc4c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34494/0 (socket says 172.21.15.103:34494) 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 -- 172.21.15.103:0/2365769362 learned_addr learned my addr 172.21.15.103:0/2365769362 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc5176d640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc4c100780 0x7fdc4c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc4c102330 0x7fdc4c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 -- 172.21.15.103:0/2365769362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc4c100780 msgr2=0x7fdc4c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc4c100780 0x7fdc4c06aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 -- 172.21.15.103:0/2365769362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc4c102330 msgr2=0x7fdc4c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc4c102330 0x7fdc4c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 -- 172.21.15.103:0/2365769362 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c19f6d0 con 0x7fdc4c101970 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c101970 0x7fdc4c19c7a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00d950 tx=0x7fdc3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc427fc640 1 -- 172.21.15.103:0/2365769362 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c017070 con 0x7fdc4c101970 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc427fc640 1 -- 172.21.15.103:0/2365769362 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc3c009c30 con 0x7fdc4c101970 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.670+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4c19f960 con 0x7fdc4c101970 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.671+0000 7fdc427fc640 1 -- 172.21.15.103:0/2365769362 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00f620 con 0x7fdc4c101970 2024-08-19T01:39:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.671+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4c19ff00 con 0x7fdc4c101970 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 Processor -- start 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 -- start start 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b64492e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b644933d0 0x558b64497e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64498460 0x558b6449a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558b643f19d0 con 0x558b64498460 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558b64399820 con 0x558b644933d0 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5686d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558b643ed990 con 0x558b64492a90 2024-08-19T01:39:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc427fc640 1 -- 172.21.15.103:0/2365769362 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdc3c0109a0 con 0x7fdc4c101970 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5176d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b644933d0 0x558b64497e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc51f6e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b64492e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5176d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b644933d0 0x558b64497e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:41400/0 (socket says 172.21.15.103:41400) 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc5176d640 1 -- 172.21.15.103:0/1730376513 learned_addr learned my addr 172.21.15.103:0/1730376513 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.674+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64498460 0x558b6449a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc51f6e640 1 -- 172.21.15.103:0/1730376513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b644933d0 msgr2=0x558b64497e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b644933d0 0x558b64497e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc51f6e640 1 -- 172.21.15.103:0/1730376513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64498460 msgr2=0x558b6449a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64498460 0x558b6449a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc51f6e640 1 -- 172.21.15.103:0/1730376513 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b64105210 con 0x558b64492a90 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b64492e90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc44007d90 tx=0x7fdc4400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb0b3dd55222ef0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc427fc640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc380a2cb0 0x7fdc380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc377fe640 1 -- 172.21.15.103:0/1730376513 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44014070 con 0x558b64492a90 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc377fe640 1 -- 172.21.15.103:0/1730376513 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc44003e20 con 0x558b64492a90 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc377fe640 1 -- 172.21.15.103:0/1730376513 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44004820 con 0x558b64492a90 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc427fc640 1 -- 172.21.15.103:0/2365769362 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdc3c0cec90 con 0x7fdc4c101970 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.675+0000 7fdc5176d640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc380a2cb0 0x7fdc380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 -- 172.21.15.103:0/1730376513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 msgr2=0x558b64492e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5176d640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc380a2cb0 0x7fdc380a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005c40 tx=0x7fdc480058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b64492e90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc44007d90 tx=0x7fdc4400e720 comp rx=0 tx=0).stop 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 -- 172.21.15.103:0/1730376513 shutdown_connections 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64498460 0x558b6449a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b644933d0 0x558b64497e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 --2- 172.21.15.103:0/1730376513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b64492e90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 -- 172.21.15.103:0/1730376513 >> 172.21.15.103:0/1730376513 conn(0x558b64489cf0 msgr2=0x558b6448a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 -- 172.21.15.103:0/1730376513 shutdown_connections 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 -- 172.21.15.103:0/1730376513 wait complete. 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 -- start start 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.676+0000 7fdc5686d740 1 Processor -- start 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64492a90 0x558b6449cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b6449d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64498460 0x558b644a4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5686d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558b6449b430 con 0x558b64492a90 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5686d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558b643ad520 con 0x558b64498460 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5686d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558b6449b5b0 con 0x558b644933d0 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5686d740 10 client.0 _ll_register_callbacks cb 0x558b64104a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64492a90 0x558b6449cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64492a90 0x558b6449cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34522/0 (socket says 172.21.15.103:34522) 2024-08-19T01:39:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 -- 172.21.15.103:0/97943712 learned_addr learned my addr 172.21.15.103:0/97943712 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b6449d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64498460 0x558b644a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 -- 172.21.15.103:0/97943712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 msgr2=0x558b6449d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b6449d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 -- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64498460 msgr2=0x558b644a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64498460 0x558b644a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc5176d640 1 -- 172.21.15.103:0/97943712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b641d1620 con 0x558b64492a90 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.677+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b6449d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc5176d640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64492a90 0x558b6449cc70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdc48033000 tx=0x7fdc4802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b4483afb146234 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc34ff9640 1 -- 172.21.15.103:0/97943712 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4804c070 con 0x558b64492a90 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc44003e20 con 0x558b64492a90 2024-08-19T01:39:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc34ff9640 1 -- 172.21.15.103:0/97943712 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdc480043d0 con 0x558b64492a90 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc34ff9640 1 -- 172.21.15.103:0/97943712 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc48058050 con 0x558b64492a90 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc5686d740 10 client.16329 fetch_fsmap learned FSMap version 31 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b644a7030 con 0x558b64492a90 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc34ff9640 1 -- 172.21.15.103:0/97943712 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc44003e20 con 0x558b64492a90 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.678+0000 7fdc34ff9640 1 -- 172.21.15.103:0/97943712 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdc4803c780 con 0x558b64492a90 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc5686d740 10 client.16329 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc5686d740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc5686d740 20 client.16329 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc5686d740 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b644a8bc0 0x558b644ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc5686d740 4 client.16329 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b643ff100 con 0x558b644a8bc0 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b644a8bc0 0x558b644ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc34ff9640 10 client.16329 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.679+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b644a8bc0 0x558b644ab000 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b9ceb0c47a3b517 server_cookie=8aa2137e02d5762b in_seq=0 out_seq=0 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc34ff9640 1 -- 172.21.15.103:0/97943712 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b643ff100 con 0x558b644a8bc0 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc34ff9640 10 client.16329 handle_command_reply: tid=0 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 client.16329 shutdown 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64492a90 msgr2=0x558b6449cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64492a90 0x558b6449cc70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdc48033000 tx=0x7fdc4802a410 comp rx=0 tx=0).stop 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 shutdown_connections 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b644a8bc0 0x558b644ab000 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64498460 0x558b644a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b6449d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc34ff9640 0 client.16329 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 --2- 172.21.15.103:0/97943712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64492a90 0x558b6449cc70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 >> 172.21.15.103:0/97943712 conn(0x558b64489cf0 msgr2=0x558b6448b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 shutdown_connections 2024-08-19T01:39:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 1 -- 172.21.15.103:0/97943712 wait complete. 2024-08-19T01:39:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.680+0000 7fdc5686d740 20 client.16329 trim_cache size 0 max 16384 2024-08-19T01:39:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.697+0000 7fdc5686d740 1 Processor -- start 2024-08-19T01:39:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.697+0000 7fdc5686d740 1 -- start start 2024-08-19T01:39:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64489cf0 0x558b6449d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b644a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b644933d0 0x558b6448b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc5686d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1a6960 con 0x558b644933d0 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc5686d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1a8590 con 0x558b64489cf0 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc5686d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1a7030 con 0x558b64492a90 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b644933d0 0x558b6448b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b644933d0 0x558b6448b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34536/0 (socket says 172.21.15.103:34536) 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 -- 172.21.15.103:0/2296149640 learned_addr learned my addr 172.21.15.103:0/2296149640 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b644a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc5176d640 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64489cf0 0x558b6449d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 -- 172.21.15.103:0/2296149640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 msgr2=0x558b644a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b644a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 -- 172.21.15.103:0/2296149640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64489cf0 msgr2=0x558b6449d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64489cf0 0x558b6449d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 -- 172.21.15.103:0/2296149640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6449b750 con 0x558b644933d0 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b644933d0 0x558b6448b0e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005c40 tx=0x7fdc44005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8a2ffb4a83686e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc36ffd640 1 -- 172.21.15.103:0/2296149640 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44014040 con 0x558b644933d0 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc36ffd640 1 -- 172.21.15.103:0/2296149640 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc44004d80 con 0x558b644933d0 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.698+0000 7fdc36ffd640 1 -- 172.21.15.103:0/2296149640 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44010720 con 0x558b644933d0 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 -- 172.21.15.103:0/2296149640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b644933d0 msgr2=0x558b6448b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b644933d0 0x558b6448b0e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005c40 tx=0x7fdc44005bd0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 -- 172.21.15.103:0/2296149640 shutdown_connections 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b644933d0 0x558b6448b0e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b64492a90 0x558b644a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 --2- 172.21.15.103:0/2296149640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64489cf0 0x558b6449d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 -- 172.21.15.103:0/2296149640 >> 172.21.15.103:0/2296149640 conn(0x558b64497f80 msgr2=0x558b64488ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 -- 172.21.15.103:0/2296149640 shutdown_connections 2024-08-19T01:39:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.699+0000 7fdc5686d740 1 -- 172.21.15.103:0/2296149640 wait complete. 2024-08-19T01:39:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 -- start start 2024-08-19T01:39:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 Processor -- start 2024-08-19T01:39:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64489cf0 0x558b644a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64492a90 0x558b644a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b644a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1a7e00 con 0x558b64489cf0 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1a7700 con 0x558b64492a90 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c1a4940 con 0x558b644933d0 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5686d740 10 client.0 _ll_register_callbacks cb 0x7fdc48040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5176d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64489cf0 0x558b644a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5176d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64489cf0 0x558b644a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34540/0 (socket says 172.21.15.103:34540) 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc5176d640 1 -- 172.21.15.103:0/442660219 learned_addr learned my addr 172.21.15.103:0/442660219 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc51f6e640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b644a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.700+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64492a90 0x558b644a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5176d640 1 -- 172.21.15.103:0/442660219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 msgr2=0x558b644a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5176d640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b644a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5176d640 1 -- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64492a90 msgr2=0x558b644a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5176d640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64492a90 0x558b644a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5176d640 1 -- 172.21.15.103:0/442660219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b644a3400 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64492a90 0x558b644a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5176d640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64489cf0 0x558b644a18a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005940 tx=0x7fdc48004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a8402d197bebba server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc34ff9640 1 -- 172.21.15.103:0/442660219 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48060070 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc34ff9640 1 -- 172.21.15.103:0/442660219 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdc4803c640 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc44004d80 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc34ff9640 1 -- 172.21.15.103:0/442660219 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc4804c040 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5686d740 10 client.16335 fetch_fsmap learned FSMap version 31 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.701+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558b645137b0 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc34ff9640 1 -- 172.21.15.103:0/442660219 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc44004d80 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc34ff9640 1 -- 172.21.15.103:0/442660219 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdc48063020 con 0x558b64489cf0 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc5686d740 10 client.16335 fetch_fsmap finished waiting for FSMap version 31 2024-08-19T01:39:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc5686d740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc5686d740 20 client.16335 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc5686d740 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b64498460 0x558b645161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc5686d740 4 client.16335 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:39:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b643f9450 con 0x558b64498460 2024-08-19T01:39:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b64498460 0x558b645161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc34ff9640 10 client.16335 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.702+0000 7fdc50f6c640 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b64498460 0x558b645161f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e71a82195e40256a server_cookie=509f6835e0691ae8 in_seq=0 out_seq=0 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.710+0000 7fdc34ff9640 1 -- 172.21.15.103:0/442660219 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b643f9450 con 0x558b64498460 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.710+0000 7fdc34ff9640 10 client.16335 handle_command_reply: tid=0 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.710+0000 7fdc5686d740 1 client.16335 shutdown 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64489cf0 msgr2=0x558b644a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64489cf0 0x558b644a18a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005940 tx=0x7fdc48004600 comp rx=0 tx=0).stop 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 shutdown_connections 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558b64498460 0x558b645161f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b644933d0 0x558b644a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b64492a90 0x558b644a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc34ff9640 0 client.16335 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 --2- 172.21.15.103:0/442660219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b64489cf0 0x558b644a18a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 >> 172.21.15.103:0/442660219 conn(0x558b64497f80 msgr2=0x558b64488ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 shutdown_connections 2024-08-19T01:39:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 1 -- 172.21.15.103:0/442660219 wait complete. 2024-08-19T01:39:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.711+0000 7fdc5686d740 20 client.16335 trim_cache size 0 max 16384 2024-08-19T01:39:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:39:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:39:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:39:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:39:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc380a2cb0 msgr2=0x7fdc380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc380a2cb0 0x7fdc380a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc48005c40 tx=0x7fdc480058d0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c101970 msgr2=0x7fdc4c19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c101970 0x7fdc4c19c7a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00d950 tx=0x7fdc3c00de10 comp rx=0 tx=0).stop 2024-08-19T01:39:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 shutdown_connections 2024-08-19T01:39:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc380a2cb0 0x7fdc380a5170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc4c102330 0x7fdc4c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc4c101970 0x7fdc4c19c7a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 --2- 172.21.15.103:0/2365769362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc4c100780 0x7fdc4c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 >> 172.21.15.103:0/2365769362 conn(0x7fdc4c0fbee0 msgr2=0x7fdc4c104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 shutdown_connections 2024-08-19T01:39:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:24.712+0000 7fdc539d1640 1 -- 172.21.15.103:0/2365769362 wait complete. 2024-08-19T01:39:26.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:25 smithi146 ceph-mon[22098]: pgmap v448: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 39 op/s 2024-08-19T01:39:26.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:25 smithi103 ceph-mon[18436]: pgmap v448: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 39 op/s 2024-08-19T01:39:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:25 smithi190 ceph-mon[22032]: pgmap v448: 97 pgs: 97 active+clean; 4.5 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 39 op/s 2024-08-19T01:39:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:26 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:39:27.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:26 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:39:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:26 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:39:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:26 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:39:27.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:26 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:39:27.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:26 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:39:28.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:27 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:39:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:27 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:39:28.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:27 smithi146 ceph-mon[22098]: pgmap v449: 97 pgs: 97 active+clean; 4.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 35 op/s 2024-08-19T01:39:28.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:27 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:39:28.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:27 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:39:28.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:27 smithi103 ceph-mon[18436]: pgmap v449: 97 pgs: 97 active+clean; 4.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 35 op/s 2024-08-19T01:39:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:27 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:39:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:27 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:39:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:27 smithi190 ceph-mon[22032]: pgmap v449: 97 pgs: 97 active+clean; 4.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 35 op/s 2024-08-19T01:39:30.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:29 smithi146 ceph-mon[22098]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 32 op/s 2024-08-19T01:39:30.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:29 smithi103 ceph-mon[18436]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 32 op/s 2024-08-19T01:39:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:29 smithi190 ceph-mon[22032]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 32 op/s 2024-08-19T01:39:32.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:31 smithi146 ceph-mon[22098]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:32.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:31 smithi103 ceph-mon[18436]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:32.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:31 smithi190 ceph-mon[22032]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:33.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:32 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:33.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:32 smithi146 ceph-mon[22098]: pgmap v452: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:39:33.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:32 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:33.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:32 smithi103 ceph-mon[18436]: pgmap v452: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:39:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:32 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:39:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:32 smithi190 ceph-mon[22032]: pgmap v452: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:39:36.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:35 smithi146 ceph-mon[22098]: pgmap v453: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:39:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:39:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:39:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:39:36.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:39:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:35 smithi103 ceph-mon[18436]: pgmap v453: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:39:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:39:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:39:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:39:36.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:39:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:35 smithi190 ceph-mon[22032]: pgmap v453: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:39:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:39:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:39:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:39:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:39:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/807155884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 msgr2=0x7ff69c101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/807155884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c101c90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff698009900 tx=0x7ff69802a410 comp rx=0 tx=0).stop 2024-08-19T01:39:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/807155884 shutdown_connections 2024-08-19T01:39:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/807155884 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff69c103440 0x7ff69c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/807155884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c102a80 0x7ff69c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/807155884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c101c90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/807155884 >> 172.21.15.103:0/807155884 conn(0x7ff69c0fd010 msgr2=0x7ff69c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/807155884 shutdown_connections 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.800+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/807155884 wait complete. 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 Processor -- start 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 -- start start 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c102a80 0x7ff69c074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff69c103440 0x7ff69c074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c10d3f0 con 0x7ff69c101890 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c10d270 con 0x7ff69c103440 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a3b1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c10d570 con 0x7ff69c102a80 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a18b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a18b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c075cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46410/0 (socket says 172.21.15.103:46410) 2024-08-19T01:39:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a18b6640 1 -- 172.21.15.103:0/2128132600 learned_addr learned my addr 172.21.15.103:0/2128132600 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c102a80 0x7ff69c074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.801+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff69c103440 0x7ff69c074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a18b6640 1 -- 172.21.15.103:0/2128132600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c102a80 msgr2=0x7ff69c074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c102a80 0x7ff69c074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a18b6640 1 -- 172.21.15.103:0/2128132600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff69c103440 msgr2=0x7ff69c074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff69c103440 0x7ff69c074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a18b6640 1 -- 172.21.15.103:0/2128132600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c0750e0 con 0x7ff69c101890 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c102a80 0x7ff69c074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff69c103440 0x7ff69c074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c075cd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff6980314e0 tx=0x7ff698003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff692ffd640 1 -- 172.21.15.103:0/2128132600 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69803f070 con 0x7ff69c101890 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff692ffd640 1 -- 172.21.15.103:0/2128132600 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff698031e40 con 0x7ff69c101890 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c1a1b10 con 0x7ff69c101890 2024-08-19T01:39:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff692ffd640 1 -- 172.21.15.103:0/2128132600 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698035030 con 0x7ff69c101890 2024-08-19T01:39:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.802+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a2060 con 0x7ff69c101890 2024-08-19T01:39:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.804+0000 7ff692ffd640 1 -- 172.21.15.103:0/2128132600 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff698042070 con 0x7ff69c101890 2024-08-19T01:39:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 Processor -- start 2024-08-19T01:39:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 -- start start 2024-08-19T01:39:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d7225a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225fc0 0x55a3d722aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d722b050 0x55a3d722d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a3d71889d0 con 0x55a3d722b050 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7130820 con 0x55a3d7225680 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.805+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7184990 con 0x55a3d7225fc0 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d722b050 0x55a3d722d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a18b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225fc0 0x55a3d722aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d722b050 0x55a3d722d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46422/0 (socket says 172.21.15.103:46422) 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 -- 172.21.15.103:0/4059440484 learned_addr learned my addr 172.21.15.103:0/4059440484 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff692ffd640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff6880a2c30 0x7ff6880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 -- 172.21.15.103:0/4059440484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225fc0 msgr2=0x55a3d722aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225fc0 0x55a3d722aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d7225a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 -- 172.21.15.103:0/4059440484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 msgr2=0x55a3d7225a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d7225a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 -- 172.21.15.103:0/4059440484 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d6e9c210 con 0x55a3d722b050 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d7225a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff692ffd640 1 -- 172.21.15.103:0/2128132600 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff6980ef9c0 con 0x7ff69c101890 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff6880a2c30 0x7ff6880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d722b050 0x55a3d722d510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00bd30 tx=0x7ff68c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b433440e52b0d77 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff687fff640 1 -- 172.21.15.103:0/4059440484 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c015070 con 0x55a3d722b050 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff687fff640 1 -- 172.21.15.103:0/4059440484 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff68c00be20 con 0x55a3d722b050 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.806+0000 7ff687fff640 1 -- 172.21.15.103:0/4059440484 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c010750 con 0x55a3d722b050 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff6880a2c30 0x7ff6880a50f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff694005fd0 tx=0x7ff694004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 -- 172.21.15.103:0/4059440484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d722b050 msgr2=0x55a3d722d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d722b050 0x55a3d722d510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00bd30 tx=0x7ff68c00f730 comp rx=0 tx=0).stop 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 -- 172.21.15.103:0/4059440484 shutdown_connections 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d722b050 0x55a3d722d510 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225fc0 0x55a3d722aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/4059440484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d7225a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 -- 172.21.15.103:0/4059440484 >> 172.21.15.103:0/4059440484 conn(0x55a3d7220cf0 msgr2=0x55a3d72210f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.807+0000 7ff6a68d9740 1 -- 172.21.15.103:0/4059440484 shutdown_connections 2024-08-19T01:39:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 -- 172.21.15.103:0/4059440484 wait complete. 2024-08-19T01:39:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 -- start start 2024-08-19T01:39:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 Processor -- start 2024-08-19T01:39:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225680 0x55a3d722f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d722fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d722b050 0x55a3d7236df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a3d722e020 con 0x55a3d7225fc0 2024-08-19T01:39:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a3d7144520 con 0x55a3d722b050 2024-08-19T01:39:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a3d722e1a0 con 0x55a3d7225680 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a68d9740 10 client.0 _ll_register_callbacks cb 0x55a3d6e9ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a10b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d722fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a10b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d722fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46426/0 (socket says 172.21.15.103:46426) 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1240312810 learned_addr learned my addr 172.21.15.103:0/1240312810 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.808+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d722b050 0x55a3d7236df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225680 0x55a3d722f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1240312810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225680 msgr2=0x55a3d722f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225680 0x55a3d722f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d722b050 msgr2=0x55a3d7236df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d722b050 0x55a3d7236df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1240312810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d6f68620 con 0x55a3d7225fc0 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d722fdb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff68c004270 tx=0x7ff68c0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82060c12adef45c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6857fa640 1 -- 172.21.15.103:0/1240312810 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c028060 con 0x55a3d7225fc0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6857fa640 1 -- 172.21.15.103:0/1240312810 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff68c005030 con 0x55a3d7225fc0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff68c00be20 con 0x55a3d7225fc0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6857fa640 1 -- 172.21.15.103:0/1240312810 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff68c015050 con 0x55a3d7225fc0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a68d9740 10 client.16347 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.809+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3d7239b50 con 0x55a3d7225fc0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6857fa640 1 -- 172.21.15.103:0/1240312810 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff68c00be20 con 0x55a3d7225fc0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6857fa640 1 -- 172.21.15.103:0/1240312810 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff68c003710 con 0x55a3d7225fc0 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6a68d9740 10 client.16347 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6a68d9740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6a68d9740 20 client.16347 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d723afb0 0x55a3d723d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6a68d9740 4 client.16347 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d7196100 con 0x55a3d723afb0 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.810+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d723afb0 0x55a3d723d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.811+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d723afb0 0x55a3d723d3a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd649723786e4e58 server_cookie=ee18366136d72166 in_seq=0 out_seq=0 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.811+0000 7ff6857fa640 10 client.16347 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.811+0000 7ff6857fa640 1 -- 172.21.15.103:0/1240312810 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3d7196100 con 0x55a3d723afb0 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.811+0000 7ff6857fa640 10 client.16347 handle_command_reply: tid=0 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.811+0000 7ff6a68d9740 1 client.16347 shutdown 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.811+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 msgr2=0x55a3d722fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.811+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d722fdb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff68c004270 tx=0x7ff68c0042a0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 shutdown_connections 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d723afb0 0x55a3d723d3a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d722b050 0x55a3d7236df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d722fdb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6857fa640 0 client.16347 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1240312810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d7225680 0x55a3d722f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 >> 172.21.15.103:0/1240312810 conn(0x55a3d7220cf0 msgr2=0x55a3d7222500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 shutdown_connections 2024-08-19T01:39:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1240312810 wait complete. 2024-08-19T01:39:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.812+0000 7ff6a68d9740 20 client.16347 trim_cache size 0 max 16384 2024-08-19T01:39:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.830+0000 7ff6a68d9740 1 Processor -- start 2024-08-19T01:39:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.830+0000 7ff6a68d9740 1 -- start start 2024-08-19T01:39:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.830+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d721fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d7236db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d72220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1aad70 con 0x55a3d7225fc0 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1ac9d0 con 0x55a3d7225680 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1ab440 con 0x55a3d722b050 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d7236db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d7236db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46434/0 (socket says 172.21.15.103:46434) 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1432525599 learned_addr learned my addr 172.21.15.103:0/1432525599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d72220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d721fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1432525599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 msgr2=0x55a3d72220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d72220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1432525599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 msgr2=0x55a3d721fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d721fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 -- 172.21.15.103:0/1432525599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d723d710 con 0x55a3d7225fc0 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d7236db0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00fc30 tx=0x7ff68c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800765a8cb2cf85 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6877fe640 1 -- 172.21.15.103:0/1432525599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c033070 con 0x55a3d7225fc0 2024-08-19T01:39:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.831+0000 7ff6877fe640 1 -- 172.21.15.103:0/1432525599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff68c00be20 con 0x55a3d7225fc0 2024-08-19T01:39:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6877fe640 1 -- 172.21.15.103:0/1432525599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c028070 con 0x55a3d7225fc0 2024-08-19T01:39:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1432525599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 msgr2=0x55a3d7236db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d7236db0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00fc30 tx=0x7ff68c00b930 comp rx=0 tx=0).stop 2024-08-19T01:39:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1432525599 shutdown_connections 2024-08-19T01:39:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d72220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225fc0 0x55a3d7236db0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1432525599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225680 0x55a3d721fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1432525599 >> 172.21.15.103:0/1432525599 conn(0x55a3d7220cf0 msgr2=0x55a3d721f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1432525599 shutdown_connections 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.832+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1432525599 wait complete. 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 -- start start 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 Processor -- start 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225680 0x55a3d7234420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225fc0 0x55a3d7234960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d7235920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1ac240 con 0x55a3d7225680 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1abb10 con 0x55a3d7225fc0 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1a8d50 con 0x55a3d722b050 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a68d9740 10 client.0 _ll_register_callbacks cb 0x7ff6980b8570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a18b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225680 0x55a3d7234420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a18b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225680 0x55a3d7234420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46446/0 (socket says 172.21.15.103:46446) 2024-08-19T01:39:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a18b6640 1 -- 172.21.15.103:0/1954069167 learned_addr learned my addr 172.21.15.103:0/1954069167 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.833+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225fc0 0x55a3d7234960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d7235920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a18b6640 1 -- 172.21.15.103:0/1954069167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 msgr2=0x55a3d7235920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d7235920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a18b6640 1 -- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225fc0 msgr2=0x55a3d7234960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225fc0 0x55a3d7234960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a18b6640 1 -- 172.21.15.103:0/1954069167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d7235f30 con 0x55a3d7225680 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a20b7640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d7235920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a18b6640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225680 0x55a3d7234420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff6980b8c20 tx=0x7ff6980dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ff4e0678bb8e25 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6857fa640 1 -- 172.21.15.103:0/1954069167 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698049040 con 0x55a3d7225680 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6857fa640 1 -- 172.21.15.103:0/1954069167 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff6980b88c0 con 0x55a3d7225680 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff68c00be20 con 0x55a3d7225680 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6857fa640 1 -- 172.21.15.103:0/1954069167 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6980bc950 con 0x55a3d7225680 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a68d9740 10 client.16353 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3d72a5c60 con 0x55a3d7225680 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.834+0000 7ff6857fa640 1 -- 172.21.15.103:0/1954069167 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff68c00be20 con 0x55a3d7225680 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6857fa640 1 -- 172.21.15.103:0/1954069167 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff6980bc4b0 con 0x55a3d7225680 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a68d9740 10 client.16353 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a68d9740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a68d9740 20 client.16353 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d72a6650 0x55a3d72a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a68d9740 4 client.16353 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3d7190450 con 0x55a3d72a6650 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d72a6650 0x55a3d72a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6857fa640 10 client.16353 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.835+0000 7ff6a10b5640 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d72a6650 0x55a3d72a8a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c3b43f7dd0f8e4 server_cookie=9e404bcda562c16b in_seq=0 out_seq=0 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.841+0000 7ff6857fa640 1 -- 172.21.15.103:0/1954069167 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3d7190450 con 0x55a3d72a6650 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.841+0000 7ff6857fa640 10 client.16353 handle_command_reply: tid=0 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.841+0000 7ff6a68d9740 1 client.16353 shutdown 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.841+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225680 msgr2=0x55a3d7234420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.841+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225680 0x55a3d7234420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff6980b8c20 tx=0x7ff6980dc040 comp rx=0 tx=0).stop 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 shutdown_connections 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55a3d72a6650 0x55a3d72a8a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a3d722b050 0x55a3d7235920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6857fa640 0 client.16353 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3d7225fc0 0x55a3d7234960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 --2- 172.21.15.103:0/1954069167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a3d7225680 0x55a3d7234420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 >> 172.21.15.103:0/1954069167 conn(0x55a3d7220cf0 msgr2=0x55a3d721fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 shutdown_connections 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 1 -- 172.21.15.103:0/1954069167 wait complete. 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a68d9740 20 client.16353 trim_cache size 0 max 16384 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:39:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:39:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff6880a2c30 msgr2=0x7ff6880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff6880a2c30 0x7ff6880a50f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff694005fd0 tx=0x7ff694004a40 comp rx=0 tx=0).stop 2024-08-19T01:39:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 msgr2=0x7ff69c075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c075cd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff6980314e0 tx=0x7ff698003c80 comp rx=0 tx=0).stop 2024-08-19T01:39:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 shutdown_connections 2024-08-19T01:39:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff6880a2c30 0x7ff6880a50f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff69c103440 0x7ff69c074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c102a80 0x7ff69c074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 --2- 172.21.15.103:0/2128132600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69c101890 0x7ff69c075cd0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.842+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 >> 172.21.15.103:0/2128132600 conn(0x7ff69c0fd010 msgr2=0x7ff69c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.843+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 shutdown_connections 2024-08-19T01:39:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:37.843+0000 7ff6a3b1a640 1 -- 172.21.15.103:0/2128132600 wait complete. 2024-08-19T01:39:38.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:37 smithi146 ceph-mon[22098]: pgmap v454: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:38.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:37 smithi103 ceph-mon[18436]: pgmap v454: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:37 smithi190 ceph-mon[22032]: pgmap v454: 97 pgs: 97 active+clean; 4.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:40.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:39 smithi146 ceph-mon[22098]: pgmap v455: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:39:40.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:39 smithi103 ceph-mon[18436]: pgmap v455: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:39:40.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:39 smithi190 ceph-mon[22032]: pgmap v455: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:39:42.182 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:41 smithi190 ceph-mon[22032]: pgmap v456: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:39:42.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:41 smithi146 ceph-mon[22098]: pgmap v456: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:39:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:41 smithi103 ceph-mon[18436]: pgmap v456: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:39:44.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:43 smithi146 ceph-mon[22098]: pgmap v457: 97 pgs: 97 active+clean; 5.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:39:44.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:43 smithi103 ceph-mon[18436]: pgmap v457: 97 pgs: 97 active+clean; 5.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:39:44.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:43 smithi190 ceph-mon[22032]: pgmap v457: 97 pgs: 97 active+clean; 5.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:39:44.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:39:44.945 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:45.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 -- 172.21.15.103:0/1838559276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404102c20 msgr2=0x7f440410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 --2- 172.21.15.103:0/1838559276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404102c20 0x7f440410d770 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009ca0 tx=0x7f43fc02a950 comp rx=0 tx=0).stop 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 -- 172.21.15.103:0/1838559276 shutdown_connections 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 --2- 172.21.15.103:0/1838559276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404102c20 0x7f440410d770 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 --2- 172.21.15.103:0/1838559276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44041022a0 0x7f44041026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 --2- 172.21.15.103:0/1838559276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404100b50 0x7f4404100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 -- 172.21.15.103:0/1838559276 >> 172.21.15.103:0/1838559276 conn(0x7f44040fc500 msgr2=0x7f44040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 -- 172.21.15.103:0/1838559276 shutdown_connections 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.474+0000 7f440af42640 1 -- 172.21.15.103:0/1838559276 wait complete. 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 Processor -- start 2024-08-19T01:39:45.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 -- start start 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404100b50 0x7f44041980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44041022a0 0x7f4404198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404102c20 0x7f440419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4404110b60 con 0x7f4404100b50 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44041109e0 con 0x7f44041022a0 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f440af42640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4404110ce0 con 0x7f4404102c20 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.475+0000 7f4408cde640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404100b50 0x7f44041980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404100b50 0x7f44041980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45664/0 (socket says 172.21.15.103:45664) 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 -- 172.21.15.103:0/3447648376 learned_addr learned my addr 172.21.15.103:0/3447648376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4403fff640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44041022a0 0x7f4404198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f44094df640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404102c20 0x7f440419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 -- 172.21.15.103:0/3447648376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404102c20 msgr2=0x7f440419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404102c20 0x7f440419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 -- 172.21.15.103:0/3447648376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44041022a0 msgr2=0x7f4404198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44041022a0 0x7f4404198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 -- 172.21.15.103:0/3447648376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440419d090 con 0x7f4404100b50 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4408cde640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404100b50 0x7f44041980e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f43f800d770 tx=0x7f43f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4401ffb640 1 -- 172.21.15.103:0/3447648376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8017070 con 0x7f4404100b50 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440419d380 con 0x7f4404100b50 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4401ffb640 1 -- 172.21.15.103:0/3447648376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43f80054d0 con 0x7f4404100b50 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.476+0000 7f4401ffb640 1 -- 172.21.15.103:0/3447648376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8004cf0 con 0x7f4404100b50 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.477+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44041a4c60 con 0x7f4404100b50 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 Processor -- start 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 -- start start 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f4401ffb640 1 -- 172.21.15.103:0/3447648376 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f43f8004e90 con 0x7f4404100b50 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5210 0x558bbe2a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558bbe20a980 con 0x558bbe2a5210 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558bbe1a2a70 con 0x558bbe2a5b30 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f440d340740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558bbe206940 con 0x558bbe2b03f0 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f44094df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5210 0x558bbe2a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f44094df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5210 0x558bbe2a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45676/0 (socket says 172.21.15.103:45676) 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.479+0000 7f44094df640 1 -- 172.21.15.103:0/1443705153 learned_addr learned my addr 172.21.15.103:0/1443705153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f4408cde640 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f4403fff640 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f44094df640 1 -- 172.21.15.103:0/1443705153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 msgr2=0x558bbe2b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f44094df640 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f44094df640 1 -- 172.21.15.103:0/1443705153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 msgr2=0x558bbe2afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f44094df640 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f44094df640 1 -- 172.21.15.103:0/1443705153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbe2056e0 con 0x558bbe2a5210 2024-08-19T01:39:45.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f44094df640 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5210 0x558bbe2a55f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f43fc035000 tx=0x7f43fc02fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c849ebefb91d1faa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f43eeffd640 1 -- 172.21.15.103:0/1443705153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc048070 con 0x558bbe2a5210 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.480+0000 7f43eeffd640 1 -- 172.21.15.103:0/1443705153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43fc02fa00 con 0x558bbe2a5210 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f43eeffd640 1 -- 172.21.15.103:0/1443705153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc034040 con 0x558bbe2a5210 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f4401ffb640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f43f40a2b80 0x7f43f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f4408cde640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f43f40a2b80 0x7f43f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f4401ffb640 1 -- 172.21.15.103:0/3447648376 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f43f80ce9d0 con 0x7f4404100b50 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f4408cde640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f43f40a2b80 0x7f43f40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43f80cc870 tx=0x7f43f801d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 -- 172.21.15.103:0/1443705153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5210 msgr2=0x558bbe2a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5210 0x558bbe2a55f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f43fc035000 tx=0x7f43fc02fe40 comp rx=0 tx=0).stop 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 -- 172.21.15.103:0/1443705153 shutdown_connections 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 --2- 172.21.15.103:0/1443705153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5210 0x558bbe2a55f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 -- 172.21.15.103:0/1443705153 >> 172.21.15.103:0/1443705153 conn(0x558bbe2a17a0 msgr2=0x558bbe2a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 -- 172.21.15.103:0/1443705153 shutdown_connections 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.481+0000 7f440d340740 1 -- 172.21.15.103:0/1443705153 wait complete. 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 -- start start 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 Processor -- start 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe2addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558bbe2b2fc0 con 0x558bbe2b03f0 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558bbe1c5aa0 con 0x558bbe2a5b30 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558bbe2b3140 con 0x558bbe2a5210 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.482+0000 7f440d340740 10 client.0 _ll_register_callbacks cb 0x558bbe1b4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe2addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe2addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45686/0 (socket says 172.21.15.103:45686) 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 -- 172.21.15.103:0/4290581040 learned_addr learned my addr 172.21.15.103:0/4290581040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f4403fff640 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 -- 172.21.15.103:0/4290581040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 msgr2=0x558bbe2ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 -- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 msgr2=0x558bbe2ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 -- 172.21.15.103:0/4290581040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbe1a9650 con 0x558bbe2b03f0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f44094df640 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe2addf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009ca0 tx=0x7f43fc003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ce554ddb4cfd78 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f43e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f43e3fff640 1 -- 172.21.15.103:0/4290581040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc048070 con 0x558bbe2b03f0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f43e3fff640 1 -- 172.21.15.103:0/4290581040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f43fc004580 con 0x558bbe2b03f0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43fc02fa00 con 0x558bbe2b03f0 2024-08-19T01:39:45.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.483+0000 7f43e3fff640 1 -- 172.21.15.103:0/4290581040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f43fc004780 con 0x558bbe2b03f0 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 10 client.16365 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558bbdf7c3c0 con 0x558bbe2b03f0 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f43e3fff640 1 -- 172.21.15.103:0/4290581040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43fc02fa00 con 0x558bbe2b03f0 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f43e3fff640 1 -- 172.21.15.103:0/4290581040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f43fc030bb0 con 0x558bbe2b03f0 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 10 client.16365 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 20 client.16365 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe2bfdf0 0x558bbe2c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 4 client.16365 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bbe217ef0 con 0x558bbe2bfdf0 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.484+0000 7f4403fff640 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe2bfdf0 0x558bbe2c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.485+0000 7f4403fff640 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe2bfdf0 0x558bbe2c21e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aeed352bf2d4ebe server_cookie=91c6d50de282211f in_seq=0 out_seq=0 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.485+0000 7f43e3fff640 10 client.16365 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:45.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.485+0000 7f43e3fff640 1 -- 172.21.15.103:0/4290581040 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558bbe217ef0 con 0x558bbe2bfdf0 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.485+0000 7f43e3fff640 10 client.16365 handle_command_reply: tid=0 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.485+0000 7f440d340740 1 client.16365 shutdown 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 msgr2=0x558bbe2addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe2addf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009ca0 tx=0x7f43fc003ee0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 shutdown_connections 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe2bfdf0 0x558bbe2c21e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe2addf0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 --2- 172.21.15.103:0/4290581040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 >> 172.21.15.103:0/4290581040 conn(0x558bbe2a17a0 msgr2=0x558bbe2a74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f43e3fff640 0 client.16365 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 shutdown_connections 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 1 -- 172.21.15.103:0/4290581040 wait complete. 2024-08-19T01:39:45.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.486+0000 7f440d340740 20 client.16365 trim_cache size 0 max 16384 2024-08-19T01:39:45.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 Processor -- start 2024-08-19T01:39:45.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 -- start start 2024-08-19T01:39:45.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5210 0x558bbe2af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5b30 0x558bbe2c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44041ad040 con 0x558bbe2a5b30 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44041aeca0 con 0x558bbe2a5210 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f440d340740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f44041ad710 con 0x558bbe2b03f0 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5b30 0x558bbe2c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5b30 0x558bbe2c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45700/0 (socket says 172.21.15.103:45700) 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 -- 172.21.15.103:0/141383633 learned_addr learned my addr 172.21.15.103:0/141383633 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f44094df640 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5210 0x558bbe2af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f44094df640 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 -- 172.21.15.103:0/141383633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 msgr2=0x558bbe2c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 -- 172.21.15.103:0/141383633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5210 msgr2=0x558bbe2af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5210 0x558bbe2af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.504+0000 7f4403fff640 1 -- 172.21.15.103:0/141383633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbe2ab8c0 con 0x558bbe2a5b30 2024-08-19T01:39:45.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.505+0000 7f4403fff640 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5b30 0x558bbe2c4e20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x558bbe2a2b20 tx=0x7f43f0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6a07c62ea130a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.505+0000 7f43ee7fc640 1 -- 172.21.15.103:0/141383633 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f00130d0 con 0x558bbe2a5b30 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.505+0000 7f43ee7fc640 1 -- 172.21.15.103:0/141383633 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43f0013270 con 0x558bbe2a5b30 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.505+0000 7f43ee7fc640 1 -- 172.21.15.103:0/141383633 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f001c460 con 0x558bbe2a5b30 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 -- 172.21.15.103:0/141383633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5b30 msgr2=0x558bbe2c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5b30 0x558bbe2c4e20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x558bbe2a2b20 tx=0x7f43f0013830 comp rx=0 tx=0).stop 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 -- 172.21.15.103:0/141383633 shutdown_connections 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2b03f0 0x558bbe2c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2a5b30 0x558bbe2c4e20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 --2- 172.21.15.103:0/141383633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5210 0x558bbe2af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 -- 172.21.15.103:0/141383633 >> 172.21.15.103:0/141383633 conn(0x558bbe2a17a0 msgr2=0x558bbe2aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 -- 172.21.15.103:0/141383633 shutdown_connections 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 -- 172.21.15.103:0/141383633 wait complete. 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 -- start start 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.506+0000 7f440d340740 1 Processor -- start 2024-08-19T01:39:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f440d340740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f440d340740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f440d340740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe3216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f440d340740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44041ae510 con 0x558bbe2b03f0 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f440d340740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44041adde0 con 0x558bbe2a5b30 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f440d340740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f44041ab040 con 0x558bbe2a5210 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f440d340740 10 client.0 _ll_register_callbacks cb 0x7f43fc046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe3216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe3216e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45702/0 (socket says 172.21.15.103:45702) 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 -- 172.21.15.103:0/1661724709 learned_addr learned my addr 172.21.15.103:0/1661724709 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f4403fff640 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 -- 172.21.15.103:0/1661724709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 msgr2=0x558bbe2c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2c26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 -- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 msgr2=0x558bbe2c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 -- 172.21.15.103:0/1661724709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbe321d80 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f44094df640 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe3216e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f43fc002230 tx=0x7f43fc02f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcacf06ac907b2c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.507+0000 7f43e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f43e3fff640 1 -- 172.21.15.103:0/1661724709 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc048070 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f43e3fff640 1 -- 172.21.15.103:0/1661724709 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f43fc040040 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f0013270 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f43e3fff640 1 -- 172.21.15.103:0/1661724709 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f43fc03a5a0 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 10 client.16371 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558bbe32a300 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f43e3fff640 1 -- 172.21.15.103:0/1661724709 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43f0013270 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f43e3fff640 1 -- 172.21.15.103:0/1661724709 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f43fc0305d0 con 0x558bbe2b03f0 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 10 client.16371 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 20 client.16371 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:45.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe32aa50 0x558bbe32ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:45.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 4 client.16371 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:39:45.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.508+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558bbe212400 con 0x558bbe32aa50 2024-08-19T01:39:45.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.509+0000 7f4403fff640 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe32aa50 0x558bbe32ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:45.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.509+0000 7f43e3fff640 10 client.16371 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:45.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.509+0000 7f4403fff640 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe32aa50 0x558bbe32ce70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2032f9cf676a11 server_cookie=765b2cbede426ddc in_seq=0 out_seq=0 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.509+0000 7f43e3fff640 1 -- 172.21.15.103:0/1661724709 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558bbe212400 con 0x558bbe32aa50 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.509+0000 7f43e3fff640 10 client.16371 handle_command_reply: tid=0 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.509+0000 7f440d340740 1 client.16371 shutdown 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 msgr2=0x558bbe3216e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe3216e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f43fc002230 tx=0x7f43fc02f3e0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 shutdown_connections 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558bbe32aa50 0x558bbe32ce70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbe2b03f0 0x558bbe3216e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f43e3fff640 0 client.16371 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558bbe2a5b30 0x558bbe2c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 --2- 172.21.15.103:0/1661724709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bbe2a5210 0x558bbe2c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 >> 172.21.15.103:0/1661724709 conn(0x558bbe2a17a0 msgr2=0x558bbe2adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 shutdown_connections 2024-08-19T01:39:45.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 1 -- 172.21.15.103:0/1661724709 wait complete. 2024-08-19T01:39:45.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.510+0000 7f440d340740 20 client.16371 trim_cache size 0 max 16384 2024-08-19T01:39:45.517 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:39:45.517 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:39:45.517 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:39:45.517 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f43f40a2b80 msgr2=0x7f43f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f43f40a2b80 0x7f43f40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43f80cc870 tx=0x7f43f801d040 comp rx=0 tx=0).stop 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404100b50 msgr2=0x7f44041980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404100b50 0x7f44041980e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f43f800d770 tx=0x7f43f800dc30 comp rx=0 tx=0).stop 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 shutdown_connections 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f43f40a2b80 0x7f43f40a5040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404102c20 0x7f440419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f44041022a0 0x7f4404198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 --2- 172.21.15.103:0/3447648376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4404100b50 0x7f44041980e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:45.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 >> 172.21.15.103:0/3447648376 conn(0x7f44040fc500 msgr2=0x7f4404069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:45.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 shutdown_connections 2024-08-19T01:39:45.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:45.511+0000 7f440af42640 1 -- 172.21.15.103:0/3447648376 wait complete. 2024-08-19T01:39:46.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:45 smithi146 ceph-mon[22098]: pgmap v458: 97 pgs: 97 active+clean; 5.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:39:46.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:45 smithi103 ceph-mon[18436]: pgmap v458: 97 pgs: 97 active+clean; 5.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:39:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:45 smithi190 ceph-mon[22032]: pgmap v458: 97 pgs: 97 active+clean; 5.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:39:46.492 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:39:46.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:39:46.780 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:47.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 -- 172.21.15.103:0/949496037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac81007d0 msgr2=0x7fdac810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:47.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 --2- 172.21.15.103:0/949496037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac81007d0 0x7fdac810e750 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009ca0 tx=0x7fdac002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 -- 172.21.15.103:0/949496037 shutdown_connections 2024-08-19T01:39:47.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 --2- 172.21.15.103:0/949496037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac81007d0 0x7fdac810e750 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 --2- 172.21.15.103:0/949496037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdac80ffe30 0x7fdac8100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 --2- 172.21.15.103:0/949496037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac81068b0 0x7fdac8106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 -- 172.21.15.103:0/949496037 >> 172.21.15.103:0/949496037 conn(0x7fdac8076f60 msgr2=0x7fdac80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 -- 172.21.15.103:0/949496037 shutdown_connections 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.280+0000 7fdacf849640 1 -- 172.21.15.103:0/949496037 wait complete. 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 Processor -- start 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 -- start start 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac80ffe30 0x7fdac81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdac81007d0 0x7fdac8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac81068b0 0x7fdac819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8111c30 con 0x7fdac80ffe30 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8111ab0 con 0x7fdac81007d0 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacf849640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8111db0 con 0x7fdac81068b0 2024-08-19T01:39:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacd5e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac80ffe30 0x7fdac81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacd5e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac80ffe30 0x7fdac81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45718/0 (socket says 172.21.15.103:45718) 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacd5e5640 1 -- 172.21.15.103:0/1441079505 learned_addr learned my addr 172.21.15.103:0/1441079505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdaccde4640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdac81007d0 0x7fdac8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacdde6640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac81068b0 0x7fdac819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.281+0000 7fdacd5e5640 1 -- 172.21.15.103:0/1441079505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac81068b0 msgr2=0x7fdac819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdacd5e5640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac81068b0 0x7fdac819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdacd5e5640 1 -- 172.21.15.103:0/1441079505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdac81007d0 msgr2=0x7fdac8198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdacd5e5640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdac81007d0 0x7fdac8198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdacd5e5640 1 -- 172.21.15.103:0/1441079505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac819d0a0 con 0x7fdac80ffe30 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdaccde4640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdac81007d0 0x7fdac8198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdacd5e5640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac80ffe30 0x7fdac81980f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c890 tx=0x7fdac400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdabe7fc640 1 -- 172.21.15.103:0/1441079505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4004220 con 0x7fdac80ffe30 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdabe7fc640 1 -- 172.21.15.103:0/1441079505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdac40043c0 con 0x7fdac80ffe30 2024-08-19T01:39:47.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdabe7fc640 1 -- 172.21.15.103:0/1441079505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4004a00 con 0x7fdac80ffe30 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac819d390 con 0x7fdac80ffe30 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.282+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a4c70 con 0x7fdac80ffe30 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.283+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac8100290 con 0x7fdac80ffe30 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.284+0000 7fdabe7fc640 1 -- 172.21.15.103:0/1441079505 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdac4004ba0 con 0x7fdac80ffe30 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.285+0000 7fdabe7fc640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdab40a2b80 0x7fdab40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.285+0000 7fdaccde4640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdab40a2b80 0x7fdab40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.285+0000 7fdabe7fc640 1 -- 172.21.15.103:0/1441079505 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdac40ced80 con 0x7fdac80ffe30 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.286+0000 7fdaccde4640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdab40a2b80 0x7fdab40a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdac80ff520 tx=0x7fdab8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:47.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.290+0000 7fdabe7fc640 1 -- 172.21.15.103:0/1441079505 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdac4014030 con 0x7fdac80ffe30 2024-08-19T01:39:47.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.549+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdac8109e10 con 0x7fdac80ffe30 2024-08-19T01:39:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.550+0000 7fdabe7fc640 1 -- 172.21.15.103:0/1441079505 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11999 (secure 0 0 0) 0x7fdac409bd10 con 0x7fdac80ffe30 2024-08-19T01:39:47.552 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:39:47.552 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":32,"btime":"2024-08-19T01:39:31:934300+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:39:31.568876+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:39:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 32 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdab40a2b80 msgr2=0x7fdab40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdab40a2b80 0x7fdab40a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdac80ff520 tx=0x7fdab8005f90 comp rx=0 tx=0).stop 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac80ffe30 msgr2=0x7fdac81980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac80ffe30 0x7fdac81980f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c890 tx=0x7fdac400cd50 comp rx=0 tx=0).stop 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 shutdown_connections 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdab40a2b80 0x7fdab40a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdac81068b0 0x7fdac819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdac81007d0 0x7fdac8198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 --2- 172.21.15.103:0/1441079505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdac80ffe30 0x7fdac81980f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 >> 172.21.15.103:0/1441079505 conn(0x7fdac8076f60 msgr2=0x7fdac8104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 shutdown_connections 2024-08-19T01:39:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:47.554+0000 7fdacf849640 1 -- 172.21.15.103:0/1441079505 wait complete. 2024-08-19T01:39:48.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:47 smithi146 ceph-mon[22098]: pgmap v459: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:39:48.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:47 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1441079505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:39:48.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:47 smithi103 ceph-mon[18436]: pgmap v459: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:39:48.224 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:47 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1441079505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:39:48.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:39:48.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:47 smithi190 ceph-mon[22032]: pgmap v459: 97 pgs: 97 active+clean; 5.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-08-19T01:39:48.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:47 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1441079505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:39:48.543 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 -- 172.21.15.103:0/2135897795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc102c20 msgr2=0x7f50dc10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 --2- 172.21.15.103:0/2135897795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc102c20 0x7f50dc10d770 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009ca0 tx=0x7f50d402a950 comp rx=0 tx=0).stop 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 -- 172.21.15.103:0/2135897795 shutdown_connections 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 --2- 172.21.15.103:0/2135897795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc102c20 0x7f50dc10d770 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 --2- 172.21.15.103:0/2135897795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50dc1022a0 0x7f50dc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 --2- 172.21.15.103:0/2135897795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50dc100b50 0x7f50dc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 -- 172.21.15.103:0/2135897795 >> 172.21.15.103:0/2135897795 conn(0x7f50dc0fc500 msgr2=0x7f50dc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 -- 172.21.15.103:0/2135897795 shutdown_connections 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.061+0000 7f50e39ae640 1 -- 172.21.15.103:0/2135897795 wait complete. 2024-08-19T01:39:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.062+0000 7f50e39ae640 1 Processor -- start 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.062+0000 7f50e39ae640 1 -- start start 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e39ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50dc100b50 0x7f50dc198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e39ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc1022a0 0x7f50dc199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e39ae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50dc102c20 0x7f50dc1a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e39ae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc110b60 con 0x7f50dc1022a0 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e39ae640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1109e0 con 0x7f50dc102c20 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e39ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc110ce0 con 0x7f50dc100b50 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc1022a0 0x7f50dc199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc1022a0 0x7f50dc199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45734/0 (socket says 172.21.15.103:45734) 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 -- 172.21.15.103:0/547559983 learned_addr learned my addr 172.21.15.103:0/547559983 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e174a640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50dc100b50 0x7f50dc198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50dc102c20 0x7f50dc1a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 -- 172.21.15.103:0/547559983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50dc100b50 msgr2=0x7f50dc198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50dc100b50 0x7f50dc198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 -- 172.21.15.103:0/547559983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50dc102c20 msgr2=0x7f50dc1a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50dc102c20 0x7f50dc1a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 -- 172.21.15.103:0/547559983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc1a4590 con 0x7f50dc1022a0 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50dc102c20 0x7f50dc1a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50e0f49640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc1022a0 0x7f50dc199510 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00d950 tx=0x7f50cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.063+0000 7f50d27fc640 1 -- 172.21.15.103:0/547559983 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc017070 con 0x7f50dc1022a0 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.064+0000 7f50d27fc640 1 -- 172.21.15.103:0/547559983 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50cc009c30 con 0x7f50dc1022a0 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.064+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc1a4760 con 0x7f50dc1022a0 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.064+0000 7f50d27fc640 1 -- 172.21.15.103:0/547559983 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00f620 con 0x7f50dc1022a0 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.064+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc197040 con 0x7f50dc1022a0 2024-08-19T01:39:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.066+0000 7f50d27fc640 1 -- 172.21.15.103:0/547559983 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f50cc0109a0 con 0x7f50dc1022a0 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 Processor -- start 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 -- start start 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea561ff0 0x55dfea5623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea56cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50d27fc640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50c80a2c00 0x7f50c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea56d1d0 0x55dfea56f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dfea4c7a40 con 0x55dfea56d1d0 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55dfea45f9d0 con 0x55dfea562910 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e5dac740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dfea4c3a00 con 0x55dfea561ff0 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e1f4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea56d1d0 0x55dfea56f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e0f49640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50c80a2c00 0x7f50c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e1f4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea56d1d0 0x55dfea56f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45742/0 (socket says 172.21.15.103:45742) 2024-08-19T01:39:49.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e1f4b640 1 -- 172.21.15.103:0/731072616 learned_addr learned my addr 172.21.15.103:0/731072616 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea561ff0 0x55dfea5623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50e174a640 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea56cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.067+0000 7f50d27fc640 1 -- 172.21.15.103:0/547559983 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f50cc0cebf0 con 0x7f50dc1022a0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e1f4b640 1 -- 172.21.15.103:0/731072616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea561ff0 msgr2=0x55dfea5623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea561ff0 0x55dfea5623d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e1f4b640 1 -- 172.21.15.103:0/731072616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 msgr2=0x55dfea56cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea56cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e1f4b640 1 -- 172.21.15.103:0/731072616 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfea466650 con 0x55dfea56d1d0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e174a640 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea56cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e0f49640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50c80a2c00 0x7f50c80a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0978e0 tx=0x7f50cc090960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea56d1d0 0x55dfea56f5c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009430 tx=0x7f50d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ee636773c75ca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50c77fe640 1 -- 172.21.15.103:0/731072616 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4034040 con 0x55dfea56d1d0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50c77fe640 1 -- 172.21.15.103:0/731072616 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d4003970 con 0x55dfea56d1d0 2024-08-19T01:39:49.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.068+0000 7f50c77fe640 1 -- 172.21.15.103:0/731072616 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4004290 con 0x55dfea56d1d0 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 -- 172.21.15.103:0/731072616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea56d1d0 msgr2=0x55dfea56f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea56d1d0 0x55dfea56f5c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009430 tx=0x7f50d4004a40 comp rx=0 tx=0).stop 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 -- 172.21.15.103:0/731072616 shutdown_connections 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea56d1d0 0x55dfea56f5c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea56cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 --2- 172.21.15.103:0/731072616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea561ff0 0x55dfea5623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 -- 172.21.15.103:0/731072616 >> 172.21.15.103:0/731072616 conn(0x55dfea55e580 msgr2=0x55dfea55e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 -- 172.21.15.103:0/731072616 shutdown_connections 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 -- 172.21.15.103:0/731072616 wait complete. 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.069+0000 7f50e5dac740 1 -- start start 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 1 Processor -- start 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea56a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea56a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea56aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dfea56fda0 con 0x55dfea562910 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55dfea482b20 con 0x55dfea561ff0 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dfea56ff20 con 0x55dfea56d1d0 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e5dac740 10 client.0 _ll_register_callbacks cb 0x55dfea480bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea56a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea56a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45750/0 (socket says 172.21.15.103:45750) 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e174a640 1 -- 172.21.15.103:0/2157346147 learned_addr learned my addr 172.21.15.103:0/2157346147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.070+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea56a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea56aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e174a640 1 -- 172.21.15.103:0/2157346147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 msgr2=0x55dfea56aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e174a640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea56aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e174a640 1 -- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 msgr2=0x55dfea56a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e174a640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea56a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e174a640 1 -- 172.21.15.103:0/2157346147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfea2393c0 con 0x55dfea562910 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea56a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e174a640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea56a5b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f50d800be80 tx=0x7f50d80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8996467babab68c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50c4ff9640 1 -- 172.21.15.103:0/2157346147 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8012650 con 0x55dfea562910 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d4003970 con 0x55dfea562910 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50c4ff9640 1 -- 172.21.15.103:0/2157346147 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f50d80127f0 con 0x55dfea562910 2024-08-19T01:39:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50c4ff9640 1 -- 172.21.15.103:0/2157346147 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f50d80129f0 con 0x55dfea562910 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e5dac740 10 client.16389 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dfea57ba00 con 0x55dfea562910 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.071+0000 7f50c4ff9640 1 -- 172.21.15.103:0/2157346147 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d4003970 con 0x55dfea562910 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50c4ff9640 1 -- 172.21.15.103:0/2157346147 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f50d801bc50 con 0x55dfea562910 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50e5dac740 10 client.16389 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50e5dac740 10 client.16389 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50e5dac740 20 client.16389 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea57cec0 0x55dfea57f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50e5dac740 4 client.16389 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfea4d4fb0 con 0x55dfea57cec0 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.072+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea57cec0 0x55dfea57f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea57cec0 0x55dfea57f2f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae393228a135f91b server_cookie=f9c13ccb13fb4600 in_seq=0 out_seq=0 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50c4ff9640 10 client.16389 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50c4ff9640 1 -- 172.21.15.103:0/2157346147 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfea4d4fb0 con 0x55dfea57cec0 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50c4ff9640 10 client.16389 handle_command_reply: tid=0 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 client.16389 shutdown 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 msgr2=0x55dfea56a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea56a5b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f50d800be80 tx=0x7f50d80117b0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 shutdown_connections 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea57cec0 0x55dfea57f2f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea56aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea56a5b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2157346147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea56a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50c4ff9640 0 client.16389 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.073+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 >> 172.21.15.103:0/2157346147 conn(0x55dfea55e580 msgr2=0x55dfea564260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.074+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 shutdown_connections 2024-08-19T01:39:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.074+0000 7f50e5dac740 1 -- 172.21.15.103:0/2157346147 wait complete. 2024-08-19T01:39:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.074+0000 7f50e5dac740 20 client.16389 trim_cache size 0 max 16384 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 Processor -- start 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 -- start start 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea569050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea584c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea585140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1af3f0 con 0x55dfea562910 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b1050 con 0x55dfea561ff0 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e5dac740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1afac0 con 0x55dfea56d1d0 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea584c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea584c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45762/0 (socket says 172.21.15.103:45762) 2024-08-19T01:39:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 -- 172.21.15.103:0/2054103729 learned_addr learned my addr 172.21.15.103:0/2054103729 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea569050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea585140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 -- 172.21.15.103:0/2054103729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 msgr2=0x55dfea585140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea585140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 -- 172.21.15.103:0/2054103729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 msgr2=0x55dfea569050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea569050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e174a640 1 -- 172.21.15.103:0/2054103729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfea55e3b0 con 0x55dfea562910 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.088+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea569050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.089+0000 7f50e174a640 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea584c00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f50d8026570 tx=0x7f50d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9057a93494ef2647 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.089+0000 7f50c6ffd640 1 -- 172.21.15.103:0/2054103729 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8026760 con 0x55dfea562910 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.089+0000 7f50c6ffd640 1 -- 172.21.15.103:0/2054103729 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d8026900 con 0x55dfea562910 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.089+0000 7f50c6ffd640 1 -- 172.21.15.103:0/2054103729 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d80052a0 con 0x55dfea562910 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 -- 172.21.15.103:0/2054103729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 msgr2=0x55dfea584c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea584c00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f50d8026570 tx=0x7f50d8000bc0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 -- 172.21.15.103:0/2054103729 shutdown_connections 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea585140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea562910 0x55dfea584c00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 --2- 172.21.15.103:0/2054103729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea561ff0 0x55dfea569050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 -- 172.21.15.103:0/2054103729 >> 172.21.15.103:0/2054103729 conn(0x55dfea55e580 msgr2=0x55dfea568940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 -- 172.21.15.103:0/2054103729 shutdown_connections 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 -- 172.21.15.103:0/2054103729 wait complete. 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 -- start start 2024-08-19T01:39:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.090+0000 7f50e5dac740 1 Processor -- start 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea561ff0 0x55dfea570160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea5706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e5dac740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea570be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e5dac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b08c0 con 0x55dfea561ff0 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e5dac740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b0190 con 0x55dfea562910 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e5dac740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1ad3f0 con 0x55dfea56d1d0 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e5dac740 10 client.0 _ll_register_callbacks cb 0x7f50d8005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:49.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea561ff0 0x55dfea570160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea561ff0 0x55dfea570160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45770/0 (socket says 172.21.15.103:45770) 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 -- 172.21.15.103:0/3567841526 learned_addr learned my addr 172.21.15.103:0/3567841526 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e174a640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea5706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea570be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 -- 172.21.15.103:0/3567841526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 msgr2=0x55dfea570be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea570be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 -- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 msgr2=0x55dfea5706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea5706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 -- 172.21.15.103:0/3567841526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfea5e5e10 con 0x55dfea561ff0 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e174a640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea5706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50e1f4b640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea561ff0 0x55dfea570160 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f50d4004af0 tx=0x7f50d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8299ef3185e7ca51 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.091+0000 7f50c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:49.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50c4ff9640 1 -- 172.21.15.103:0/3567841526 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4034040 con 0x55dfea561ff0 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d8026900 con 0x55dfea561ff0 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50c4ff9640 1 -- 172.21.15.103:0/3567841526 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f50d402f460 con 0x55dfea561ff0 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50c4ff9640 1 -- 172.21.15.103:0/3567841526 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f50d405d020 con 0x55dfea561ff0 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 10 client.16395 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dfea5e70f0 con 0x55dfea561ff0 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50c4ff9640 1 -- 172.21.15.103:0/3567841526 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50d8026900 con 0x55dfea561ff0 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50c4ff9640 1 -- 172.21.15.103:0/3567841526 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f50d4030440 con 0x55dfea561ff0 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 10 client.16395 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 10 client.16395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 20 client.16395 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea5e7370 0x55dfea5e9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 4 client.16395 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:49.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.092+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfea4cf4c0 con 0x55dfea5e7370 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e174a640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea5e7370 0x55dfea5e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e174a640 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea5e7370 0x55dfea5e9760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2418efa031d4cc93 server_cookie=af6b73dd315c5602 in_seq=0 out_seq=0 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50c4ff9640 10 client.16395 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50c4ff9640 1 -- 172.21.15.103:0/3567841526 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfea4cf4c0 con 0x55dfea5e7370 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50c4ff9640 10 client.16395 handle_command_reply: tid=0 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 client.16395 shutdown 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea561ff0 msgr2=0x55dfea570160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea561ff0 0x55dfea570160 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f50d4004af0 tx=0x7f50d4004a40 comp rx=0 tx=0).stop 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 shutdown_connections 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dfea5e7370 0x55dfea5e9760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dfea56d1d0 0x55dfea570be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfea562910 0x55dfea5706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 --2- 172.21.15.103:0/3567841526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dfea561ff0 0x55dfea570160 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 >> 172.21.15.103:0/3567841526 conn(0x55dfea55e580 msgr2=0x55dfea569160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:49.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.093+0000 7f50c4ff9640 0 client.16395 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:39:49.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 shutdown_connections 2024-08-19T01:39:49.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e5dac740 1 -- 172.21.15.103:0/3567841526 wait complete. 2024-08-19T01:39:49.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e5dac740 20 client.16395 trim_cache size 0 max 16384 2024-08-19T01:39:49.101 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50c80a2c00 msgr2=0x7f50c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50c80a2c00 0x7f50c80a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0978e0 tx=0x7f50cc090960 comp rx=0 tx=0).stop 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc1022a0 msgr2=0x7f50dc199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc1022a0 0x7f50dc199510 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00d950 tx=0x7f50cc00de10 comp rx=0 tx=0).stop 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 shutdown_connections 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f50c80a2c00 0x7f50c80a50c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50dc102c20 0x7f50dc1a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50dc1022a0 0x7f50dc199510 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 --2- 172.21.15.103:0/547559983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50dc100b50 0x7f50dc198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:49.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.094+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 >> 172.21.15.103:0/547559983 conn(0x7f50dc0fc500 msgr2=0x7f50dc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:49.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.095+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 shutdown_connections 2024-08-19T01:39:49.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:49.095+0000 7f50e39ae640 1 -- 172.21.15.103:0/547559983 wait complete. 2024-08-19T01:39:49.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:49 smithi103 ceph-mon[18436]: pgmap v460: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:39:49.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:39:50.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:49 smithi146 ceph-mon[22098]: pgmap v460: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:39:50.242 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:50.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:49 smithi190 ceph-mon[22032]: pgmap v460: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 -- 172.21.15.103:0/758125521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 msgr2=0x7f3768100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 --2- 172.21.15.103:0/758125521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f3768100ac0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3764009ca0 tx=0x7f376402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 -- 172.21.15.103:0/758125521 shutdown_connections 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 --2- 172.21.15.103:0/758125521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f376810e200 0x7f3768110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 --2- 172.21.15.103:0/758125521 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3768101090 0x7f376810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 --2- 172.21.15.103:0/758125521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f3768100ac0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 -- 172.21.15.103:0/758125521 >> 172.21.15.103:0/758125521 conn(0x7f37680fc500 msgr2=0x7f37680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 -- 172.21.15.103:0/758125521 shutdown_connections 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.744+0000 7f376f316640 1 -- 172.21.15.103:0/758125521 wait complete. 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 Processor -- start 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 -- start start 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f376819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3768101090 0x7f376819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376810e200 0x7f376819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3768113e20 con 0x7f37681006e0 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3768113ca0 con 0x7f3768101090 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376f316640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3768113fa0 con 0x7f376810e200 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376d0b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f376819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376d0b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f376819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45776/0 (socket says 172.21.15.103:45776) 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376d0b2640 1 -- 172.21.15.103:0/3825647243 learned_addr learned my addr 172.21.15.103:0/3825647243 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376c8b1640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3768101090 0x7f376819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.745+0000 7f376d8b3640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376810e200 0x7f376819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376c8b1640 1 -- 172.21.15.103:0/3825647243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376810e200 msgr2=0x7f376819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376c8b1640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376810e200 0x7f376819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376c8b1640 1 -- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 msgr2=0x7f376819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376c8b1640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f376819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376c8b1640 1 -- 172.21.15.103:0/3825647243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376819f370 con 0x7f3768101090 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376d0b2640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f376819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376c8b1640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3768101090 0x7f376819a900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f375800e9f0 tx=0x7f375800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f375e7fc640 1 -- 172.21.15.103:0/3825647243 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375800ccc0 con 0x7f3768101090 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376819f660 con 0x7f3768101090 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f375e7fc640 1 -- 172.21.15.103:0/3825647243 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3758005020 con 0x7f3768101090 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f375e7fc640 1 -- 172.21.15.103:0/3825647243 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758004a00 con 0x7f3768101090 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.746+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a6f40 con 0x7f3768101090 2024-08-19T01:39:50.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f375e7fc640 1 -- 172.21.15.103:0/3825647243 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3758004ba0 con 0x7f3768101090 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 Processor -- start 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 -- start start 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09667c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a0966a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a095c2a40 con 0x556a0965cff0 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556a0955a9d0 con 0x556a096681d0 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f3771714740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556a095bea00 con 0x556a0965d910 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f376d8b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f376d8b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45790/0 (socket says 172.21.15.103:45790) 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f376d8b3640 1 -- 172.21.15.103:0/2357786682 learned_addr learned my addr 172.21.15.103:0/2357786682 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f376c8b1640 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a0966a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f376d0b2640 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09667c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f376d8b3640 1 -- 172.21.15.103:0/2357786682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 msgr2=0x556a09667c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f376d8b3640 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09667c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.749+0000 7f375e7fc640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37540a2b80 0x7f37540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f376d8b3640 1 -- 172.21.15.103:0/2357786682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 msgr2=0x556a0966a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f376d8b3640 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a0966a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f376d8b3640 1 -- 172.21.15.103:0/2357786682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a09561650 con 0x556a0965cff0 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f376c8b1640 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a0966a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f376d0b2640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37540a2b80 0x7f37540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f375e7fc640 1 -- 172.21.15.103:0/3825647243 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f37580cefe0 con 0x7f3768101090 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f376d8b3640 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965d3d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3760004620 tx=0x7f376000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f9a2eca68a7e1d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f376d0b2640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37540a2b80 0x7f37540a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3764004ad0 tx=0x7f3764004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f37537fe640 1 -- 172.21.15.103:0/2357786682 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760003ba0 con 0x556a0965cff0 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f37537fe640 1 -- 172.21.15.103:0/2357786682 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3760015070 con 0x556a0965cff0 2024-08-19T01:39:50.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.750+0000 7f37537fe640 1 -- 172.21.15.103:0/2357786682 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37600114a0 con 0x556a0965cff0 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 -- 172.21.15.103:0/2357786682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 msgr2=0x556a0965d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965d3d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3760004620 tx=0x7f376000bb10 comp rx=0 tx=0).stop 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 -- 172.21.15.103:0/2357786682 shutdown_connections 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a0966a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09667c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 --2- 172.21.15.103:0/2357786682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965d3d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 -- 172.21.15.103:0/2357786682 >> 172.21.15.103:0/2357786682 conn(0x556a09659580 msgr2=0x556a09659960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 -- 172.21.15.103:0/2357786682 shutdown_connections 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 -- 172.21.15.103:0/2357786682 wait complete. 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.751+0000 7f3771714740 1 -- start start 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 1 Processor -- start 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 0x556a096650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a096655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a096681d0 0x556a09665b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a0966ada0 con 0x556a096681d0 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556a0957db20 con 0x556a0965cff0 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556a0966af20 con 0x556a0965d910 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f3771714740 10 client.0 _ll_register_callbacks cb 0x556a0957bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f376d8b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a096681d0 0x556a09665b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f376d0b2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 0x556a096650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f376d8b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a096681d0 0x556a09665b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45798/0 (socket says 172.21.15.103:45798) 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f376d8b3640 1 -- 172.21.15.103:0/4265195486 learned_addr learned my addr 172.21.15.103:0/4265195486 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f376c8b1640 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a096655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f376d8b3640 1 -- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 msgr2=0x556a096655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.752+0000 7f376d8b3640 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a096655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f376d8b3640 1 -- 172.21.15.103:0/4265195486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 msgr2=0x556a096650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f376d8b3640 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 0x556a096650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f376d8b3640 1 -- 172.21.15.103:0/4265195486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a093343c0 con 0x556a096681d0 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f376c8b1640 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a096655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f376d8b3640 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a096681d0 0x556a09665b20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f376000b600 tx=0x7f3760005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49260aa19c7108d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3750ff9640 1 -- 172.21.15.103:0/4265195486 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376001a470 con 0x556a096681d0 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3750ff9640 1 -- 172.21.15.103:0/4265195486 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f376001a610 con 0x556a096681d0 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3760015070 con 0x556a096681d0 2024-08-19T01:39:50.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3750ff9640 1 -- 172.21.15.103:0/4265195486 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f376001a810 con 0x556a096681d0 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3771714740 10 client.16407 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556a0967f1e0 con 0x556a096681d0 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.753+0000 7f3750ff9640 1 -- 172.21.15.103:0/4265195486 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3760015070 con 0x556a096681d0 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3750ff9640 1 -- 172.21.15.103:0/4265195486 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3760031420 con 0x556a096681d0 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3771714740 10 client.16407 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3771714740 10 client.16407 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3771714740 20 client.16407 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3771714740 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a09680640 0x556a09682a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3771714740 4 client.16407 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a095cffb0 con 0x556a09680640 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f376d0b2640 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a09680640 0x556a09682a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f3750ff9640 10 client.16407 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.754+0000 7f376d0b2640 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a09680640 0x556a09682a30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d912552da0e69873 server_cookie=6babc3ef84ebe86b in_seq=0 out_seq=0 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3750ff9640 1 -- 172.21.15.103:0/4265195486 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a095cffb0 con 0x556a09680640 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3750ff9640 10 client.16407 handle_command_reply: tid=0 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 client.16407 shutdown 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a096681d0 msgr2=0x556a09665b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a096681d0 0x556a09665b20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f376000b600 tx=0x7f3760005760 comp rx=0 tx=0).stop 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 shutdown_connections 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a09680640 0x556a09682a30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a096681d0 0x556a09665b20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a096655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 --2- 172.21.15.103:0/4265195486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 0x556a096650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3750ff9640 0 client.16407 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:50.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 >> 172.21.15.103:0/4265195486 conn(0x556a09659580 msgr2=0x556a0965f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.755+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 shutdown_connections 2024-08-19T01:39:50.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.756+0000 7f3771714740 1 -- 172.21.15.103:0/4265195486 wait complete. 2024-08-19T01:39:50.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.756+0000 7f3771714740 20 client.16407 trim_cache size 0 max 16384 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.771+0000 7f3771714740 1 Processor -- start 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.771+0000 7f3771714740 1 -- start start 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.771+0000 7f3771714740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.771+0000 7f3771714740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09663f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f3771714740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a09688150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f3771714740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37681aafd0 con 0x556a0965cff0 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f3771714740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37681acc30 con 0x556a096681d0 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f3771714740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ab6a0 con 0x556a0965d910 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45810/0 (socket says 172.21.15.103:45810) 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 -- 172.21.15.103:0/240889238 learned_addr learned my addr 172.21.15.103:0/240889238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376c8b1640 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09663f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 -- 172.21.15.103:0/240889238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 msgr2=0x556a09663f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09663f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 -- 172.21.15.103:0/240889238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 msgr2=0x556a09688150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a09688150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.772+0000 7f376d0b2640 1 -- 172.21.15.103:0/240889238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a09667b50 con 0x556a0965cff0 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.773+0000 7f376d0b2640 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965fa50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f376402ade0 tx=0x7f3764044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b599a0589b786a8c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.773+0000 7f3752ffd640 1 -- 172.21.15.103:0/240889238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376405d070 con 0x556a0965cff0 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.773+0000 7f3752ffd640 1 -- 172.21.15.103:0/240889238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3764036050 con 0x556a0965cff0 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.773+0000 7f3752ffd640 1 -- 172.21.15.103:0/240889238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37640309e0 con 0x556a0965cff0 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 -- 172.21.15.103:0/240889238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 msgr2=0x556a0965fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965fa50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f376402ade0 tx=0x7f3764044000 comp rx=0 tx=0).stop 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 -- 172.21.15.103:0/240889238 shutdown_connections 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a096681d0 0x556a09688150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a0965d910 0x556a09663f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 --2- 172.21.15.103:0/240889238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965cff0 0x556a0965fa50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 -- 172.21.15.103:0/240889238 >> 172.21.15.103:0/240889238 conn(0x556a09659580 msgr2=0x556a09663840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 -- 172.21.15.103:0/240889238 shutdown_connections 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 -- 172.21.15.103:0/240889238 wait complete. 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.774+0000 7f3771714740 1 -- start start 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 1 Processor -- start 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 0x556a09673f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965d910 0x556a096744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a096681d0 0x556a096e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ac4a0 con 0x556a0965d910 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37681abd70 con 0x556a0965cff0 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37681a8fb0 con 0x556a096681d0 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f3771714740 10 client.0 _ll_register_callbacks cb 0x7f3760004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:50.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965d910 0x556a096744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965d910 0x556a096744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45818/0 (socket says 172.21.15.103:45818) 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 -- 172.21.15.103:0/3272572153 learned_addr learned my addr 172.21.15.103:0/3272572153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 -- 172.21.15.103:0/3272572153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a096681d0 msgr2=0x556a096e9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a096681d0 0x556a096e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 -- 172.21.15.103:0/3272572153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 msgr2=0x556a09673f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 0x556a09673f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.775+0000 7f376c8b1640 1 -- 172.21.15.103:0/3272572153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a096e9700 con 0x556a0965d910 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f376c8b1640 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965d910 0x556a096744d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f375800c6a0 tx=0x7f37580974b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c635cd3b5395dfdb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3750ff9640 1 -- 172.21.15.103:0/3272572153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375801c040 con 0x556a0965d910 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3750ff9640 1 -- 172.21.15.103:0/3272572153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f37580051d0 con 0x556a0965d910 2024-08-19T01:39:50.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3764036050 con 0x556a0965d910 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3750ff9640 1 -- 172.21.15.103:0/3272572153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f375809b810 con 0x556a0965d910 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3771714740 10 client.16413 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556a096ea9c0 con 0x556a0965d910 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.776+0000 7f3750ff9640 1 -- 172.21.15.103:0/3272572153 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3764036050 con 0x556a0965d910 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f3750ff9640 1 -- 172.21.15.103:0/3272572153 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f37580a3ff0 con 0x556a0965d910 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f3771714740 10 client.16413 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f3771714740 10 client.16413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f3771714740 20 client.16413 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f3771714740 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a096eb170 0x556a096ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f3771714740 4 client.16413 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a095ca4c0 con 0x556a096eb170 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.777+0000 7f376d0b2640 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a096eb170 0x556a096ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.778+0000 7f376d0b2640 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a096eb170 0x556a096ed590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c24e63864799ae88 server_cookie=f99040ecf4f2afd in_seq=0 out_seq=0 2024-08-19T01:39:50.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.778+0000 7f3750ff9640 10 client.16413 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.778+0000 7f3750ff9640 1 -- 172.21.15.103:0/3272572153 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a095ca4c0 con 0x556a096eb170 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.778+0000 7f3750ff9640 10 client.16413 handle_command_reply: tid=0 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.778+0000 7f3771714740 1 client.16413 shutdown 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965d910 msgr2=0x556a096744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965d910 0x556a096744d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f375800c6a0 tx=0x7f37580974b0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 shutdown_connections 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x556a096eb170 0x556a096ed590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556a096681d0 0x556a096e9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3750ff9640 0 client.16413 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a0965d910 0x556a096744d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 --2- 172.21.15.103:0/3272572153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556a0965cff0 0x556a09673f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 >> 172.21.15.103:0/3272572153 conn(0x556a09659580 msgr2=0x556a09664060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 shutdown_connections 2024-08-19T01:39:50.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 1 -- 172.21.15.103:0/3272572153 wait complete. 2024-08-19T01:39:50.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.779+0000 7f3771714740 20 client.16413 trim_cache size 0 max 16384 2024-08-19T01:39:50.789 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:50.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37540a2b80 msgr2=0x7f37540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37540a2b80 0x7f37540a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3764004ad0 tx=0x7f3764004a40 comp rx=0 tx=0).stop 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3768101090 msgr2=0x7f376819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3768101090 0x7f376819a900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f375800e9f0 tx=0x7f375800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 shutdown_connections 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37540a2b80 0x7f37540a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f376810e200 0x7f376819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3768101090 0x7f376819a900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 --2- 172.21.15.103:0/3825647243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37681006e0 0x7f376819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 >> 172.21.15.103:0/3825647243 conn(0x7f37680fc500 msgr2=0x7f37680fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 shutdown_connections 2024-08-19T01:39:50.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:50.780+0000 7f376f316640 1 -- 172.21.15.103:0/3825647243 wait complete. 2024-08-19T01:39:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 -- 172.21.15.103:0/4026176572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 msgr2=0x7fdb6c107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 --2- 172.21.15.103:0/4026176572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c107010 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009900 tx=0x7fdb5402a410 comp rx=0 tx=0).stop 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 -- 172.21.15.103:0/4026176572 shutdown_connections 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 --2- 172.21.15.103:0/4026176572 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb6c107550 0x7fdb6c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 --2- 172.21.15.103:0/4026176572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c107010 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 --2- 172.21.15.103:0/4026176572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c1022f0 0x7fdb6c1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 -- 172.21.15.103:0/4026176572 >> 172.21.15.103:0/4026176572 conn(0x7fdb6c0fbf10 msgr2=0x7fdb6c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 -- 172.21.15.103:0/4026176572 shutdown_connections 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.938+0000 7fdb7130c640 1 -- 172.21.15.103:0/4026176572 wait complete. 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 Processor -- start 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 -- start start 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c1022f0 0x7fdb6c101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb6c107550 0x7fdb6c1007e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c10d3d0 con 0x7fdb6c104c20 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c10d250 con 0x7fdb6c107550 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb7130c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c10d550 con 0x7fdb6c1022f0 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb63fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb63fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c1002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45842/0 (socket says 172.21.15.103:45842) 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb63fff640 1 -- 172.21.15.103:0/3363119522 learned_addr learned my addr 172.21.15.103:0/3363119522 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb6affd640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c1022f0 0x7fdb6c101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb6c107550 0x7fdb6c1007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb63fff640 1 -- 172.21.15.103:0/3363119522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c1022f0 msgr2=0x7fdb6c101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb63fff640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c1022f0 0x7fdb6c101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb63fff640 1 -- 172.21.15.103:0/3363119522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb6c107550 msgr2=0x7fdb6c1007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.939+0000 7fdb63fff640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb6c107550 0x7fdb6c1007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.940+0000 7fdb63fff640 1 -- 172.21.15.103:0/3363119522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c101070 con 0x7fdb6c104c20 2024-08-19T01:39:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.940+0000 7fdb63fff640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c1002a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdb540314e0 tx=0x7fdb54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.940+0000 7fdb68ff9640 1 -- 172.21.15.103:0/3363119522 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5403f030 con 0x7fdb6c104c20 2024-08-19T01:39:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.940+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6c1a1b70 con 0x7fdb6c104c20 2024-08-19T01:39:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.940+0000 7fdb68ff9640 1 -- 172.21.15.103:0/3363119522 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb54003aa0 con 0x7fdb6c104c20 2024-08-19T01:39:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.940+0000 7fdb68ff9640 1 -- 172.21.15.103:0/3363119522 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54035070 con 0x7fdb6c104c20 2024-08-19T01:39:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.940+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6c1a20c0 con 0x7fdb6c104c20 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.942+0000 7fdb68ff9640 1 -- 172.21.15.103:0/3363119522 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdb54042030 con 0x7fdb6c104c20 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.943+0000 7fdb740d0740 1 Processor -- start 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.943+0000 7fdb68ff9640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb580a2c30 0x7fdb580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.943+0000 7fdb740d0740 1 -- start start 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.943+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb580a2c30 0x7fdb580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.943+0000 7fdb68ff9640 1 -- 172.21.15.103:0/3363119522 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdb540ef7b0 con 0x7fdb6c104c20 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5e680 0x564d2de5ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5efc0 0x564d2de63a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de66510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb740d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d2ddc19d0 con 0x564d2de64050 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb740d0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564d2dd69820 con 0x564d2de5efc0 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb740d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564d2ddbd990 con 0x564d2de5e680 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5e680 0x564d2de5ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5e680 0x564d2de5ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:42744/0 (socket says 172.21.15.103:42744) 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 -- 172.21.15.103:0/1656103482 learned_addr learned my addr 172.21.15.103:0/1656103482 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb63fff640 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5efc0 0x564d2de63a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de66510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb580a2c30 0x7fdb580a50f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c06a6b0 tx=0x7fdb64009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 -- 172.21.15.103:0/1656103482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5efc0 msgr2=0x564d2de63a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5efc0 0x564d2de63a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 -- 172.21.15.103:0/1656103482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 msgr2=0x564d2de66510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de66510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.944+0000 7fdb6affd640 1 -- 172.21.15.103:0/1656103482 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2dad5210 con 0x564d2de5e680 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.945+0000 7fdb6affd640 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5e680 0x564d2de5ea80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009d20 tx=0x7fdb5c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ac7ccea3b9f0179 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.945+0000 7fdb617fa640 1 -- 172.21.15.103:0/1656103482 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c01e070 con 0x564d2de5e680 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.945+0000 7fdb617fa640 1 -- 172.21.15.103:0/1656103482 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb5c019ae0 con 0x564d2de5e680 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.945+0000 7fdb617fa640 1 -- 172.21.15.103:0/1656103482 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c00ea80 con 0x564d2de5e680 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.945+0000 7fdb740d0740 1 -- 172.21.15.103:0/1656103482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5e680 msgr2=0x564d2de5ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.945+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5e680 0x564d2de5ea80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009d20 tx=0x7fdb5c011810 comp rx=0 tx=0).stop 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- 172.21.15.103:0/1656103482 shutdown_connections 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de66510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5efc0 0x564d2de63a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1656103482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5e680 0x564d2de5ea80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- 172.21.15.103:0/1656103482 >> 172.21.15.103:0/1656103482 conn(0x564d2de59cf0 msgr2=0x564d2de5a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- 172.21.15.103:0/1656103482 shutdown_connections 2024-08-19T01:39:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- 172.21.15.103:0/1656103482 wait complete. 2024-08-19T01:39:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- start start 2024-08-19T01:39:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 Processor -- start 2024-08-19T01:39:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de5efc0 0x564d2de68cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de64050 0x564d2de6fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d2de67020 con 0x564d2de5efc0 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564d2dd7d520 con 0x564d2de5e680 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564d2de671a0 con 0x564d2de64050 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb740d0740 10 client.0 _ll_register_callbacks cb 0x564d2dad4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.946+0000 7fdb6affd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb63fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de5efc0 0x564d2de68cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb6affd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de687b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:48904/0 (socket says 172.21.15.103:48904) 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb6affd640 1 -- 172.21.15.103:0/524938333 learned_addr learned my addr 172.21.15.103:0/524938333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de64050 0x564d2de6fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb63fff640 1 -- 172.21.15.103:0/524938333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de64050 msgr2=0x564d2de6fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb63fff640 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de64050 0x564d2de6fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb63fff640 1 -- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 msgr2=0x564d2de687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb63fff640 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb63fff640 1 -- 172.21.15.103:0/524938333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2dba1620 con 0x564d2de5efc0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb6affd640 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb63fff640 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de5efc0 0x564d2de68cf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdb540ed330 tx=0x7fdb540bb9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f89cef1a2e0327 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb527fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb527fc640 1 -- 172.21.15.103:0/524938333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54049040 con 0x564d2de5efc0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb5c019ae0 con 0x564d2de5efc0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb527fc640 1 -- 172.21.15.103:0/524938333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdb540c00d0 con 0x564d2de5efc0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb527fc640 1 -- 172.21.15.103:0/524938333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb540c03f0 con 0x564d2de5efc0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb740d0740 10 client.16425 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.947+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d2de72b20 con 0x564d2de5efc0 2024-08-19T01:39:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb527fc640 1 -- 172.21.15.103:0/524938333 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb5c019ae0 con 0x564d2de5efc0 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb527fc640 1 -- 172.21.15.103:0/524938333 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdb5403bca0 con 0x564d2de5efc0 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb740d0740 10 client.16425 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb740d0740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb740d0740 20 client.16425 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb740d0740 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2de742e0 0x564d2de76740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb740d0740 4 client.16425 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d2ddcf100 con 0x564d2de742e0 2024-08-19T01:39:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.948+0000 7fdb6affd640 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2de742e0 0x564d2de76740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.949+0000 7fdb6affd640 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2de742e0 0x564d2de76740 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a50a33306d275400 server_cookie=bd1ba9bed1de9d61 in_seq=0 out_seq=0 2024-08-19T01:39:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.949+0000 7fdb527fc640 10 client.16425 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.949+0000 7fdb527fc640 1 -- 172.21.15.103:0/524938333 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d2ddcf100 con 0x564d2de742e0 2024-08-19T01:39:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.949+0000 7fdb527fc640 10 client.16425 handle_command_reply: tid=0 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.949+0000 7fdb740d0740 1 client.16425 shutdown 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de5efc0 msgr2=0x564d2de68cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de5efc0 0x564d2de68cf0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdb540ed330 tx=0x7fdb540bb9a0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 shutdown_connections 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2de742e0 0x564d2de76740 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de64050 0x564d2de6fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de5efc0 0x564d2de68cf0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 --2- 172.21.15.103:0/524938333 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb527fc640 0 client.16425 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 >> 172.21.15.103:0/524938333 conn(0x564d2de59cf0 msgr2=0x564d2de5b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 shutdown_connections 2024-08-19T01:39:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 1 -- 172.21.15.103:0/524938333 wait complete. 2024-08-19T01:39:50.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.950+0000 7fdb740d0740 20 client.16425 trim_cache size 0 max 16384 2024-08-19T01:39:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 Processor -- start 2024-08-19T01:39:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 -- start start 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de5a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1aad20 con 0x564d2de64050 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1ac980 con 0x564d2de5e680 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.968+0000 7fdb740d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1ab3f0 con 0x564d2de5efc0 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de5a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de5a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45856/0 (socket says 172.21.15.103:45856) 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/1418443300 learned_addr learned my addr 172.21.15.103:0/1418443300 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb63fff640 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6affd640 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/1418443300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 msgr2=0x564d2de6fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/1418443300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 msgr2=0x564d2de703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/1418443300 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2de5f4a0 con 0x564d2de64050 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb63fff640 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de5a0d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdb64011890 tx=0x7fdb64013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6d50b193e79859 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb60ff9640 1 -- 172.21.15.103:0/1418443300 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6401fbb0 con 0x564d2de64050 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb60ff9640 1 -- 172.21.15.103:0/1418443300 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb64003e20 con 0x564d2de64050 2024-08-19T01:39:50.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.969+0000 7fdb60ff9640 1 -- 172.21.15.103:0/1418443300 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6401d440 con 0x564d2de64050 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 -- 172.21.15.103:0/1418443300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 msgr2=0x564d2de5a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de5a0d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdb64011890 tx=0x7fdb64013000 comp rx=0 tx=0).stop 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 -- 172.21.15.103:0/1418443300 shutdown_connections 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de5a0d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 --2- 172.21.15.103:0/1418443300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 -- 172.21.15.103:0/1418443300 >> 172.21.15.103:0/1418443300 conn(0x564d2de59cf0 msgr2=0x564d2de60130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 -- 172.21.15.103:0/1418443300 shutdown_connections 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.970+0000 7fdb740d0740 1 -- 172.21.15.103:0/1418443300 wait complete. 2024-08-19T01:39:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 -- start start 2024-08-19T01:39:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 Processor -- start 2024-08-19T01:39:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de6cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de6e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1ac1f0 con 0x564d2de64050 2024-08-19T01:39:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1abac0 con 0x564d2de5e680 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1a8d00 con 0x564d2de5efc0 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb740d0740 10 client.0 _ll_register_callbacks cb 0x7fdb540b8320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de6e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de6e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45866/0 (socket says 172.21.15.103:45866) 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/2578431081 learned_addr learned my addr 172.21.15.103:0/2578431081 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb63fff640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6affd640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de6cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/2578431081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 msgr2=0x564d2de6d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 msgr2=0x564d2de6cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de6cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 -- 172.21.15.103:0/2578431081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2de6e940 con 0x564d2de64050 2024-08-19T01:39:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb63fff640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb6b7fe640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de6e330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdb64003fd0 tx=0x7fdb640043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63d13ad0ebc92d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.971+0000 7fdb527fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb527fc640 1 -- 172.21.15.103:0/2578431081 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64037060 con 0x564d2de64050 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb527fc640 1 -- 172.21.15.103:0/2578431081 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fdb64004a40 con 0x564d2de64050 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb64003e20 con 0x564d2de64050 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb527fc640 1 -- 172.21.15.103:0/2578431081 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb64004c40 con 0x564d2de64050 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 10 client.16431 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d2de6fc50 con 0x564d2de64050 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb527fc640 1 -- 172.21.15.103:0/2578431081 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb64003e20 con 0x564d2de64050 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb527fc640 1 -- 172.21.15.103:0/2578431081 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdb64005500 con 0x564d2de64050 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 10 client.16431 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 20 client.16431 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2dedf3e0 0x564d2dee1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 4 client.16431 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:39:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.972+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d2ddc9450 con 0x564d2dedf3e0 2024-08-19T01:39:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.973+0000 7fdb6affd640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2dedf3e0 0x564d2dee1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.973+0000 7fdb6affd640 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2dedf3e0 0x564d2dee1800 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f0c7c203d5d176 server_cookie=bd79bfe51266b863 in_seq=0 out_seq=0 2024-08-19T01:39:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.973+0000 7fdb527fc640 10 client.16431 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb527fc640 1 -- 172.21.15.103:0/2578431081 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d2ddc9450 con 0x564d2dedf3e0 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb527fc640 10 client.16431 handle_command_reply: tid=0 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 client.16431 shutdown 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 msgr2=0x564d2de6e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de6e330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdb64003fd0 tx=0x7fdb640043e0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 shutdown_connections 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564d2dedf3e0 0x564d2dee1800 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d2de64050 0x564d2de6e330 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564d2de5efc0 0x564d2de6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 --2- 172.21.15.103:0/2578431081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564d2de5e680 0x564d2de6cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 >> 172.21.15.103:0/2578431081 conn(0x564d2de59cf0 msgr2=0x564d2de58680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.979+0000 7fdb527fc640 0 client.16431 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 shutdown_connections 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb740d0740 1 -- 172.21.15.103:0/2578431081 wait complete. 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb740d0740 20 client.16431 trim_cache size 0 max 16384 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:39:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:39:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb580a2c30 msgr2=0x7fdb580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb580a2c30 0x7fdb580a50f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c06a6b0 tx=0x7fdb64009500 comp rx=0 tx=0).stop 2024-08-19T01:39:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 msgr2=0x7fdb6c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c1002a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdb540314e0 tx=0x7fdb54004270 comp rx=0 tx=0).stop 2024-08-19T01:39:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 shutdown_connections 2024-08-19T01:39:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdb580a2c30 0x7fdb580a50f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb6c107550 0x7fdb6c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb6c104c20 0x7fdb6c1002a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 --2- 172.21.15.103:0/3363119522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb6c1022f0 0x7fdb6c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 >> 172.21.15.103:0/3363119522 conn(0x7fdb6c0fbf10 msgr2=0x7fdb6c102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 shutdown_connections 2024-08-19T01:39:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:39:50.980+0000 7fdb7130c640 1 -- 172.21.15.103:0/3363119522 wait complete. 2024-08-19T01:39:51.670 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:39:51.951 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:52.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:51 smithi146 ceph-mon[22098]: pgmap v461: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:39:52.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:51 smithi103 ceph-mon[18436]: pgmap v461: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:39:52.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:51 smithi190 ceph-mon[22032]: pgmap v461: 97 pgs: 97 active+clean; 5.2 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 -- 172.21.15.103:0/627045741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 msgr2=0x7fc01c10b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 --2- 172.21.15.103:0/627045741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c10b710 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc004009900 tx=0x7fc00402a480 comp rx=0 tx=0).stop 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 -- 172.21.15.103:0/627045741 shutdown_connections 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 --2- 172.21.15.103:0/627045741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c10bd30 0x7fc01c10e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 --2- 172.21.15.103:0/627045741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c10b710 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 --2- 172.21.15.103:0/627045741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 0x7fc01c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 -- 172.21.15.103:0/627045741 >> 172.21.15.103:0/627045741 conn(0x7fc01c0fc360 msgr2=0x7fc01c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 -- 172.21.15.103:0/627045741 shutdown_connections 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.503+0000 7fc021803640 1 -- 172.21.15.103:0/627045741 wait complete. 2024-08-19T01:39:52.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 Processor -- start 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 -- start start 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 0x7fc01c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c10bd30 0x7fc01c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1119d0 con 0x7fc01c103220 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c111850 con 0x7fc01c10bd30 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc021803640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c111b50 con 0x7fc01c102900 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc01affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 0x7fc01c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc01affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 0x7fc01c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:40628/0 (socket says 172.21.15.103:40628) 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60656/0 (socket says 172.21.15.103:60656) 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc01affd640 1 -- 172.21.15.103:0/1513751020 learned_addr learned my addr 172.21.15.103:0/1513751020 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:52.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c10bd30 0x7fc01c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc01affd640 1 -- 172.21.15.103:0/1513751020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c10bd30 msgr2=0x7fc01c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.504+0000 7fc01affd640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c10bd30 0x7fc01c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc01affd640 1 -- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 msgr2=0x7fc01c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc01affd640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc01affd640 1 -- 172.21.15.103:0/1513751020 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c19cf00 con 0x7fc01c102900 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc013fff640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc01affd640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 0x7fc01c197f50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00d770 tx=0x7fc00c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc020801640 1 -- 172.21.15.103:0/1513751020 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c017070 con 0x7fc01c102900 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc020801640 1 -- 172.21.15.103:0/1513751020 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc00c0054d0 con 0x7fc01c102900 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc020801640 1 -- 172.21.15.103:0/1513751020 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c004cf0 con 0x7fc01c102900 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c19d1f0 con 0x7fc01c102900 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.505+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1a4ad0 con 0x7fc01c102900 2024-08-19T01:39:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.508+0000 7fc020801640 1 -- 172.21.15.103:0/1513751020 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc00c004e90 con 0x7fc01c102900 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.508+0000 7fc023c01740 1 Processor -- start 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.508+0000 7fc023c01740 1 -- start start 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc023c01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f2995175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc023c01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299521e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc023c01740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f299524790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc023c01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f29947ca40 con 0x55f299517ae0 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc023c01740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f2994149d0 con 0x55f2995223a0 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc023c01740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f299478a00 con 0x55f2995171c0 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc020801640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0080a2c00 0x7fc0080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299521e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f2995175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f2995175a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:40640/0 (socket says 172.21.15.103:40640) 2024-08-19T01:39:52.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01b7fe640 1 -- 172.21.15.103:0/696910260 learned_addr learned my addr 172.21.15.103:0/696910260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0080a2c00 0x7fc0080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc013fff640 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f299524790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc020801640 1 -- 172.21.15.103:0/1513751020 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc00c0ce9b0 con 0x7fc01c102900 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01affd640 1 -- 172.21.15.103:0/696910260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 msgr2=0x55f2995175a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01affd640 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f2995175a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01affd640 1 -- 172.21.15.103:0/696910260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 msgr2=0x55f299524790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01affd640 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f299524790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.509+0000 7fc01affd640 1 -- 172.21.15.103:0/696910260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f29941b650 con 0x55f299517ae0 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.510+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f2995175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.510+0000 7fc01affd640 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299521e60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc00c097c60 tx=0x7fc00c06d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4351db1bd8c9b419 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.510+0000 7fc011ffb640 1 -- 172.21.15.103:0/696910260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c020070 con 0x55f299517ae0 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.510+0000 7fc011ffb640 1 -- 172.21.15.103:0/696910260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc00c009c30 con 0x55f299517ae0 2024-08-19T01:39:52.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.510+0000 7fc011ffb640 1 -- 172.21.15.103:0/696910260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c097900 con 0x55f299517ae0 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 -- 172.21.15.103:0/696910260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 msgr2=0x55f299521e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299521e60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc00c097c60 tx=0x7fc00c06d2a0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0080a2c00 0x7fc0080a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc014007310 tx=0x7fc014007520 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 -- 172.21.15.103:0/696910260 shutdown_connections 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f299524790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299521e60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 --2- 172.21.15.103:0/696910260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f2995175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 -- 172.21.15.103:0/696910260 >> 172.21.15.103:0/696910260 conn(0x55f299513750 msgr2=0x55f299513b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 -- 172.21.15.103:0/696910260 shutdown_connections 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 -- 172.21.15.103:0/696910260 wait complete. 2024-08-19T01:39:52.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.511+0000 7fc023c01740 1 -- start start 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc023c01740 1 Processor -- start 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc023c01740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29951f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc023c01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f29951f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc023c01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995223a0 0x55f29951fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f29951f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f29951f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60686/0 (socket says 172.21.15.103:60686) 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc013fff640 1 -- 172.21.15.103:0/3753536167 learned_addr learned my addr 172.21.15.103:0/3753536167 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc01affd640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29951f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f299529320 con 0x55f299517ae0 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.512+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f299437b20 con 0x55f2995171c0 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f2995294a0 con 0x55f2995223a0 2024-08-19T01:39:52.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995223a0 0x55f29951fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc013fff640 1 -- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995223a0 msgr2=0x55f29951fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc013fff640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995223a0 0x55f29951fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc013fff640 1 -- 172.21.15.103:0/3753536167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 msgr2=0x55f29951f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc013fff640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29951f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc013fff640 1 -- 172.21.15.103:0/3753536167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2991ee3c0 con 0x55f299517ae0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc01affd640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29951f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc023c01740 10 client.0 _ll_register_callbacks cb 0x55f299435bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f299489fb0 con 0x55f299517ae0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fc013fff640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f29951f7e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc004033000 tx=0x7fc004031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7b2a5b6c4a7436 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.513+0000 7fbffaffd640 10 client.16443 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.514+0000 7fbffaffd640 1 -- 172.21.15.103:0/3753536167 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004035040 con 0x55f299517ae0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.514+0000 7fbffaffd640 1 -- 172.21.15.103:0/3753536167 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc004004380 con 0x55f299517ae0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.514+0000 7fbffaffd640 1 -- 172.21.15.103:0/3753536167 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc004005030 con 0x55f299517ae0 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.514+0000 7fc023c01740 10 client.16443 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:52.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.514+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55f299534d60 con 0x55f299517ae0 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.514+0000 7fbffaffd640 1 -- 172.21.15.103:0/3753536167 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc0040051d0 con 0x55f299517ae0 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc023c01740 10 client.16443 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc023c01740 10 client.16443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc023c01740 20 client.16443 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc023c01740 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f299538640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc023c01740 4 client.16443 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2994844c0 con 0x55f2995361b0 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc01affd640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f299538640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fc01affd640 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f299538640 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c40265aa62babaaf server_cookie=65c15e835d440b7c in_seq=0 out_seq=0 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.515+0000 7fbffaffd640 10 client.16443 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.516+0000 7fbffaffd640 1 -- 172.21.15.103:0/3753536167 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2994844c0 con 0x55f2995361b0 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.516+0000 7fbffaffd640 10 client.16443 handle_command_reply: tid=0 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.516+0000 7fc023c01740 1 client.16443 shutdown 2024-08-19T01:39:52.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.516+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 msgr2=0x55f29951f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.516+0000 7fc023c01740 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f29951f7e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc004033000 tx=0x7fc004031930 comp rx=0 tx=0).stop 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.516+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 shutdown_connections 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.516+0000 7fc023c01740 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f299538640 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fc023c01740 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995223a0 0x55f29951fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fbffaffd640 0 client.16443 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fc023c01740 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f29951f7e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fc023c01740 1 --2- 172.21.15.103:0/3753536167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29951f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 >> 172.21.15.103:0/3753536167 conn(0x55f299513750 msgr2=0x55f299519480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 shutdown_connections 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fc023c01740 1 -- 172.21.15.103:0/3753536167 wait complete. 2024-08-19T01:39:52.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.517+0000 7fc023c01740 20 client.16443 trim_cache size 0 max 16384 2024-08-19T01:39:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 Processor -- start 2024-08-19T01:39:52.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 -- start start 2024-08-19T01:39:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f29951fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299512800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f29953df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1aac90 con 0x55f299517ae0 2024-08-19T01:39:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ac8f0 con 0x55f2995223a0 2024-08-19T01:39:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.532+0000 7fc023c01740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ab360 con 0x55f2995171c0 2024-08-19T01:39:52.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299512800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299512800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60688/0 (socket says 172.21.15.103:60688) 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 -- 172.21.15.103:0/3226948975 learned_addr learned my addr 172.21.15.103:0/3226948975 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc01affd640 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f29951fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f29953df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 -- 172.21.15.103:0/3226948975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 msgr2=0x55f29951fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f29951fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 -- 172.21.15.103:0/3226948975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 msgr2=0x55f29953df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f29953df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 -- 172.21.15.103:0/3226948975 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f299517910 con 0x55f299517ae0 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f29953df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc013fff640 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299512800 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc00402a9a0 tx=0x7fc00403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63903aa315b40a7d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc0117fa640 1 -- 172.21.15.103:0/3226948975 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004035070 con 0x55f299517ae0 2024-08-19T01:39:52.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc0117fa640 1 -- 172.21.15.103:0/3226948975 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f299489fb0 con 0x55f299517ae0 2024-08-19T01:39:52.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.533+0000 7fc0117fa640 1 -- 172.21.15.103:0/3226948975 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0040043e0 con 0x55f299517ae0 2024-08-19T01:39:52.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 -- 172.21.15.103:0/3226948975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 msgr2=0x55f299512800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299512800 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc00402a9a0 tx=0x7fc00403f000 comp rx=0 tx=0).stop 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 -- 172.21.15.103:0/3226948975 shutdown_connections 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995223a0 0x55f29953df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f299517ae0 0x55f299512800 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 --2- 172.21.15.103:0/3226948975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f2995171c0 0x55f29951fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 -- 172.21.15.103:0/3226948975 >> 172.21.15.103:0/3226948975 conn(0x55f299513750 msgr2=0x55f2995120f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 -- 172.21.15.103:0/3226948975 shutdown_connections 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.534+0000 7fc023c01740 1 -- 172.21.15.103:0/3226948975 wait complete. 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 -- start start 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 Processor -- start 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29952eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f299517ae0 0x55f299529e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2995223a0 0x55f29952a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ac160 con 0x55f2995223a0 2024-08-19T01:39:52.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1aba30 con 0x55f2995171c0 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1a8c90 con 0x55f299517ae0 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc023c01740 10 client.0 _ll_register_callbacks cb 0x7fc004009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.535+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2995223a0 0x55f29952a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2995223a0 0x55f29952a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60702/0 (socket says 172.21.15.103:60702) 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 -- 172.21.15.103:0/1492482366 learned_addr learned my addr 172.21.15.103:0/1492482366 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01affd640 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29952eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc013fff640 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f299517ae0 0x55f299529e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 -- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f299517ae0 msgr2=0x55f299529e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f299517ae0 0x55f299529e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 -- 172.21.15.103:0/1492482366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 msgr2=0x55f29952eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29952eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 -- 172.21.15.103:0/1492482366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f29952a8d0 con 0x55f2995223a0 2024-08-19T01:39:52.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc01b7fe640 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2995223a0 0x55f29952a390 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc014003f50 tx=0x7fc014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d7b6331e9719bf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fbffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fbffaffd640 1 -- 172.21.15.103:0/1492482366 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014027070 con 0x55f2995223a0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fbffaffd640 1 -- 172.21.15.103:0/1492482366 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc014004d20 con 0x55f2995223a0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f299489fb0 con 0x55f2995223a0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fbffaffd640 1 -- 172.21.15.103:0/1492482366 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc014032050 con 0x55f2995223a0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.536+0000 7fc023c01740 10 client.16449 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2995a0be0 con 0x55f2995223a0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fbffaffd640 1 -- 172.21.15.103:0/1492482366 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f299489fb0 con 0x55f2995223a0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fbffaffd640 1 -- 172.21.15.103:0/1492482366 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc0140053f0 con 0x55f2995223a0 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc023c01740 10 client.16449 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc023c01740 10 client.16449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc023c01740 20 client.16449 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc023c01740 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f2995a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:52.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc023c01740 4 client.16449 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:52.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f299401170 con 0x55f2995361b0 2024-08-19T01:39:52.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.537+0000 7fc01affd640 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f2995a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:52.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fbffaffd640 10 client.16449 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:52.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc01affd640 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f2995a3310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=518ebf099f066f80 server_cookie=1abc5bea9d7c3002 in_seq=0 out_seq=0 2024-08-19T01:39:52.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fbffaffd640 1 -- 172.21.15.103:0/1492482366 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f299401170 con 0x55f2995361b0 2024-08-19T01:39:52.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fbffaffd640 10 client.16449 handle_command_reply: tid=0 2024-08-19T01:39:52.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 client.16449 shutdown 2024-08-19T01:39:52.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2995223a0 msgr2=0x55f29952a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2995223a0 0x55f29952a390 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc014003f50 tx=0x7fc014004a40 comp rx=0 tx=0).stop 2024-08-19T01:39:52.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 shutdown_connections 2024-08-19T01:39:52.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55f2995361b0 0x55f2995a3310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2995223a0 0x55f29952a390 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f299517ae0 0x55f299529e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 --2- 172.21.15.103:0/1492482366 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f2995171c0 0x55f29952eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 >> 172.21.15.103:0/1492482366 conn(0x55f299513750 msgr2=0x55f299519430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fbffaffd640 0 client.16449 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:39:52.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 shutdown_connections 2024-08-19T01:39:52.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.538+0000 7fc023c01740 1 -- 172.21.15.103:0/1492482366 wait complete. 2024-08-19T01:39:52.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc023c01740 20 client.16449 trim_cache size 0 max 16384 2024-08-19T01:39:52.545 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:52.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0080a2c00 msgr2=0x7fc0080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0080a2c00 0x7fc0080a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc014007310 tx=0x7fc014007520 comp rx=0 tx=0).stop 2024-08-19T01:39:52.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 msgr2=0x7fc01c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:52.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 0x7fc01c197f50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00d770 tx=0x7fc00c00dc30 comp rx=0 tx=0).stop 2024-08-19T01:39:52.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 shutdown_connections 2024-08-19T01:39:52.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc0080a2c00 0x7fc0080a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc01c10bd30 0x7fc01c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc01c103220 0x7fc01c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 --2- 172.21.15.103:0/1513751020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc01c102900 0x7fc01c197f50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:52.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.539+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 >> 172.21.15.103:0/1513751020 conn(0x7fc01c0fc360 msgr2=0x7fc01c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:52.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.540+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 shutdown_connections 2024-08-19T01:39:52.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:52.540+0000 7fc021803640 1 -- 172.21.15.103:0/1513751020 wait complete. 2024-08-19T01:39:53.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:39:53.668 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:54.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:53 smithi103 ceph-mon[18436]: pgmap v462: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-08-19T01:39:54.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:53 smithi146 ceph-mon[22098]: pgmap v462: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-08-19T01:39:54.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1571484502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 msgr2=0x7f4dfc10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1571484502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 0x7f4dfc10e970 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009cd0 tx=0x7f4df402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1571484502 shutdown_connections 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1571484502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 0x7f4dfc10e970 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1571484502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc103410 0x7f4dfc1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1571484502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dfc102a60 0x7f4dfc102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1571484502 >> 172.21.15.103:0/1571484502 conn(0x7f4dfc0fc500 msgr2=0x7f4dfc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1571484502 shutdown_connections 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1571484502 wait complete. 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.198+0000 7f4e02f5c640 1 Processor -- start 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e02f5c640 1 -- start start 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e02f5c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc102a60 0x7f4dfc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e02f5c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dfc103410 0x7f4dfc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e02f5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 0x7f4dfc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e02f5c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc111d30 con 0x7f4dfc107fd0 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e02f5c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc111bb0 con 0x7f4dfc102a60 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e02f5c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc111eb0 con 0x7f4dfc103410 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc102a60 0x7f4dfc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e014f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 0x7f4dfc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc102a60 0x7f4dfc198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:36862/0 (socket says 172.21.15.103:36862) 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 -- 172.21.15.103:0/1889220456 learned_addr learned my addr 172.21.15.103:0/1889220456 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dfc103410 0x7f4dfc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 -- 172.21.15.103:0/1889220456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dfc103410 msgr2=0x7f4dfc198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dfc103410 0x7f4dfc198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 -- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 msgr2=0x7f4dfc19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 0x7f4dfc19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e00cf8640 1 -- 172.21.15.103:0/1889220456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc19d1d0 con 0x7f4dfc102a60 2024-08-19T01:39:54.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.199+0000 7f4e014f9640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 0x7f4dfc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.200+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc102a60 0x7f4dfc198220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4df000b4a0 tx=0x7f4df000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.200+0000 7f4df9ffb640 1 -- 172.21.15.103:0/1889220456 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0014070 con 0x7f4dfc102a60 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.200+0000 7f4df9ffb640 1 -- 172.21.15.103:0/1889220456 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df0010ac0 con 0x7f4dfc102a60 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.200+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc19d4c0 con 0x7f4dfc102a60 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.200+0000 7f4df9ffb640 1 -- 172.21.15.103:0/1889220456 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0017c30 con 0x7f4dfc102a60 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.200+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1a4da0 con 0x7f4dfc102a60 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4df9ffb640 1 -- 172.21.15.103:0/1889220456 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4df0017070 con 0x7f4dfc102a60 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 Processor -- start 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 -- start start 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4df9ffb640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4dec0a2b80 0x7f4dec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e0ff0 0x5623c00e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00ec1d0 0x5623c00ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623c0046a40 con 0x5623c00e1910 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5623bffde9d0 con 0x5623c00ec1d0 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4df9ffb640 1 -- 172.21.15.103:0/1889220456 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4df00ce6f0 con 0x7f4dfc102a60 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e0535a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5623c0042a00 con 0x5623c00e0ff0 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.203+0000 7f4e014f9640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4dec0a2b80 0x7f4dec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60730/0 (socket says 172.21.15.103:60730) 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 -- 172.21.15.103:0/723075404 learned_addr learned my addr 172.21.15.103:0/723075404 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4e014f9640 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00ec1d0 0x5623c00ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e0ff0 0x5623c00e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 -- 172.21.15.103:0/723075404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e0ff0 msgr2=0x5623c00e13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e0ff0 0x5623c00e13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 -- 172.21.15.103:0/723075404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00ec1d0 msgr2=0x5623c00ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00ec1d0 0x5623c00ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 -- 172.21.15.103:0/723075404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623bffe5650 con 0x5623c00e1910 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e0ff0 0x5623c00e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4e014f9640 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00ec1d0 0x5623c00ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4e014f9640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4dec0a2b80 0x7f4dec0a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4df4004af0 tx=0x7f4df4031c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ebc90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009bb0 tx=0x7f4de80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8f07303c095946 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4de6ffd640 1 -- 172.21.15.103:0/723075404 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8012450 con 0x5623c00e1910 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4de6ffd640 1 -- 172.21.15.103:0/723075404 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4de8012aa0 con 0x5623c00e1910 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.204+0000 7f4de6ffd640 1 -- 172.21.15.103:0/723075404 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de800f990 con 0x5623c00e1910 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 -- 172.21.15.103:0/723075404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 msgr2=0x5623c00ebc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ebc90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009bb0 tx=0x7f4de80117b0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 -- 172.21.15.103:0/723075404 shutdown_connections 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00ec1d0 0x5623c00ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ebc90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 --2- 172.21.15.103:0/723075404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e0ff0 0x5623c00e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 -- 172.21.15.103:0/723075404 >> 172.21.15.103:0/723075404 conn(0x5623c00dd580 msgr2=0x5623c00dd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 -- 172.21.15.103:0/723075404 shutdown_connections 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.205+0000 7f4e0535a740 1 -- 172.21.15.103:0/723075404 wait complete. 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 -- start start 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 Processor -- start 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e0ff0 0x5623c00ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e1910 0x5623c00e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 0x5623c00e9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623c00eeda0 con 0x5623c00e0ff0 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5623c0001b20 con 0x5623c00e1910 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5623c00eef20 con 0x5623c00ec1d0 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e0535a740 10 client.0 _ll_register_callbacks cb 0x5623bffffbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e00cf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e0ff0 0x5623c00ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e00cf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e0ff0 0x5623c00ebc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60738/0 (socket says 172.21.15.103:60738) 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4e00cf8640 1 -- 172.21.15.103:0/3935687296 learned_addr learned my addr 172.21.15.103:0/3935687296 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.206+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e1910 0x5623c00e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e00cf8640 1 -- 172.21.15.103:0/3935687296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 msgr2=0x5623c00e9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 0x5623c00e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e00cf8640 1 -- 172.21.15.103:0/3935687296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e1910 msgr2=0x5623c00e8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e1910 0x5623c00e8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e00cf8640 1 -- 172.21.15.103:0/3935687296 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623bfdb83c0 con 0x5623c00e0ff0 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e1910 0x5623c00e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e0ff0 0x5623c00ebc70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4df00973e0 tx=0x7f4df009b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181eaf8facb5ca05 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4ddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4ddbfff640 1 -- 172.21.15.103:0/3935687296 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df001e050 con 0x5623c00e0ff0 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4ddbfff640 1 -- 172.21.15.103:0/3935687296 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4df00975f0 con 0x5623c00e0ff0 2024-08-19T01:39:54.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de8012aa0 con 0x5623c00e0ff0 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4ddbfff640 1 -- 172.21.15.103:0/3935687296 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4df009f330 con 0x5623c00e0ff0 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e0535a740 10 client.16461 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5623c01030c0 con 0x5623c00e0ff0 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4ddbfff640 1 -- 172.21.15.103:0/3935687296 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df00b5060 con 0x5623c00e0ff0 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4ddbfff640 1 -- 172.21.15.103:0/3935687296 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4df00b9bc0 con 0x5623c00e0ff0 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.207+0000 7f4e0535a740 10 client.16461 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4e0535a740 10 client.16461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4e0535a740 20 client.16461 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4e0535a740 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c01069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4e0535a740 4 client.16461 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4de8012aa0 con 0x5623c0104580 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c01069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c01069d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce1071ddfb08e808 server_cookie=6757c360c85a8da4 in_seq=0 out_seq=0 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.208+0000 7f4ddbfff640 10 client.16461 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.209+0000 7f4ddbfff640 1 -- 172.21.15.103:0/3935687296 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4de8012aa0 con 0x5623c0104580 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.209+0000 7f4ddbfff640 10 client.16461 handle_command_reply: tid=0 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.209+0000 7f4e0535a740 1 client.16461 shutdown 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.209+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e0ff0 msgr2=0x5623c00ebc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.209+0000 7f4e0535a740 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e0ff0 0x5623c00ebc70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4df00973e0 tx=0x7f4df009b700 comp rx=0 tx=0).stop 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.209+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 shutdown_connections 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.209+0000 7f4e0535a740 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c01069d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4e0535a740 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 0x5623c00e9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4ddbfff640 0 client.16461 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4e0535a740 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e1910 0x5623c00e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4e0535a740 1 --2- 172.21.15.103:0/3935687296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e0ff0 0x5623c00ebc70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 >> 172.21.15.103:0/3935687296 conn(0x5623c00dd580 msgr2=0x5623c00e3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:54.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 shutdown_connections 2024-08-19T01:39:54.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4e0535a740 1 -- 172.21.15.103:0/3935687296 wait complete. 2024-08-19T01:39:54.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.210+0000 7f4e0535a740 20 client.16461 trim_cache size 0 max 16384 2024-08-19T01:39:54.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 Processor -- start 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 -- start start 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e1910 0x5623c010c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00ec1d0 0x5623c010c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1aaf80 con 0x5623c00ec1d0 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1acbe0 con 0x5623c00e0ff0 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.224+0000 7f4e0535a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1ab650 con 0x5623c00e1910 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00ec1d0 0x5623c010c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00ec1d0 0x5623c010c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60752/0 (socket says 172.21.15.103:60752) 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 -- 172.21.15.103:0/2188069542 learned_addr learned my addr 172.21.15.103:0/2188069542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e1910 0x5623c010c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 -- 172.21.15.103:0/2188069542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e1910 msgr2=0x5623c010c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e1910 0x5623c010c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 -- 172.21.15.103:0/2188069542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 msgr2=0x5623c00e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00e88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 -- 172.21.15.103:0/2188069542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623c00de200 con 0x5623c00ec1d0 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4e014f9640 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00ec1d0 0x5623c010c7e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009cd0 tx=0x7f4df4045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48a9c4f1e12f2c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4de67fc640 1 -- 172.21.15.103:0/2188069542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4044040 con 0x5623c00ec1d0 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4de67fc640 1 -- 172.21.15.103:0/2188069542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df4004490 con 0x5623c00ec1d0 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.225+0000 7f4de67fc640 1 -- 172.21.15.103:0/2188069542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4030a10 con 0x5623c00ec1d0 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 -- 172.21.15.103:0/2188069542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00ec1d0 msgr2=0x5623c010c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00ec1d0 0x5623c010c7e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009cd0 tx=0x7f4df4045000 comp rx=0 tx=0).stop 2024-08-19T01:39:54.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 -- 172.21.15.103:0/2188069542 shutdown_connections 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00ec1d0 0x5623c010c7e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00e1910 0x5623c010c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 --2- 172.21.15.103:0/2188069542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 -- 172.21.15.103:0/2188069542 >> 172.21.15.103:0/2188069542 conn(0x5623c00dd580 msgr2=0x5623c00dbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 -- 172.21.15.103:0/2188069542 shutdown_connections 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.226+0000 7f4e0535a740 1 -- 172.21.15.103:0/2188069542 wait complete. 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 -- start start 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 Processor -- start 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00ed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 0x5623c00edee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1ac450 con 0x5623c00e1910 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1abd20 con 0x5623c00e0ff0 2024-08-19T01:39:54.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1a8f60 con 0x5623c00ec1d0 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e0535a740 10 client.0 _ll_register_callbacks cb 0x7f4df0017360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ed9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60758/0 (socket says 172.21.15.103:60758) 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4dfbfff640 1 -- 172.21.15.103:0/1480849486 learned_addr learned my addr 172.21.15.103:0/1480849486 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.227+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4e014f9640 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 0x5623c00edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4dfbfff640 1 -- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 msgr2=0x5623c00edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 0x5623c00edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4dfbfff640 1 -- 172.21.15.103:0/1480849486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 msgr2=0x5623c00ed460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00ed460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4dfbfff640 1 -- 172.21.15.103:0/1480849486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623c00fb180 con 0x5623c00e1910 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4dfbfff640 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ed9a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009ad0 tx=0x7f4de80097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7644c03dde410d7b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4ddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4ddbfff640 1 -- 172.21.15.103:0/1480849486 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8012450 con 0x5623c00e1910 2024-08-19T01:39:54.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df4004490 con 0x5623c00e1910 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.228+0000 7f4ddbfff640 1 -- 172.21.15.103:0/1480849486 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4de80125f0 con 0x5623c00e1910 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4ddbfff640 1 -- 172.21.15.103:0/1480849486 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4de80127f0 con 0x5623c00e1910 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 10 client.16467 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5623c016f760 con 0x5623c00e1910 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4ddbfff640 1 -- 172.21.15.103:0/1480849486 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df4004490 con 0x5623c00e1910 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4ddbfff640 1 -- 172.21.15.103:0/1480849486 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4de8024400 con 0x5623c00e1910 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 10 client.16467 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 10 client.16467 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 20 client.16467 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c0171e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 4 client.16467 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623c0053fb0 con 0x5623c0104580 2024-08-19T01:39:54.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.229+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c0171e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e00cf8640 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c0171e60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea408c953e9ca47d server_cookie=603ec0d16ed05a61 in_seq=0 out_seq=0 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4ddbfff640 10 client.16467 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4ddbfff640 1 -- 172.21.15.103:0/1480849486 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623c0053fb0 con 0x5623c0104580 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4ddbfff640 10 client.16467 handle_command_reply: tid=0 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 client.16467 shutdown 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 msgr2=0x5623c00ed9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ed9a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009ad0 tx=0x7f4de80097e0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 shutdown_connections 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x5623c0104580 0x5623c0171e60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623c00ec1d0 0x5623c00edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623c00e1910 0x5623c00ed9a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4ddbfff640 0 client.16467 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 --2- 172.21.15.103:0/1480849486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623c00e0ff0 0x5623c00ed460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 >> 172.21.15.103:0/1480849486 conn(0x5623c00dd580 msgr2=0x5623c00ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:54.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 shutdown_connections 2024-08-19T01:39:54.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 1 -- 172.21.15.103:0/1480849486 wait complete. 2024-08-19T01:39:54.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.230+0000 7f4e0535a740 20 client.16467 trim_cache size 0 max 16384 2024-08-19T01:39:54.244 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4dec0a2b80 msgr2=0x7f4dec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4dec0a2b80 0x7f4dec0a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4df4004af0 tx=0x7f4df4031c00 comp rx=0 tx=0).stop 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc102a60 msgr2=0x7f4dfc198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc102a60 0x7f4dfc198220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4df000b4a0 tx=0x7f4df000b960 comp rx=0 tx=0).stop 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 shutdown_connections 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4dec0a2b80 0x7f4dec0a5040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc107fd0 0x7f4dfc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4dfc103410 0x7f4dfc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 --2- 172.21.15.103:0/1889220456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4dfc102a60 0x7f4dfc198220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:54.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 >> 172.21.15.103:0/1889220456 conn(0x7f4dfc0fc500 msgr2=0x7f4dfc105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:54.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 shutdown_connections 2024-08-19T01:39:54.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:54.231+0000 7f4e02f5c640 1 -- 172.21.15.103:0/1889220456 wait complete. 2024-08-19T01:39:54.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:53 smithi190 ceph-mon[22032]: pgmap v462: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-08-19T01:39:55.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:39:55.315 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:55.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 -- 172.21.15.103:0/1828503772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 msgr2=0x7ff31810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1828503772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff31810e970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff310009cd0 tx=0x7ff31002a8e0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 -- 172.21.15.103:0/1828503772 shutdown_connections 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1828503772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff31810e970 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1828503772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff318103410 0x7ff3181078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1828503772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff318102a60 0x7ff318102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 -- 172.21.15.103:0/1828503772 >> 172.21.15.103:0/1828503772 conn(0x7ff3180fc500 msgr2=0x7ff3180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 -- 172.21.15.103:0/1828503772 shutdown_connections 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 -- 172.21.15.103:0/1828503772 wait complete. 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 Processor -- start 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.862+0000 7ff3204ab640 1 -- start start 2024-08-19T01:39:55.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff3204ab640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff318102a60 0x7ff3181006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff3204ab640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff318103410 0x7ff318100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff3204ab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff318101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff3204ab640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff318111c80 con 0x7ff318107fd0 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff3204ab640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff318111b00 con 0x7ff318103410 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff3204ab640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff318111e00 con 0x7ff318102a60 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff318101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff318101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60774/0 (socket says 172.21.15.103:60774) 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 -- 172.21.15.103:0/1047233986 learned_addr learned my addr 172.21.15.103:0/1047233986 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31e247640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff318102a60 0x7ff3181006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31da46640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff318103410 0x7ff318100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 -- 172.21.15.103:0/1047233986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff318102a60 msgr2=0x7ff3181006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff318102a60 0x7ff3181006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 -- 172.21.15.103:0/1047233986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff318103410 msgr2=0x7ff318100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff318103410 0x7ff318100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 -- 172.21.15.103:0/1047233986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3181a6540 con 0x7ff318107fd0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31e247640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff318102a60 0x7ff3181006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.863+0000 7ff31ea48640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff318101140 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff310005cc0 tx=0x7ff310003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.864+0000 7ff30f7fe640 1 -- 172.21.15.103:0/1047233986 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310004240 con 0x7ff318107fd0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.864+0000 7ff30f7fe640 1 -- 172.21.15.103:0/1047233986 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3100305a0 con 0x7ff318107fd0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.864+0000 7ff30f7fe640 1 -- 172.21.15.103:0/1047233986 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3100466c0 con 0x7ff318107fd0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.864+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3181a67d0 con 0x7ff318107fd0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.864+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181a6cc0 con 0x7ff318107fd0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.867+0000 7ff30f7fe640 1 -- 172.21.15.103:0/1047233986 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff310030ac0 con 0x7ff318107fd0 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.867+0000 7ff3228a9740 1 Processor -- start 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.867+0000 7ff3228a9740 1 -- start start 2024-08-19T01:39:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff30f7fe640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3040a2c00 0x7ff3040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ea23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eacc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff3228a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5605b1e07a40 con 0x5605b1ea2910 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff3228a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5605b1d9f9d0 con 0x5605b1ead1d0 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff3228a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5605b1e03a00 con 0x5605b1ea1ff0 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff30f7fe640 1 -- 172.21.15.103:0/1047233986 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff31002e030 con 0x7ff318107fd0 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31da46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eacc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31da46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eacc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60778/0 (socket says 172.21.15.103:60778) 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31e247640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ea23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31da46640 1 -- 172.21.15.103:0/317202403 learned_addr learned my addr 172.21.15.103:0/317202403 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31da46640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3040a2c00 0x7ff3040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31ea48640 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31e247640 1 -- 172.21.15.103:0/317202403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 msgr2=0x5605b1eaf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31e247640 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31e247640 1 -- 172.21.15.103:0/317202403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 msgr2=0x5605b1eacc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31e247640 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eacc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31e247640 1 -- 172.21.15.103:0/317202403 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b1da6650 con 0x5605b1ea1ff0 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31da46640 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eacc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.868+0000 7ff31ea48640 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.869+0000 7ff31e247640 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ea23d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff314008b80 tx=0x7ff31400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83979cfa9f29e26f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.869+0000 7ff31da46640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3040a2c00 0x7ff3040a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b310 tx=0x7ff308016ea0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.869+0000 7ff30cff9640 1 -- 172.21.15.103:0/317202403 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314014070 con 0x5605b1ea1ff0 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.869+0000 7ff30cff9640 1 -- 172.21.15.103:0/317202403 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3140054d0 con 0x5605b1ea1ff0 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.869+0000 7ff30cff9640 1 -- 172.21.15.103:0/317202403 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314004ab0 con 0x5605b1ea1ff0 2024-08-19T01:39:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 -- 172.21.15.103:0/317202403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 msgr2=0x5605b1ea23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ea23d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff314008b80 tx=0x7ff31400e720 comp rx=0 tx=0).stop 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 -- 172.21.15.103:0/317202403 shutdown_connections 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eacc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 --2- 172.21.15.103:0/317202403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ea23d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 -- 172.21.15.103:0/317202403 >> 172.21.15.103:0/317202403 conn(0x5605b1e9e580 msgr2=0x5605b1e9e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 -- 172.21.15.103:0/317202403 shutdown_connections 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 -- 172.21.15.103:0/317202403 wait complete. 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 -- start start 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.870+0000 7ff3228a9740 1 Processor -- start 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1eaa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eaa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff3228a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5605b1eafda0 con 0x5605b1ea2910 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff3228a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5605b1dc2b20 con 0x5605b1ead1d0 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff3228a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5605b1eaff20 con 0x5605b1ea1ff0 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff3228a9740 10 client.0 _ll_register_callbacks cb 0x5605b1dc0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eaa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eaa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60788/0 (socket says 172.21.15.103:60788) 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 -- 172.21.15.103:0/2355756466 learned_addr learned my addr 172.21.15.103:0/2355756466 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31e247640 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1eaa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31ea48640 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 -- 172.21.15.103:0/2355756466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 msgr2=0x5605b1eaa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1eaa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 -- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 msgr2=0x5605b1eaaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 -- 172.21.15.103:0/2355756466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b1b793c0 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff31da46640 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eaa590 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff308005e10 tx=0x7ff308016c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f88d28721859c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.871+0000 7ff301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff301ffb640 1 -- 172.21.15.103:0/2355756466 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3080160c0 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff301ffb640 1 -- 172.21.15.103:0/2355756466 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff308016260 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3140054d0 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff301ffb640 1 -- 172.21.15.103:0/2355756466 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff308035070 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 10 client.16479 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5605b1ebb950 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff301ffb640 1 -- 172.21.15.103:0/2355756466 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3140054d0 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff301ffb640 1 -- 172.21.15.103:0/2355756466 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff30802e780 con 0x5605b1ea2910 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 10 client.16479 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 20 client.16479 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1ebcf20 0x5605b1ebf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 4 client.16479 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.872+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b1e14fb0 con 0x5605b1ebcf20 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.873+0000 7ff31e247640 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1ebcf20 0x5605b1ebf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.873+0000 7ff301ffb640 10 client.16479 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.873+0000 7ff31e247640 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1ebcf20 0x5605b1ebf340 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17e06fdec0514bd server_cookie=783a42c7910527bc in_seq=0 out_seq=0 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff301ffb640 1 -- 172.21.15.103:0/2355756466 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605b1e14fb0 con 0x5605b1ebcf20 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff301ffb640 10 client.16479 handle_command_reply: tid=0 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 client.16479 shutdown 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 msgr2=0x5605b1eaa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eaa590 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff308005e10 tx=0x7ff308016c20 comp rx=0 tx=0).stop 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 shutdown_connections 2024-08-19T01:39:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1ebcf20 0x5605b1ebf340 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1eaaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1eaa590 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff301ffb640 0 client.16479 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 --2- 172.21.15.103:0/2355756466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1eaa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 >> 172.21.15.103:0/2355756466 conn(0x5605b1e9e580 msgr2=0x5605b1ea42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 shutdown_connections 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 1 -- 172.21.15.103:0/2355756466 wait complete. 2024-08-19T01:39:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.874+0000 7ff3228a9740 20 client.16479 trim_cache size 0 max 16384 2024-08-19T01:39:55.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.891+0000 7ff3228a9740 1 Processor -- start 2024-08-19T01:39:55.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.891+0000 7ff3228a9740 1 -- start start 2024-08-19T01:39:55.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.891+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ec4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.891+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1ec5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.891+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1ea87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.891+0000 7ff3228a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181aadf0 con 0x5605b1ea2910 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff3228a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181aca50 con 0x5605b1ead1d0 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff3228a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181ab4c0 con 0x5605b1ea1ff0 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1ec5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1ec5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60796/0 (socket says 172.21.15.103:60796) 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 -- 172.21.15.103:0/463267162 learned_addr learned my addr 172.21.15.103:0/463267162 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31e247640 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ec4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31ea48640 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1ea87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 -- 172.21.15.103:0/463267162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 msgr2=0x5605b1ec4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ec4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 -- 172.21.15.103:0/463267162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 msgr2=0x5605b1ea87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1ea87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 -- 172.21.15.103:0/463267162 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b1ea1e20 con 0x5605b1ea2910 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff31da46640 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1ec5190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff30802ebf0 tx=0x7ff30800afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803739b6d2816150 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff303fff640 1 -- 172.21.15.103:0/463267162 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30803e020 con 0x5605b1ea2910 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff303fff640 1 -- 172.21.15.103:0/463267162 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff308014e50 con 0x5605b1ea2910 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.892+0000 7ff303fff640 1 -- 172.21.15.103:0/463267162 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30802ee60 con 0x5605b1ea2910 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 -- 172.21.15.103:0/463267162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 msgr2=0x5605b1ec5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1ec5190 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff30802ebf0 tx=0x7ff30800afa0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 -- 172.21.15.103:0/463267162 shutdown_connections 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ead1d0 0x5605b1ea87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ea2910 0x5605b1ec5190 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 --2- 172.21.15.103:0/463267162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea1ff0 0x5605b1ec4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 -- 172.21.15.103:0/463267162 >> 172.21.15.103:0/463267162 conn(0x5605b1e9e580 msgr2=0x5605b1e9cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:55.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 -- 172.21.15.103:0/463267162 shutdown_connections 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.893+0000 7ff3228a9740 1 -- 172.21.15.103:0/463267162 wait complete. 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 -- start start 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 Processor -- start 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ea1ff0 0x5605b1eb5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea2910 0x5605b1eb5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ead1d0 0x5605b1ea6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181ac2c0 con 0x5605b1ead1d0 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181abb90 con 0x5605b1ea1ff0 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181a8e10 con 0x5605b1ea2910 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff3228a9740 10 client.0 _ll_register_callbacks cb 0x7ff308005d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff31e247640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ea1ff0 0x5605b1eb5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff31ea48640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ead1d0 0x5605b1ea6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff31e247640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ea1ff0 0x5605b1eb5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:36928/0 (socket says 172.21.15.103:36928) 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff31e247640 1 -- 172.21.15.103:0/3113386010 learned_addr learned my addr 172.21.15.103:0/3113386010 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.894+0000 7ff31da46640 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea2910 0x5605b1eb5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff31e247640 1 -- 172.21.15.103:0/3113386010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea2910 msgr2=0x5605b1eb5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff31e247640 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea2910 0x5605b1eb5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff31e247640 1 -- 172.21.15.103:0/3113386010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ead1d0 msgr2=0x5605b1ea6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff31e247640 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ead1d0 0x5605b1ea6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff31e247640 1 -- 172.21.15.103:0/3113386010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b1ea4e60 con 0x5605b1ea1ff0 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff31ea48640 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ead1d0 0x5605b1ea6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff31e247640 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ea1ff0 0x5605b1eb5500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff31400f010 tx=0x7ff314008780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9151490699e2e3d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff301ffb640 1 -- 172.21.15.103:0/3113386010 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31402a070 con 0x5605b1ea1ff0 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff308014e50 con 0x5605b1ea1ff0 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff301ffb640 1 -- 172.21.15.103:0/3113386010 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff314005520 con 0x5605b1ea1ff0 2024-08-19T01:39:55.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff301ffb640 1 -- 172.21.15.103:0/3113386010 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff314036050 con 0x5605b1ea1ff0 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff3228a9740 10 client.25892 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.895+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5605b1f27780 con 0x5605b1ea1ff0 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff301ffb640 1 -- 172.21.15.103:0/3113386010 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff308014e50 con 0x5605b1ea1ff0 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff301ffb640 1 -- 172.21.15.103:0/3113386010 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff3140105d0 con 0x5605b1ea1ff0 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff3228a9740 10 client.25892 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff3228a9740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff3228a9740 20 client.25892 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff3228a9740 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1f27d20 0x5605b1f2a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff3228a9740 4 client.25892 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605b1e0f4c0 con 0x5605b1f27d20 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff31da46640 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1f27d20 0x5605b1f2a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff31da46640 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1f27d20 0x5605b1f2a140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40bc6faba39e5313 server_cookie=e7808cf0b34b8e9e in_seq=0 out_seq=0 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.896+0000 7ff301ffb640 10 client.25892 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:55.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff301ffb640 1 -- 172.21.15.103:0/3113386010 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605b1e0f4c0 con 0x5605b1f27d20 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff301ffb640 10 client.25892 handle_command_reply: tid=0 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 client.25892 shutdown 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ea1ff0 msgr2=0x5605b1eb5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ea1ff0 0x5605b1eb5500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff31400f010 tx=0x7ff314008780 comp rx=0 tx=0).stop 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 shutdown_connections 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5605b1f27d20 0x5605b1f2a140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605b1ead1d0 0x5605b1ea6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605b1ea2910 0x5605b1eb5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 --2- 172.21.15.103:0/3113386010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5605b1ea1ff0 0x5605b1eb5500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff301ffb640 0 client.25892 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 >> 172.21.15.103:0/3113386010 conn(0x5605b1e9e580 msgr2=0x5605b1ea3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 shutdown_connections 2024-08-19T01:39:55.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 1 -- 172.21.15.103:0/3113386010 wait complete. 2024-08-19T01:39:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.897+0000 7ff3228a9740 20 client.25892 trim_cache size 0 max 16384 2024-08-19T01:39:55.907 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3040a2c00 msgr2=0x7ff3040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3040a2c00 0x7ff3040a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff30800b310 tx=0x7ff308016ea0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 msgr2=0x7ff318101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff318101140 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff310005cc0 tx=0x7ff310003ba0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 shutdown_connections 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff3040a2c00 0x7ff3040a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff318107fd0 0x7ff318101140 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff318103410 0x7ff318100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 --2- 172.21.15.103:0/1047233986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff318102a60 0x7ff3181006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 >> 172.21.15.103:0/1047233986 conn(0x7ff3180fc500 msgr2=0x7ff3180fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 shutdown_connections 2024-08-19T01:39:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:55.898+0000 7ff3204ab640 1 -- 172.21.15.103:0/1047233986 wait complete. 2024-08-19T01:39:56.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:55 smithi103 ceph-mon[18436]: pgmap v463: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:39:56.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:55 smithi146 ceph-mon[22098]: pgmap v463: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:39:56.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:55 smithi190 ceph-mon[22032]: pgmap v463: 97 pgs: 97 active+clean; 5.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:39:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:56 smithi103 ceph-mon[18436]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:56 smithi190 ceph-mon[22032]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:57.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:56 smithi146 ceph-mon[22098]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:39:57.759 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:39:57.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:39:58.037 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:39:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 -- 172.21.15.103:0/1666268765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 msgr2=0x7f5400100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 --2- 172.21.15.103:0/1666268765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f5400100f30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009900 tx=0x7f53f002a480 comp rx=0 tx=0).stop 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 -- 172.21.15.103:0/1666268765 shutdown_connections 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 --2- 172.21.15.103:0/1666268765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5400102c20 0x7f540010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 --2- 172.21.15.103:0/1666268765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54001022a0 0x7f54001026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 --2- 172.21.15.103:0/1666268765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f5400100f30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 -- 172.21.15.103:0/1666268765 >> 172.21.15.103:0/1666268765 conn(0x7f54000fc500 msgr2=0x7f54000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 -- 172.21.15.103:0/1666268765 shutdown_connections 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.575+0000 7f54060e9640 1 -- 172.21.15.103:0/1666268765 wait complete. 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.576+0000 7f54060e9640 1 Processor -- start 2024-08-19T01:39:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.576+0000 7f54060e9640 1 -- start start 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.576+0000 7f54060e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f54001970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f54060e9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54001022a0 0x7f5400197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f54060e9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5400102c20 0x7f540019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f54060e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5400110b20 con 0x7f5400100b50 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f54060e9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54001109a0 con 0x7f5400102c20 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f54060e9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5400110ca0 con 0x7f54001022a0 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f54001970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f54001970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60814/0 (socket says 172.21.15.103:60814) 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 -- 172.21.15.103:0/3810772321 learned_addr learned my addr 172.21.15.103:0/3810772321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53fffff640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5400102c20 0x7f540019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53feffd640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54001022a0 0x7f5400197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 -- 172.21.15.103:0/3810772321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54001022a0 msgr2=0x7f5400197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54001022a0 0x7f5400197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 -- 172.21.15.103:0/3810772321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5400102c20 msgr2=0x7f540019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5400102c20 0x7f540019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 -- 172.21.15.103:0/3810772321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540019c070 con 0x7f5400100b50 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53fffff640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5400102c20 0x7f540019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f54001970c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f53f0006010 tx=0x7f53f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53fcff9640 1 -- 172.21.15.103:0/3810772321 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0046070 con 0x7f5400100b50 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540019c300 con 0x7f5400100b50 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.577+0000 7f53fcff9640 1 -- 172.21.15.103:0/3810772321 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f0003aa0 con 0x7f5400100b50 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.578+0000 7f53fcff9640 1 -- 172.21.15.103:0/3810772321 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0035050 con 0x7f5400100b50 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.578+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5400069ca0 con 0x7f5400100b50 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.580+0000 7f53fcff9640 1 -- 172.21.15.103:0/3810772321 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f53f0049070 con 0x7f5400100b50 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 Processor -- start 2024-08-19T01:39:58.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53fcff9640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53e80a2b80 0x7f53e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 -- start start 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53fffff640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53e80a2b80 0x7f53e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53fcff9640 1 -- 172.21.15.103:0/3810772321 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f53f00f0cb0 con 0x7f5400100b50 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261100 0x564fea2614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea2702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea272be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564fea224e40 con 0x564fea2707f0 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564fea1c67f0 con 0x564fea261100 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f54084e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564fea16dfb0 con 0x564fea261a20 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea272be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53feffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea2702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53ff7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261100 0x564fea2614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53feffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea2702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:40830/0 (socket says 172.21.15.103:40830) 2024-08-19T01:39:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.581+0000 7f53feffd640 1 -- 172.21.15.103:0/683063528 learned_addr learned my addr 172.21.15.103:0/683063528 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53fffff640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53e80a2b80 0x7f53e80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f53f80092f0 tx=0x7f53f8010740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53fffff640 1 -- 172.21.15.103:0/683063528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 msgr2=0x564fea2702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53fffff640 1 --2- 172.21.15.103:0/683063528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea2702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53fffff640 1 -- 172.21.15.103:0/683063528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261100 msgr2=0x564fea2614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53fffff640 1 --2- 172.21.15.103:0/683063528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261100 0x564fea2614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53fffff640 1 -- 172.21.15.103:0/683063528 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe9f386a0 con 0x564fea2707f0 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53fffff640 1 --2- 172.21.15.103:0/683063528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea272be0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f53f8005fd0 tx=0x7f53f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3ca2f4ceeb3dfc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53e5ffb640 1 -- 172.21.15.103:0/683063528 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f801e6e0 con 0x564fea2707f0 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53e5ffb640 1 -- 172.21.15.103:0/683063528 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f801ed30 con 0x564fea2707f0 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.582+0000 7f53e5ffb640 1 -- 172.21.15.103:0/683063528 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8011d90 con 0x564fea2707f0 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 -- 172.21.15.103:0/683063528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 msgr2=0x564fea272be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 --2- 172.21.15.103:0/683063528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea272be0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f53f8005fd0 tx=0x7f53f801b000 comp rx=0 tx=0).stop 2024-08-19T01:39:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 -- 172.21.15.103:0/683063528 shutdown_connections 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 --2- 172.21.15.103:0/683063528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea272be0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 --2- 172.21.15.103:0/683063528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea2702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 --2- 172.21.15.103:0/683063528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261100 0x564fea2614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 -- 172.21.15.103:0/683063528 >> 172.21.15.103:0/683063528 conn(0x564fea25d790 msgr2=0x564fea25db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 -- 172.21.15.103:0/683063528 shutdown_connections 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 -- 172.21.15.103:0/683063528 wait complete. 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.583+0000 7f54084e7740 1 -- start start 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 1 Processor -- start 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea26d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261a20 0x564fea26dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea26e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564fea181cb0 con 0x564fea2707f0 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564fea1c27b0 con 0x564fea261a20 2024-08-19T01:39:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564fea273420 con 0x564fea261100 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f54084e7740 10 client.0 _ll_register_callbacks cb 0x564fe9ed5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f53fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea26e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f53fffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea26e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60834/0 (socket says 172.21.15.103:60834) 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f53fffff640 1 -- 172.21.15.103:0/3431813909 learned_addr learned my addr 172.21.15.103:0/3431813909 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea26d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.584+0000 7f53feffd640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261a20 0x564fea26dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53fffff640 1 -- 172.21.15.103:0/3431813909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 msgr2=0x564fea26d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53fffff640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea26d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53fffff640 1 -- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261a20 msgr2=0x564fea26dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53fffff640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261a20 0x564fea26dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53fffff640 1 -- 172.21.15.103:0/3431813909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fea180bc0 con 0x564fea2707f0 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53feffd640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261a20 0x564fea26dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53fffff640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea26e0e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011df0 tx=0x7f53f8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c172bf1fa54943 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53deffd640 1 -- 172.21.15.103:0/3431813909 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8034060 con 0x564fea2707f0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f801ed30 con 0x564fea2707f0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53deffd640 1 -- 172.21.15.103:0/3431813909 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f53f8013260 con 0x564fea2707f0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53deffd640 1 -- 172.21.15.103:0/3431813909 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53f80113e0 con 0x564fea2707f0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f54084e7740 10 client.16497 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564fea27f2f0 con 0x564fea2707f0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.585+0000 7f53deffd640 1 -- 172.21.15.103:0/3431813909 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f801ed30 con 0x564fea2707f0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f53deffd640 1 -- 172.21.15.103:0/3431813909 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f53f800f040 con 0x564fea2707f0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f54084e7740 10 client.16497 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f54084e7740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f54084e7740 20 client.16497 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f54084e7740 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2806a0 0x564fea282a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f54084e7740 4 client.16497 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fea1d3d60 con 0x564fea2806a0 2024-08-19T01:39:58.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.586+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2806a0 0x564fea282a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.587+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2806a0 0x564fea282a90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dee74b16ca6593a server_cookie=8d7360a1edcf3177 in_seq=0 out_seq=0 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.587+0000 7f53deffd640 10 client.16497 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.587+0000 7f53deffd640 1 -- 172.21.15.103:0/3431813909 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fea1d3d60 con 0x564fea2806a0 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.587+0000 7f53deffd640 10 client.16497 handle_command_reply: tid=0 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.587+0000 7f54084e7740 1 client.16497 shutdown 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.587+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 msgr2=0x564fea26e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.587+0000 7f54084e7740 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea26e0e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011df0 tx=0x7f53f8011e20 comp rx=0 tx=0).stop 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 shutdown_connections 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2806a0 0x564fea282a90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea2707f0 0x564fea26e0e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea261a20 0x564fea26dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 --2- 172.21.15.103:0/3431813909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea26d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f53deffd640 0 client.16497 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 >> 172.21.15.103:0/3431813909 conn(0x564fea25d790 msgr2=0x564fea2634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:58.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 shutdown_connections 2024-08-19T01:39:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 1 -- 172.21.15.103:0/3431813909 wait complete. 2024-08-19T01:39:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.588+0000 7f54084e7740 20 client.16497 trim_cache size 0 max 16384 2024-08-19T01:39:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.608+0000 7f54084e7740 1 Processor -- start 2024-08-19T01:39:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.608+0000 7f54084e7740 1 -- start start 2024-08-19T01:39:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.608+0000 7f54084e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea2584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f54084e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261a20 0x564fea26f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f54084e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea288180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f54084e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54001aadf0 con 0x564fea261a20 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f54084e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54001aca50 con 0x564fea2707f0 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f54084e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54001ab4c0 con 0x564fea261100 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261a20 0x564fea26f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261a20 0x564fea26f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60846/0 (socket says 172.21.15.103:60846) 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 -- 172.21.15.103:0/2387320849 learned_addr learned my addr 172.21.15.103:0/2387320849 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53fffff640 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea288180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea2584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 -- 172.21.15.103:0/2387320849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 msgr2=0x564fea2584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea2584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 -- 172.21.15.103:0/2387320849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 msgr2=0x564fea288180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea288180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 -- 172.21.15.103:0/2387320849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fea26c610 con 0x564fea261a20 2024-08-19T01:39:58.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53feffd640 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261a20 0x564fea26f5d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f53f400cc20 tx=0x7f53f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3df7e4cbc3e5ce5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:58.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.609+0000 7f53e57fa640 1 -- 172.21.15.103:0/2387320849 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4004020 con 0x564fea261a20 2024-08-19T01:39:58.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.610+0000 7f53e57fa640 1 -- 172.21.15.103:0/2387320849 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f40054d0 con 0x564fea261a20 2024-08-19T01:39:58.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.610+0000 7f53e57fa640 1 -- 172.21.15.103:0/2387320849 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4004c60 con 0x564fea261a20 2024-08-19T01:39:58.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.610+0000 7f54084e7740 1 -- 172.21.15.103:0/2387320849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261a20 msgr2=0x564fea26f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.610+0000 7f54084e7740 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261a20 0x564fea26f5d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f53f400cc20 tx=0x7f53f4011810 comp rx=0 tx=0).stop 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 -- 172.21.15.103:0/2387320849 shutdown_connections 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea288180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261a20 0x564fea26f5d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 --2- 172.21.15.103:0/2387320849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261100 0x564fea2584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 -- 172.21.15.103:0/2387320849 >> 172.21.15.103:0/2387320849 conn(0x564fea25d790 msgr2=0x564fea26eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 -- 172.21.15.103:0/2387320849 shutdown_connections 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 -- 172.21.15.103:0/2387320849 wait complete. 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 -- start start 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 Processor -- start 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261100 0x564fea273df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea274330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea2e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54001ac2c0 con 0x564fea261100 2024-08-19T01:39:58.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.611+0000 7f54084e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f54001abb90 con 0x564fea2707f0 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f54084e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54001a8df0 con 0x564fea261a20 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f54084e7740 10 client.0 _ll_register_callbacks cb 0x7f53f8013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261100 0x564fea273df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261100 0x564fea273df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60852/0 (socket says 172.21.15.103:60852) 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 -- 172.21.15.103:0/1579500100 learned_addr learned my addr 172.21.15.103:0/1579500100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53feffd640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea274330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53fffff640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea2e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 -- 172.21.15.103:0/1579500100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 msgr2=0x564fea274330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea274330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 -- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 msgr2=0x564fea2e3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea2e3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 -- 172.21.15.103:0/1579500100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fea261850 con 0x564fea261100 2024-08-19T01:39:58.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53feffd640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea274330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53ff7fe640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261100 0x564fea273df0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f53f00b9eb0 tx=0x7f53f0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79775c9030e0781c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53deffd640 1 -- 172.21.15.103:0/1579500100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00d6060 con 0x564fea261100 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f53deffd640 1 -- 172.21.15.103:0/1579500100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f53f0003c50 con 0x564fea261100 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.612+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f40054d0 con 0x564fea261100 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f53deffd640 1 -- 172.21.15.103:0/1579500100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53f0038340 con 0x564fea261100 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 10 client.16503 fetch_fsmap learned FSMap version 32 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564fea2ea9f0 con 0x564fea261100 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f53deffd640 1 -- 172.21.15.103:0/1579500100 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f40054d0 con 0x564fea261100 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f53deffd640 1 -- 172.21.15.103:0/1579500100 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f53f00d6200 con 0x564fea261100 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 10 client.16503 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:39:58.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 20 client.16503 populate_metadata read hostname 'smithi103' 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2eb130 0x564fea2ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 4 client.16503 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564fea1ce270 con 0x564fea2eb130 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.613+0000 7f53feffd640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2eb130 0x564fea2ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.614+0000 7f53deffd640 10 client.16503 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.614+0000 7f53feffd640 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2eb130 0x564fea2ed550 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfefe612be170c90 server_cookie=3cec20bbf1026c22 in_seq=0 out_seq=0 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f53deffd640 1 -- 172.21.15.103:0/1579500100 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564fea1ce270 con 0x564fea2eb130 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f53deffd640 10 client.16503 handle_command_reply: tid=0 2024-08-19T01:39:58.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 client.16503 shutdown 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261100 msgr2=0x564fea273df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261100 0x564fea273df0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f53f00b9eb0 tx=0x7f53f0009680 comp rx=0 tx=0).stop 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 shutdown_connections 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564fea2eb130 0x564fea2ed550 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564fea2707f0 0x564fea2e3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564fea261a20 0x564fea274330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 --2- 172.21.15.103:0/1579500100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564fea261100 0x564fea273df0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 >> 172.21.15.103:0/1579500100 conn(0x564fea25d790 msgr2=0x564fea26f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:58.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f53deffd640 0 client.16503 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:39:58.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 shutdown_connections 2024-08-19T01:39:58.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 1 -- 172.21.15.103:0/1579500100 wait complete. 2024-08-19T01:39:58.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.615+0000 7f54084e7740 20 client.16503 trim_cache size 0 max 16384 2024-08-19T01:39:58.622 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:39:58.622 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:39:58.622 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "3888db24-ee35-4859-8224-ecad58145aed", 2024-08-19T01:39:58.623 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:39:58.623 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:39:58.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53e80a2b80 msgr2=0x7f53e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53e80a2b80 0x7f53e80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f53f80092f0 tx=0x7f53f8010740 comp rx=0 tx=0).stop 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 msgr2=0x7f54001970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f54001970c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f53f0006010 tx=0x7f53f0004270 comp rx=0 tx=0).stop 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 shutdown_connections 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f53e80a2b80 0x7f53e80a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5400102c20 0x7f540019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54001022a0 0x7f5400197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 --2- 172.21.15.103:0/3810772321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5400100b50 0x7f54001970c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 >> 172.21.15.103:0/3810772321 conn(0x7f54000fc500 msgr2=0x7f54000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 shutdown_connections 2024-08-19T01:39:58.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:39:58.616+0000 7f54060e9640 1 -- 172.21.15.103:0/3810772321 wait complete. 2024-08-19T01:39:59.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:39:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:00.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:39:59 smithi103 ceph-mon[18436]: pgmap v465: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:40:00.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:40:00.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:00.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:39:59 smithi146 ceph-mon[22098]: pgmap v465: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:40:00.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 -- 172.21.15.103:0/2708848770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241007b0 msgr2=0x7f9024100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 --2- 172.21.15.103:0/2708848770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241007b0 0x7f9024100c10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9018009930 tx=0x7f901802a420 comp rx=0 tx=0).stop 2024-08-19T01:40:00.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 -- 172.21.15.103:0/2708848770 shutdown_connections 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 --2- 172.21.15.103:0/2708848770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9024101150 0x7f902410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 --2- 172.21.15.103:0/2708848770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241007b0 0x7f9024100c10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 --2- 172.21.15.103:0/2708848770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90241067b0 0x7f9024106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 -- 172.21.15.103:0/2708848770 >> 172.21.15.103:0/2708848770 conn(0x7f90240fc4d0 msgr2=0x7f90240fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 -- 172.21.15.103:0/2708848770 shutdown_connections 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 -- 172.21.15.103:0/2708848770 wait complete. 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.254+0000 7f9029e21640 1 Processor -- start 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9029e21640 1 -- start start 2024-08-19T01:40:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9029e21640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90241007b0 0x7f9024198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9029e21640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9024101150 0x7f90241994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9029e21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241067b0 0x7f90241a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9029e21640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110a20 con 0x7f90241067b0 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9029e21640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90241108a0 con 0x7f90241007b0 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9029e21640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110ba0 con 0x7f9024101150 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9023fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241067b0 0x7f90241a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9023fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241067b0 0x7f90241a3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60870/0 (socket says 172.21.15.103:60870) 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9023fff640 1 -- 172.21.15.103:0/2265584402 learned_addr learned my addr 172.21.15.103:0/2265584402 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9022ffd640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9024101150 0x7f90241994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f90237fe640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90241007b0 0x7f9024198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9023fff640 1 -- 172.21.15.103:0/2265584402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9024101150 msgr2=0x7f90241994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9023fff640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9024101150 0x7f90241994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.255+0000 7f9023fff640 1 -- 172.21.15.103:0/2265584402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90241007b0 msgr2=0x7f9024198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9023fff640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90241007b0 0x7f9024198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9023fff640 1 -- 172.21.15.103:0/2265584402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90241a4530 con 0x7f90241067b0 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f90237fe640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90241007b0 0x7f9024198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9023fff640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241067b0 0x7f90241a3ff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f901c00cad0 tx=0x7f901c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9020ff9640 1 -- 172.21.15.103:0/2265584402 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c00ae20 con 0x7f90241067b0 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9020ff9640 1 -- 172.21.15.103:0/2265584402 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f901c005020 con 0x7f90241067b0 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90241a4700 con 0x7f90241067b0 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90241a4bc0 con 0x7f90241067b0 2024-08-19T01:40:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.256+0000 7f9020ff9640 1 -- 172.21.15.103:0/2265584402 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c00f6b0 con 0x7f90241067b0 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.259+0000 7f9020ff9640 1 -- 172.21.15.103:0/2265584402 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f901c0051d0 con 0x7f90241067b0 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.259+0000 7f902c21f740 1 Processor -- start 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.259+0000 7f902c21f740 1 -- start start 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.259+0000 7f902c21f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7210 0x5615238b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f902c21f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7b30 0x5615238c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f902c21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238c23f0 0x5615238c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f902c21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56152381c980 con 0x5615238c23f0 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f902c21f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5615237b4a70 con 0x5615238b7210 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f902c21f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561523818940 con 0x5615238b7b30 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9020ff9640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f900c0a2b80 0x7f900c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9022ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238c23f0 0x5615238c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f90237fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7b30 0x5615238c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9022ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238c23f0 0x5615238c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60884/0 (socket says 172.21.15.103:60884) 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9022ffd640 1 -- 172.21.15.103:0/937387614 learned_addr learned my addr 172.21.15.103:0/937387614 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9022ffd640 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7210 0x5615238b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9023fff640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f900c0a2b80 0x7f900c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9020ff9640 1 -- 172.21.15.103:0/2265584402 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f901c0cf0b0 con 0x7f90241067b0 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f90237fe640 1 -- 172.21.15.103:0/937387614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7210 msgr2=0x5615238b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f90237fe640 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7210 0x5615238b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f90237fe640 1 -- 172.21.15.103:0/937387614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238c23f0 msgr2=0x5615238c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f90237fe640 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238c23f0 0x5615238c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f90237fe640 1 -- 172.21.15.103:0/937387614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615238176e0 con 0x5615238b7b30 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f9022ffd640 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238c23f0 0x5615238c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.260+0000 7f90237fe640 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7b30 0x5615238c1eb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9014005fd0 tx=0x7f901400b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c799b9dd5c9241f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f9023fff640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f900c0a2b80 0x7f900c0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f901c098310 tx=0x7f901c01baf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f9009ffb640 1 -- 172.21.15.103:0/937387614 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014004030 con 0x5615238b7b30 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f9009ffb640 1 -- 172.21.15.103:0/937387614 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90140041d0 con 0x5615238b7b30 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f9009ffb640 1 -- 172.21.15.103:0/937387614 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901401a660 con 0x5615238b7b30 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f902c21f740 1 -- 172.21.15.103:0/937387614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7b30 msgr2=0x5615238c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f902c21f740 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7b30 0x5615238c1eb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9014005fd0 tx=0x7f901400b770 comp rx=0 tx=0).stop 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f902c21f740 1 -- 172.21.15.103:0/937387614 shutdown_connections 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f902c21f740 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238c23f0 0x5615238c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.261+0000 7f902c21f740 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7b30 0x5615238c1eb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 --2- 172.21.15.103:0/937387614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7210 0x5615238b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 -- 172.21.15.103:0/937387614 >> 172.21.15.103:0/937387614 conn(0x5615238b37a0 msgr2=0x5615238b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 -- 172.21.15.103:0/937387614 shutdown_connections 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 -- 172.21.15.103:0/937387614 wait complete. 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 -- start start 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 Processor -- start 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.262+0000 7f902c21f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 0x5615238bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f902c21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615238c4fc0 con 0x5615238b7210 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f902c21f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5615237d7aa0 con 0x5615238b7b30 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f902c21f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5615238c5140 con 0x5615238c23f0 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f902c21f740 10 client.0 _ll_register_callbacks cb 0x5615237c6e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238bf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60892/0 (socket says 172.21.15.103:60892) 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 -- 172.21.15.103:0/1164043984 learned_addr learned my addr 172.21.15.103:0/1164043984 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f90237fe640 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 -- 172.21.15.103:0/1164043984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 msgr2=0x5615238bfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 0x5615238bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 -- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 msgr2=0x5615238bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 -- 172.21.15.103:0/1164043984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615237bb650 con 0x5615238b7210 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9022ffd640 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238bf2a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9018009900 tx=0x7f9018031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56138357dddb5e54 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9002ffd640 1 -- 172.21.15.103:0/1164043984 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018035050 con 0x5615238b7210 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f9002ffd640 1 -- 172.21.15.103:0/1164043984 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9018004240 con 0x5615238b7210 2024-08-19T01:40:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.263+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90140041d0 con 0x5615238b7210 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f9002ffd640 1 -- 172.21.15.103:0/1164043984 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9018004440 con 0x5615238b7210 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 10 client.16515 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56152358e3c0 con 0x5615238b7210 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f9002ffd640 1 -- 172.21.15.103:0/1164043984 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90140041d0 con 0x5615238b7210 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f9002ffd640 1 -- 172.21.15.103:0/1164043984 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f901803d6e0 con 0x5615238b7210 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 10 client.16515 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 20 client.16515 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5615238d1ed0 0x5615238d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 4 client.16515 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561523829ef0 con 0x5615238d1ed0 2024-08-19T01:40:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.264+0000 7f90237fe640 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5615238d1ed0 0x5615238d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.265+0000 7f90237fe640 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5615238d1ed0 0x5615238d4310 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f5e02c4ec98603e server_cookie=7f4a409aca87c89e in_seq=0 out_seq=0 2024-08-19T01:40:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.265+0000 7f9002ffd640 10 client.16515 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.265+0000 7f9002ffd640 1 -- 172.21.15.103:0/1164043984 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561523829ef0 con 0x5615238d1ed0 2024-08-19T01:40:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.265+0000 7f9002ffd640 10 client.16515 handle_command_reply: tid=0 2024-08-19T01:40:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 client.16515 shutdown 2024-08-19T01:40:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 msgr2=0x5615238bf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238bf2a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9018009900 tx=0x7f9018031760 comp rx=0 tx=0).stop 2024-08-19T01:40:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 shutdown_connections 2024-08-19T01:40:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5615238d1ed0 0x5615238d4310 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 0x5615238bfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f9002ffd640 0 client.16515 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 --2- 172.21.15.103:0/1164043984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238bf2a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 >> 172.21.15.103:0/1164043984 conn(0x5615238b37a0 msgr2=0x5615238b9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 shutdown_connections 2024-08-19T01:40:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 1 -- 172.21.15.103:0/1164043984 wait complete. 2024-08-19T01:40:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.266+0000 7f902c21f740 20 client.16515 trim_cache size 0 max 16384 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 Processor -- start 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 -- start start 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 0x5615238da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90241aaf10 con 0x5615238b7210 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90241acb70 con 0x5615238b7b30 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.287+0000 7f902c21f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f90241ab5e0 con 0x5615238c23f0 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238b9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60894/0 (socket says 172.21.15.103:60894) 2024-08-19T01:40:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 -- 172.21.15.103:0/2246006268 learned_addr learned my addr 172.21.15.103:0/2246006268 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f90237fe640 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 -- 172.21.15.103:0/2246006268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 msgr2=0x5615238da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 0x5615238da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 -- 172.21.15.103:0/2246006268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 msgr2=0x5615238d9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238d9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 -- 172.21.15.103:0/2246006268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615238b7960 con 0x5615238b7210 2024-08-19T01:40:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f9022ffd640 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238b9c70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f901802a940 tx=0x7f9018032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0e2310d80a4188 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f90097fa640 1 -- 172.21.15.103:0/2246006268 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018059070 con 0x5615238b7210 2024-08-19T01:40:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f90097fa640 1 -- 172.21.15.103:0/2246006268 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9018043070 con 0x5615238b7210 2024-08-19T01:40:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.288+0000 7f90097fa640 1 -- 172.21.15.103:0/2246006268 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018035070 con 0x5615238b7210 2024-08-19T01:40:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 -- 172.21.15.103:0/2246006268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 msgr2=0x5615238b9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238b9c70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f901802a940 tx=0x7f9018032c20 comp rx=0 tx=0).stop 2024-08-19T01:40:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 -- 172.21.15.103:0/2246006268 shutdown_connections 2024-08-19T01:40:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238c23f0 0x5615238da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238b7b30 0x5615238d9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 --2- 172.21.15.103:0/2246006268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7210 0x5615238b9c70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 -- 172.21.15.103:0/2246006268 >> 172.21.15.103:0/2246006268 conn(0x5615238b37a0 msgr2=0x5615238c1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 -- 172.21.15.103:0/2246006268 shutdown_connections 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.289+0000 7f902c21f740 1 -- 172.21.15.103:0/2246006268 wait complete. 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 -- start start 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 Processor -- start 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7210 0x5615238c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7b30 0x5615238c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238c23f0 0x5615238ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90241ac3e0 con 0x5615238b7b30 2024-08-19T01:40:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90241abcb0 con 0x5615238c23f0 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f90241a8ef0 con 0x5615238b7210 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f902c21f740 10 client.0 _ll_register_callbacks cb 0x7f9018032e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.290+0000 7f90237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7b30 0x5615238c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7b30 0x5615238c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60896/0 (socket says 172.21.15.103:60896) 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 -- 172.21.15.103:0/586249091 learned_addr learned my addr 172.21.15.103:0/586249091 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f9022ffd640 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238c23f0 0x5615238ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f9022ffd640 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7210 0x5615238c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 -- 172.21.15.103:0/586249091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7210 msgr2=0x5615238c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7210 0x5615238c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 -- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238c23f0 msgr2=0x5615238ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238c23f0 0x5615238ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 -- 172.21.15.103:0/586249091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615238ca770 con 0x5615238b7b30 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f90237fe640 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7b30 0x5615238c6110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x5615238b4600 tx=0x7f90140041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691aa18b025e873c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f9002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f9002ffd640 1 -- 172.21.15.103:0/586249091 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014033060 con 0x5615238b7b30 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9018043070 con 0x5615238b7b30 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.291+0000 7f9002ffd640 1 -- 172.21.15.103:0/586249091 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9014020040 con 0x5615238b7b30 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f9002ffd640 1 -- 172.21.15.103:0/586249091 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9014004400 con 0x5615238b7b30 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 10 client.16521 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56152393c670 con 0x5615238b7b30 2024-08-19T01:40:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f9002ffd640 1 -- 172.21.15.103:0/586249091 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9014003aa0 con 0x5615238b7b30 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f9002ffd640 1 -- 172.21.15.103:0/586249091 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f901402d030 con 0x5615238b7b30 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 10 client.16521 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 20 client.16521 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56152393cb60 0x56152393ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 4 client.16521 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.292+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9018043070 con 0x56152393cb60 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f9022ffd640 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56152393cb60 0x56152393ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f9022ffd640 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56152393cb60 0x56152393ef80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b6de5502a3eb3 server_cookie=6a1f700143689863 in_seq=0 out_seq=0 2024-08-19T01:40:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f9002ffd640 10 client.16521 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f9002ffd640 1 -- 172.21.15.103:0/586249091 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f9018043070 con 0x56152393cb60 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f9002ffd640 10 client.16521 handle_command_reply: tid=0 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 client.16521 shutdown 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7b30 msgr2=0x5615238c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7b30 0x5615238c6110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x5615238b4600 tx=0x7f90140041d0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 shutdown_connections 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56152393cb60 0x56152393ef80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5615238c23f0 0x5615238ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615238b7b30 0x5615238c6110 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 --2- 172.21.15.103:0/586249091 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5615238b7210 0x5615238c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f9002ffd640 0 client.16521 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.293+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 >> 172.21.15.103:0/586249091 conn(0x5615238b37a0 msgr2=0x5615238c0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 shutdown_connections 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f902c21f740 1 -- 172.21.15.103:0/586249091 wait complete. 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f902c21f740 20 client.16521 trim_cache size 0 max 16384 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stdout: "3888db24-ee35-4859-8224-ecad58145aed": { 2024-08-19T01:40:00.302 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:40:00.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "3888db24-ee35-4859-8224-ecad58145aed", 2024-08-19T01:40:00.303 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:40:00.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:40:00.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:40:00.303 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:40:00.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f900c0a2b80 msgr2=0x7f900c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f900c0a2b80 0x7f900c0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f901c098310 tx=0x7f901c01baf0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241067b0 msgr2=0x7f90241a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241067b0 0x7f90241a3ff0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f901c00cad0 tx=0x7f901c00cf90 comp rx=0 tx=0).stop 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 shutdown_connections 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f900c0a2b80 0x7f900c0a5040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90241067b0 0x7f90241a3ff0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9024101150 0x7f90241994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 --2- 172.21.15.103:0/2265584402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90241007b0 0x7f9024198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.294+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 >> 172.21.15.103:0/2265584402 conn(0x7f90240fc4d0 msgr2=0x7f90240fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.295+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 shutdown_connections 2024-08-19T01:40:00.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:00.295+0000 7f9029e21640 1 -- 172.21.15.103:0/2265584402 wait complete. 2024-08-19T01:40:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:40:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:00.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:00.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:00.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:00.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:00.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:39:59 smithi190 ceph-mon[22032]: pgmap v465: 97 pgs: 97 active+clean; 5.4 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:40:01.129 INFO:tasks.cephfs.filesystem:scrub status for tag:3888db24-ee35-4859-8224-ecad58145aed - {'path': '/', 'tag': '3888db24-ee35-4859-8224-ecad58145aed', 'options': 'recursive,force'} 2024-08-19T01:40:01.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:00 smithi146 ceph-mon[22098]: overall HEALTH_OK 2024-08-19T01:40:01.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:00 smithi103 ceph-mon[18436]: overall HEALTH_OK 2024-08-19T01:40:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:00 smithi190 ceph-mon[22032]: overall HEALTH_OK 2024-08-19T01:40:02.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:01 smithi146 ceph-mon[22098]: pgmap v466: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 32 op/s 2024-08-19T01:40:02.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:01 smithi103 ceph-mon[18436]: pgmap v466: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 32 op/s 2024-08-19T01:40:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:01 smithi190 ceph-mon[22032]: pgmap v466: 97 pgs: 97 active+clean; 5.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 32 op/s 2024-08-19T01:40:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 -- 172.21.15.103:0/4233987411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 msgr2=0x7f84e41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/4233987411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e41099c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009ca0 tx=0x7f84dc02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 -- 172.21.15.103:0/4233987411 shutdown_connections 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/4233987411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e41099c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/4233987411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e4104c80 0x7f84e4107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/4233987411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4102350 0x7f84e4104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 -- 172.21.15.103:0/4233987411 >> 172.21.15.103:0/4233987411 conn(0x7f84e40fbf10 msgr2=0x7f84e40fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 -- 172.21.15.103:0/4233987411 shutdown_connections 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.076+0000 7f84e8bb9640 1 -- 172.21.15.103:0/4233987411 wait complete. 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 Processor -- start 2024-08-19T01:40:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 -- start start 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e4102350 0x7f84e4101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4104c80 0x7f84e4100300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e4100840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84e410d3e0 con 0x7f84e41075b0 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f84e410d260 con 0x7f84e4104c80 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e8bb9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84e410d560 con 0x7f84e4102350 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e4100840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e4100840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34986/0 (socket says 172.21.15.103:34986) 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.077+0000 7f84e2d9d640 1 -- 172.21.15.103:0/1002879246 learned_addr learned my addr 172.21.15.103:0/1002879246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e259c640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e4102350 0x7f84e4101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4104c80 0x7f84e4100300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e2d9d640 1 -- 172.21.15.103:0/1002879246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e4102350 msgr2=0x7f84e4101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e4102350 0x7f84e4101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e2d9d640 1 -- 172.21.15.103:0/1002879246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4104c80 msgr2=0x7f84e4100300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4104c80 0x7f84e4100300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e2d9d640 1 -- 172.21.15.103:0/1002879246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e41010d0 con 0x7f84e41075b0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e4100840 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84dc031040 tx=0x7f84dc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84d37fe640 1 -- 172.21.15.103:0/1002879246 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004040 con 0x7f84e41075b0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e41a1b20 con 0x7f84e41075b0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84d37fe640 1 -- 172.21.15.103:0/1002879246 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84dc004690 con 0x7f84e41075b0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84d37fe640 1 -- 172.21.15.103:0/1002879246 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc030ad0 con 0x7f84e41075b0 2024-08-19T01:40:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.078+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e41a2070 con 0x7f84e41075b0 2024-08-19T01:40:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84d37fe640 1 -- 172.21.15.103:0/1002879246 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f84dc030c90 con 0x7f84e41075b0 2024-08-19T01:40:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 Processor -- start 2024-08-19T01:40:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 -- start start 2024-08-19T01:40:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb1aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1afc0 0x55f56cb1fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb20050 0x55f56cb22510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f56ca7d9d0 con 0x55f56cb1afc0 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f56ca25820 con 0x55f56cb20050 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84eba51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f56ca79990 con 0x55f56cb1a680 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.081+0000 7f84e259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1afc0 0x55f56cb1fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1afc0 0x55f56cb1fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34988/0 (socket says 172.21.15.103:34988) 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 -- 172.21.15.103:0/801759294 learned_addr learned my addr 172.21.15.103:0/801759294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb1aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb20050 0x55f56cb22510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 -- 172.21.15.103:0/801759294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 msgr2=0x55f56cb1aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb1aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 -- 172.21.15.103:0/801759294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb20050 msgr2=0x55f56cb22510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb20050 0x55f56cb22510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 -- 172.21.15.103:0/801759294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56c791210 con 0x55f56cb1afc0 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84d37fe640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f84cc0a2c30 0x7f84cc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb20050 0x55f56cb22510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f84cc0a2c30 0x7f84cc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84d37fe640 1 -- 172.21.15.103:0/1002879246 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f84dc0f0490 con 0x7f84e41075b0 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84e259c640 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1afc0 0x55f56cb1fa80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f84d400ad00 tx=0x7f84d400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1360ef60b4842a92 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84d0ff9640 1 -- 172.21.15.103:0/801759294 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d401d070 con 0x55f56cb1afc0 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84d0ff9640 1 -- 172.21.15.103:0/801759294 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84d4018af0 con 0x55f56cb1afc0 2024-08-19T01:40:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.082+0000 7f84d0ff9640 1 -- 172.21.15.103:0/801759294 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4010ae0 con 0x55f56cb1afc0 2024-08-19T01:40:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 -- 172.21.15.103:0/801759294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1afc0 msgr2=0x55f56cb1fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f84cc0a2c30 0x7f84cc0a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f84e4101a30 tx=0x7f84d8013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1afc0 0x55f56cb1fa80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f84d400ad00 tx=0x7f84d400f790 comp rx=0 tx=0).stop 2024-08-19T01:40:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 -- 172.21.15.103:0/801759294 shutdown_connections 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb20050 0x55f56cb22510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1afc0 0x55f56cb1fa80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 --2- 172.21.15.103:0/801759294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb1aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 -- 172.21.15.103:0/801759294 >> 172.21.15.103:0/801759294 conn(0x55f56cb15cf0 msgr2=0x55f56cb160f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 -- 172.21.15.103:0/801759294 shutdown_connections 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.083+0000 7f84eba51740 1 -- 172.21.15.103:0/801759294 wait complete. 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 -- start start 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 Processor -- start 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1a680 0x55f56cb247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb24d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb20050 0x55f56cb2bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f56cb23020 con 0x55f56cb1a680 2024-08-19T01:40:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f56ca39520 con 0x55f56cb1afc0 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f56cb231a0 con 0x55f56cb20050 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84eba51740 10 client.0 _ll_register_callbacks cb 0x55f56c790a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1a680 0x55f56cb247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1a680 0x55f56cb247d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35002/0 (socket says 172.21.15.103:35002) 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 -- 172.21.15.103:0/331832893 learned_addr learned my addr 172.21.15.103:0/331832893 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb20050 0x55f56cb2bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb24d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 -- 172.21.15.103:0/331832893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb20050 msgr2=0x55f56cb2bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb20050 0x55f56cb2bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 -- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 msgr2=0x55f56cb24d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb24d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 -- 172.21.15.103:0/331832893 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56c85d620 con 0x55f56cb1a680 2024-08-19T01:40:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84e259c640 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1a680 0x55f56cb247d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f84d4000c00 tx=0x7f84d4018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1b492e4258bf08 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.084+0000 7f84c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84c9ffb640 1 -- 172.21.15.103:0/331832893 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4026020 con 0x55f56cb1a680 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84c9ffb640 1 -- 172.21.15.103:0/331832893 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f84d4010a80 con 0x55f56cb1a680 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d4018af0 con 0x55f56cb1a680 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84c9ffb640 1 -- 172.21.15.103:0/331832893 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84d401d070 con 0x55f56cb1a680 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 10 client.16533 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f56cb2eab0 con 0x55f56cb1a680 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84c9ffb640 1 -- 172.21.15.103:0/331832893 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84d4018af0 con 0x55f56cb1a680 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84c9ffb640 1 -- 172.21.15.103:0/331832893 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f84d4004750 con 0x55f56cb1a680 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 10 client.16533 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 20 client.16533 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb2ff60 0x55f56cb323c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 4 client.16533 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.085+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f56ca8b100 con 0x55f56cb2ff60 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.086+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb2ff60 0x55f56cb323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.086+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb2ff60 0x55f56cb323c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e479bd46572303b server_cookie=c698a93486ba22e0 in_seq=0 out_seq=0 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.086+0000 7f84c9ffb640 10 client.16533 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84c9ffb640 1 -- 172.21.15.103:0/331832893 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f56ca8b100 con 0x55f56cb2ff60 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84c9ffb640 10 client.16533 handle_command_reply: tid=0 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 client.16533 shutdown 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1a680 msgr2=0x55f56cb247d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1a680 0x55f56cb247d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f84d4000c00 tx=0x7f84d4018670 comp rx=0 tx=0).stop 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 shutdown_connections 2024-08-19T01:40:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb2ff60 0x55f56cb323c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb20050 0x55f56cb2bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb24d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 --2- 172.21.15.103:0/331832893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb1a680 0x55f56cb247d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 >> 172.21.15.103:0/331832893 conn(0x55f56cb15cf0 msgr2=0x55f56cb17500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84c9ffb640 0 client.16533 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 shutdown_connections 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 1 -- 172.21.15.103:0/331832893 wait complete. 2024-08-19T01:40:04.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.087+0000 7f84eba51740 20 client.16533 trim_cache size 0 max 16384 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.101+0000 7f84eba51740 1 Processor -- start 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.101+0000 7f84eba51740 1 -- start start 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84eba51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb24de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84eba51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb2bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84eba51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb14d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84eba51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84e41aad80 con 0x55f56cb20050 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84eba51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f84e41ac9e0 con 0x55f56cb1afc0 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84eba51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84e41ab450 con 0x55f56cb1a680 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb14d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb14d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35010/0 (socket says 172.21.15.103:35010) 2024-08-19T01:40:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb2bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 -- 172.21.15.103:0/2592669467 learned_addr learned my addr 172.21.15.103:0/2592669467 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e259c640 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb24de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 -- 172.21.15.103:0/2592669467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 msgr2=0x55f56cb24de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb24de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 -- 172.21.15.103:0/2592669467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 msgr2=0x55f56cb2bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb2bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 -- 172.21.15.103:0/2592669467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56cb2ce60 con 0x55f56cb20050 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb2bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e259c640 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb24de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb14d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0098b0 tx=0x7f84dc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c442af2edb11eaa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.102+0000 7f84cbfff640 1 -- 172.21.15.103:0/2592669467 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004840 con 0x55f56cb20050 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84cbfff640 1 -- 172.21.15.103:0/2592669467 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84dc036040 con 0x55f56cb20050 2024-08-19T01:40:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84cbfff640 1 -- 172.21.15.103:0/2592669467 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0bafc0 con 0x55f56cb20050 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 -- 172.21.15.103:0/2592669467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 msgr2=0x55f56cb14d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb14d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0098b0 tx=0x7f84dc009360 comp rx=0 tx=0).stop 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 -- 172.21.15.103:0/2592669467 shutdown_connections 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb14d90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1afc0 0x55f56cb2bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 --2- 172.21.15.103:0/2592669467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1a680 0x55f56cb24de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 -- 172.21.15.103:0/2592669467 >> 172.21.15.103:0/2592669467 conn(0x55f56cb15cf0 msgr2=0x55f56cb14680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 -- 172.21.15.103:0/2592669467 shutdown_connections 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.103+0000 7f84eba51740 1 -- 172.21.15.103:0/2592669467 wait complete. 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 -- start start 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 Processor -- start 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1a680 0x55f56cb28940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1afc0 0x55f56cb28e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb98c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84e41ac250 con 0x55f56cb20050 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f84e41abb20 con 0x55f56cb1a680 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84e41a8d60 con 0x55f56cb1afc0 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84eba51740 10 client.0 _ll_register_callbacks cb 0x7f84d401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb98c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84e2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb98c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35022/0 (socket says 172.21.15.103:35022) 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84e2d9d640 1 -- 172.21.15.103:0/3010142535 learned_addr learned my addr 172.21.15.103:0/3010142535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84e1d9b640 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1afc0 0x55f56cb28e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.104+0000 7f84e259c640 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1a680 0x55f56cb28940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84e2d9d640 1 -- 172.21.15.103:0/3010142535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1afc0 msgr2=0x55f56cb28e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1afc0 0x55f56cb28e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84e2d9d640 1 -- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1a680 msgr2=0x55f56cb28940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1a680 0x55f56cb28940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84e2d9d640 1 -- 172.21.15.103:0/3010142535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56cb2bf00 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84e2d9d640 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb98c50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0bcc50 tx=0x7f84dc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dde94d6288240cd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84c9ffb640 1 -- 172.21.15.103:0/3010142535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0bccc0 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84c9ffb640 1 -- 172.21.15.103:0/3010142535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f84dc0b9330 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84dc036040 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84c9ffb640 1 -- 172.21.15.103:0/3010142535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84dc0bd3c0 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84eba51740 10 client.16539 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f56cb9a1a0 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84c9ffb640 1 -- 172.21.15.103:0/3010142535 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84dc036040 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84c9ffb640 1 -- 172.21.15.103:0/3010142535 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f84dc0c54a0 con 0x55f56cb20050 2024-08-19T01:40:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84eba51740 10 client.16539 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84eba51740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.105+0000 7f84eba51740 20 client.16539 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.106+0000 7f84eba51740 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb9ac30 0x55f56cb9d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.106+0000 7f84eba51740 4 client.16539 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.106+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f56ca85450 con 0x55f56cb9ac30 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.106+0000 7f84e259c640 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb9ac30 0x55f56cb9d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.106+0000 7f84e259c640 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb9ac30 0x55f56cb9d050 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47951fd58066c43d server_cookie=e4dc6c26f314c0c in_seq=0 out_seq=0 2024-08-19T01:40:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.106+0000 7f84c9ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84c9ffb640 1 -- 172.21.15.103:0/3010142535 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f56ca85450 con 0x55f56cb9ac30 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84c9ffb640 10 client.16539 handle_command_reply: tid=0 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 client.16539 shutdown 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 msgr2=0x55f56cb98c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb98c50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0bcc50 tx=0x7f84dc02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 shutdown_connections 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55f56cb9ac30 0x55f56cb9d050 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f56cb20050 0x55f56cb98c50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f56cb1afc0 0x55f56cb28e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 --2- 172.21.15.103:0/3010142535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f56cb1a680 0x55f56cb28940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 >> 172.21.15.103:0/3010142535 conn(0x55f56cb15cf0 msgr2=0x55f56cb14ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84c9ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 shutdown_connections 2024-08-19T01:40:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.112+0000 7f84eba51740 1 -- 172.21.15.103:0/3010142535 wait complete. 2024-08-19T01:40:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84eba51740 20 client.16539 trim_cache size 0 max 16384 2024-08-19T01:40:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:40:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:40:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:40:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f84cc0a2c30 msgr2=0x7f84cc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f84cc0a2c30 0x7f84cc0a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f84e4101a30 tx=0x7f84d8013040 comp rx=0 tx=0).stop 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 msgr2=0x7f84e4100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e4100840 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84dc031040 tx=0x7f84dc003ba0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 shutdown_connections 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f84cc0a2c30 0x7f84cc0a50f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84e41075b0 0x7f84e4100840 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84e4104c80 0x7f84e4100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 --2- 172.21.15.103:0/1002879246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e4102350 0x7f84e4101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 >> 172.21.15.103:0/1002879246 conn(0x7f84e40fbf10 msgr2=0x7f84e4102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 shutdown_connections 2024-08-19T01:40:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:04.113+0000 7f84e8bb9640 1 -- 172.21.15.103:0/1002879246 wait complete. 2024-08-19T01:40:04.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:03 smithi146 ceph-mon[22098]: pgmap v467: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:04.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:03 smithi103 ceph-mon[18436]: pgmap v467: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:04.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:03 smithi190 ceph-mon[22032]: pgmap v467: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:06.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:05 smithi146 ceph-mon[22098]: pgmap v468: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 43 op/s 2024-08-19T01:40:06.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:05 smithi103 ceph-mon[18436]: pgmap v468: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 43 op/s 2024-08-19T01:40:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:05 smithi190 ceph-mon[22032]: pgmap v468: 97 pgs: 97 active+clean; 5.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 43 op/s 2024-08-19T01:40:08.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:07 smithi146 ceph-mon[22098]: pgmap v469: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 44 op/s 2024-08-19T01:40:08.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:07 smithi103 ceph-mon[18436]: pgmap v469: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 44 op/s 2024-08-19T01:40:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:07 smithi190 ceph-mon[22032]: pgmap v469: 97 pgs: 97 active+clean; 5.7 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 MiB/s wr, 44 op/s 2024-08-19T01:40:10.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:09 smithi146 ceph-mon[22098]: pgmap v470: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:10.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:09 smithi103 ceph-mon[18436]: pgmap v470: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:10.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:09 smithi190 ceph-mon[22032]: pgmap v470: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:12.182 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:11 smithi190 ceph-mon[22032]: pgmap v471: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 46 op/s 2024-08-19T01:40:12.183 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:11 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:12.183 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:11 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:12.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:11 smithi146 ceph-mon[22098]: pgmap v471: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 46 op/s 2024-08-19T01:40:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:11 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:11 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:12.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:11 smithi103 ceph-mon[18436]: pgmap v471: 97 pgs: 97 active+clean; 5.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 46 op/s 2024-08-19T01:40:12.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:11 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:12.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:11 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:13.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:12 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:13.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:12 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:13.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:12 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:13.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:12 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:12 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:12 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:14.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:13 smithi146 ceph-mon[22098]: pgmap v472: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 39 op/s 2024-08-19T01:40:14.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:13 smithi103 ceph-mon[18436]: pgmap v472: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 39 op/s 2024-08-19T01:40:14.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:13 smithi190 ceph-mon[22032]: pgmap v472: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 39 op/s 2024-08-19T01:40:15.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:14 smithi103 ceph-mon[18436]: pgmap v473: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:40:15.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:14 smithi190 ceph-mon[22032]: pgmap v473: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:40:15.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:14 smithi146 ceph-mon[22098]: pgmap v473: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/3625264780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8101960 msgr2=0x7fe8b8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/3625264780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8101960 0x7fe8b8101de0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009900 tx=0x7fe8ac02a410 comp rx=0 tx=0).stop 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/3625264780 shutdown_connections 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/3625264780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8102320 0x7fe8b8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/3625264780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8101960 0x7fe8b8101de0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/3625264780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b8100770 0x7fe8b8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/3625264780 >> 172.21.15.103:0/3625264780 conn(0x7fe8b80fbf10 msgr2=0x7fe8b80fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/3625264780 shutdown_connections 2024-08-19T01:40:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.212+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/3625264780 wait complete. 2024-08-19T01:40:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.213+0000 7fe8bdfb9640 1 Processor -- start 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.213+0000 7fe8bdfb9640 1 -- start start 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.213+0000 7fe8bdfb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8100770 0x7fe8b8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.213+0000 7fe8bdfb9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8101960 0x7fe8b8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.213+0000 7fe8bdfb9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b8102320 0x7fe8b819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8bdfb9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810c280 con 0x7fe8b8100770 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8bdfb9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810c100 con 0x7fe8b8101960 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8bdfb9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810c400 con 0x7fe8b8102320 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8100770 0x7fe8b8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8100770 0x7fe8b8197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53696/0 (socket says 172.21.15.103:53696) 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1241574182 learned_addr learned my addr 172.21.15.103:0/1241574182 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b8102320 0x7fe8b819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b6ffd640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8101960 0x7fe8b8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1241574182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b8102320 msgr2=0x7fe8b819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b8102320 0x7fe8b819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1241574182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8101960 msgr2=0x7fe8b8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8101960 0x7fe8b8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1241574182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b819f6c0 con 0x7fe8b8100770 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b8102320 0x7fe8b819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8100770 0x7fe8b8197aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800d770 tx=0x7fe8a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b4ff9640 1 -- 172.21.15.103:0/1241574182 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8017070 con 0x7fe8b8100770 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b4ff9640 1 -- 172.21.15.103:0/1241574182 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a8009c30 con 0x7fe8b8100770 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b819f9b0 con 0x7fe8b8100770 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8b4ff9640 1 -- 172.21.15.103:0/1241574182 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8003d00 con 0x7fe8b8100770 2024-08-19T01:40:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.214+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b819ff00 con 0x7fe8b8100770 2024-08-19T01:40:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.217+0000 7fe8b4ff9640 1 -- 172.21.15.103:0/1241574182 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe8a8003ea0 con 0x7fe8b8100770 2024-08-19T01:40:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.217+0000 7fe8c0e53740 1 Processor -- start 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.217+0000 7fe8c0e53740 1 -- start start 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.217+0000 7fe8b4ff9640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe8a00a2c30 0x7fe8a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaa2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa2fc0 0x558ecfaa7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa8050 0x558ecfaaa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ecfa059d0 con 0x558ecfaa2fc0 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558ecf9ad820 con 0x558ecfaa2680 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ecfa01990 con 0x558ecfaa8050 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe8a00a2c30 0x7fe8a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b4ff9640 1 -- 172.21.15.103:0/1241574182 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe8a80cee30 con 0x7fe8b8100770 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa2fc0 0x558ecfaa7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaa2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa2fc0 0x558ecfaa7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53712/0 (socket says 172.21.15.103:53712) 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b6ffd640 1 -- 172.21.15.103:0/2754901706 learned_addr learned my addr 172.21.15.103:0/2754901706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa8050 0x558ecfaaa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b77fe640 1 -- 172.21.15.103:0/2754901706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa8050 msgr2=0x558ecfaaa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa8050 0x558ecfaaa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b77fe640 1 -- 172.21.15.103:0/2754901706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa2fc0 msgr2=0x558ecfaa7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa2fc0 0x558ecfaa7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b77fe640 1 -- 172.21.15.103:0/2754901706 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ecf719210 con 0x558ecfaa2680 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe8a00a2c30 0x7fe8a00a50f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00126c0 tx=0x7fe8b0012ac0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.218+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaa2a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cccd0 tx=0x7fe8a807f3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71d467da98c4791 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe89dffb640 1 -- 172.21.15.103:0/2754901706 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8020070 con 0x558ecfaa2680 2024-08-19T01:40:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe89dffb640 1 -- 172.21.15.103:0/2754901706 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a8009de0 con 0x558ecfaa2680 2024-08-19T01:40:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe89dffb640 1 -- 172.21.15.103:0/2754901706 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80999c0 con 0x558ecfaa2680 2024-08-19T01:40:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2754901706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 msgr2=0x558ecfaa2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaa2a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cccd0 tx=0x7fe8a807f3f0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2754901706 shutdown_connections 2024-08-19T01:40:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa8050 0x558ecfaaa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa2fc0 0x558ecfaa7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2754901706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaa2a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2754901706 >> 172.21.15.103:0/2754901706 conn(0x558ecfa9dcf0 msgr2=0x558ecfa9e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2754901706 shutdown_connections 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.219+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2754901706 wait complete. 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 -- start start 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 Processor -- start 2024-08-19T01:40:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfaacd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa8050 0x558ecfab3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ecfaaaf60 con 0x558ecfaa8050 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558ecf9c1520 con 0x558ecfaa2680 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558ecfaab0e0 con 0x558ecfaa2fc0 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8c0e53740 10 client.0 _ll_register_callbacks cb 0x558ecf718a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa8050 0x558ecfab3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa8050 0x558ecfab3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53720/0 (socket says 172.21.15.103:53720) 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.220+0000 7fe8b7fff640 1 -- 172.21.15.103:0/3578923363 learned_addr learned my addr 172.21.15.103:0/3578923363 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b6ffd640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfaacd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b7fff640 1 -- 172.21.15.103:0/3578923363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 msgr2=0x558ecfaacd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfaacd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b7fff640 1 -- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 msgr2=0x558ecfaac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b7fff640 1 -- 172.21.15.103:0/3578923363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ecf7e5620 con 0x558ecfaa8050 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b6ffd640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfaacd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa8050 0x558ecfab3ce0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00109e0 tx=0x7fe8b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed4ed43fef9222e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe896ffd640 1 -- 172.21.15.103:0/3578923363 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0024030 con 0x558ecfaa8050 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe896ffd640 1 -- 172.21.15.103:0/3578923363 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe8b0011b70 con 0x558ecfaa8050 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a8009de0 con 0x558ecfaa8050 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe896ffd640 1 -- 172.21.15.103:0/3578923363 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8b0011d70 con 0x558ecfaa8050 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8c0e53740 10 client.16551 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.221+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ecfab6b40 con 0x558ecfaa8050 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe896ffd640 1 -- 172.21.15.103:0/3578923363 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a8009de0 con 0x558ecfaa8050 2024-08-19T01:40:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe896ffd640 1 -- 172.21.15.103:0/3578923363 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe8b0005760 con 0x558ecfaa8050 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8c0e53740 10 client.16551 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8c0e53740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8c0e53740 20 client.16551 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfab82f0 0x558ecfaba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8c0e53740 4 client.16551 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ecfa13100 con 0x558ecfab82f0 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfab82f0 0x558ecfaba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe896ffd640 10 client.16551 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.222+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfab82f0 0x558ecfaba770 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad606c3ff546c988 server_cookie=e1ac5ad96e530758 in_seq=0 out_seq=0 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe896ffd640 1 -- 172.21.15.103:0/3578923363 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ecfa13100 con 0x558ecfab82f0 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe896ffd640 10 client.16551 handle_command_reply: tid=0 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 client.16551 shutdown 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa8050 msgr2=0x558ecfab3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa8050 0x558ecfab3ce0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00109e0 tx=0x7fe8b0004ab0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 shutdown_connections 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfab82f0 0x558ecfaba770 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfaa8050 0x558ecfab3ce0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfaacd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe896ffd640 0 client.16551 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/3578923363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfaac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 >> 172.21.15.103:0/3578923363 conn(0x558ecfa9dcf0 msgr2=0x558ecfa9f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 shutdown_connections 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.223+0000 7fe8c0e53740 1 -- 172.21.15.103:0/3578923363 wait complete. 2024-08-19T01:40:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.224+0000 7fe8c0e53740 20 client.16551 trim_cache size 0 max 16384 2024-08-19T01:40:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.245+0000 7fe8c0e53740 1 Processor -- start 2024-08-19T01:40:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.245+0000 7fe8c0e53740 1 -- start start 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfaabdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfa9eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a8c50 con 0x558ecfa9dcf0 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81aa8b0 con 0x558ecfaa2680 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a9320 con 0x558ecfaa2fc0 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfaabdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfaabdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53730/0 (socket says 172.21.15.103:53730) 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1120362845 learned_addr learned my addr 172.21.15.103:0/1120362845 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b6ffd640 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1120362845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 msgr2=0x558ecfa9eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfa9eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1120362845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 msgr2=0x558ecfab3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 -- 172.21.15.103:0/1120362845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ecfaaa9c0 con 0x558ecfa9dcf0 2024-08-19T01:40:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfaabdd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80037f0 tx=0x7fe8a8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af14dda5c976c4c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.246+0000 7fe89d7fa640 1 -- 172.21.15.103:0/1120362845 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8020040 con 0x558ecfa9dcf0 2024-08-19T01:40:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe89d7fa640 1 -- 172.21.15.103:0/1120362845 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a80997d0 con 0x558ecfa9dcf0 2024-08-19T01:40:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe89d7fa640 1 -- 172.21.15.103:0/1120362845 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80a3ea0 con 0x558ecfa9dcf0 2024-08-19T01:40:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 -- 172.21.15.103:0/1120362845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 msgr2=0x558ecfaabdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfaabdd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80037f0 tx=0x7fe8a8086040 comp rx=0 tx=0).stop 2024-08-19T01:40:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 -- 172.21.15.103:0/1120362845 shutdown_connections 2024-08-19T01:40:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfa9eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/1120362845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfaabdd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 -- 172.21.15.103:0/1120362845 >> 172.21.15.103:0/1120362845 conn(0x558ecfaa7be0 msgr2=0x558ecfaac7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 -- 172.21.15.103:0/1120362845 shutdown_connections 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.247+0000 7fe8c0e53740 1 -- 172.21.15.103:0/1120362845 wait complete. 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 -- start start 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 Processor -- start 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfab12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfab2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81aa120 con 0x558ecfa9dcf0 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a99f0 con 0x558ecfaa2680 2024-08-19T01:40:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a6c30 con 0x558ecfaa2fc0 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8c0e53740 10 client.0 _ll_register_callbacks cb 0x7fe8b0012340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfab12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfab12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53736/0 (socket says 172.21.15.103:53736) 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 -- 172.21.15.103:0/2062354027 learned_addr learned my addr 172.21.15.103:0/2062354027 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b7fff640 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfab2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b6ffd640 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 -- 172.21.15.103:0/2062354027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 msgr2=0x558ecfab2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfab2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 -- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 msgr2=0x558ecfab1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.248+0000 7fe8b77fe640 1 -- 172.21.15.103:0/2062354027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ecfab2e40 con 0x558ecfa9dcf0 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8b77fe640 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfab12e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8012040 tx=0x7fe8a8097320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d61e18754b449ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe896ffd640 1 -- 172.21.15.103:0/2062354027 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8020030 con 0x558ecfa9dcf0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe896ffd640 1 -- 172.21.15.103:0/2062354027 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe8a80860b0 con 0x558ecfa9dcf0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a80997d0 con 0x558ecfa9dcf0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe896ffd640 1 -- 172.21.15.103:0/2062354027 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8a80afbe0 con 0x558ecfa9dcf0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 10 client.16557 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ecfb23010 con 0x558ecfa9dcf0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe896ffd640 1 -- 172.21.15.103:0/2062354027 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a80997d0 con 0x558ecfa9dcf0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe896ffd640 1 -- 172.21.15.103:0/2062354027 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe8a8004190 con 0x558ecfa9dcf0 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 10 client.16557 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 20 client.16557 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfaa8050 0x558ecfb25520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 4 client.16557 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.249+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ecfa0d450 con 0x558ecfaa8050 2024-08-19T01:40:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.250+0000 7fe8b6ffd640 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfaa8050 0x558ecfb25520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.250+0000 7fe8b6ffd640 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfaa8050 0x558ecfb25520 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=715633775f42c38b server_cookie=c54cea42641ed7f2 in_seq=0 out_seq=0 2024-08-19T01:40:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.250+0000 7fe896ffd640 10 client.16557 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe896ffd640 1 -- 172.21.15.103:0/2062354027 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ecfa0d450 con 0x558ecfaa8050 2024-08-19T01:40:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe896ffd640 10 client.16557 handle_command_reply: tid=0 2024-08-19T01:40:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 client.16557 shutdown 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 msgr2=0x558ecfab12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfab12e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8012040 tx=0x7fe8a8097320 comp rx=0 tx=0).stop 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 shutdown_connections 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558ecfaa8050 0x558ecfb25520 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558ecfaa2fc0 0x558ecfab2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558ecfaa2680 0x558ecfab1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe896ffd640 0 client.16557 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 --2- 172.21.15.103:0/2062354027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ecfa9dcf0 0x558ecfab12e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 >> 172.21.15.103:0/2062354027 conn(0x558ecfaa7be0 msgr2=0x558ecfa9c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 shutdown_connections 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 1 -- 172.21.15.103:0/2062354027 wait complete. 2024-08-19T01:40:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.257+0000 7fe8c0e53740 20 client.16557 trim_cache size 0 max 16384 2024-08-19T01:40:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:40:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:40:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:40:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe8a00a2c30 msgr2=0x7fe8a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe8a00a2c30 0x7fe8a00a50f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00126c0 tx=0x7fe8b0012ac0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8100770 msgr2=0x7fe8b8197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8100770 0x7fe8b8197aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800d770 tx=0x7fe8a800dc30 comp rx=0 tx=0).stop 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 shutdown_connections 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe8a00a2c30 0x7fe8a00a50f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8b8102320 0x7fe8b819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8b8101960 0x7fe8b8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 --2- 172.21.15.103:0/1241574182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b8100770 0x7fe8b8197aa0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 >> 172.21.15.103:0/1241574182 conn(0x7fe8b80fbf10 msgr2=0x7fe8b80695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 shutdown_connections 2024-08-19T01:40:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:17.258+0000 7fe8bdfb9640 1 -- 172.21.15.103:0/1241574182 wait complete. 2024-08-19T01:40:18.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:17 smithi146 ceph-mon[22098]: pgmap v474: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-08-19T01:40:18.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:17 smithi103 ceph-mon[18436]: pgmap v474: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-08-19T01:40:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:17 smithi190 ceph-mon[22032]: pgmap v474: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 MiB/s wr, 23 op/s 2024-08-19T01:40:20.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:19 smithi146 ceph-mon[22098]: pgmap v475: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:40:20.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:19 smithi103 ceph-mon[18436]: pgmap v475: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:40:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:19 smithi190 ceph-mon[22032]: pgmap v475: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:40:22.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:21 smithi146 ceph-mon[22098]: pgmap v476: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:40:22.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:21 smithi103 ceph-mon[18436]: pgmap v476: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:40:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:21 smithi190 ceph-mon[22032]: pgmap v476: 97 pgs: 97 active+clean; 6.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:40:24.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:23 smithi146 ceph-mon[22098]: pgmap v477: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:40:24.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:23 smithi103 ceph-mon[18436]: pgmap v477: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:40:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:23 smithi190 ceph-mon[22032]: pgmap v477: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:40:26.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:25 smithi146 ceph-mon[22098]: pgmap v478: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:40:26.307 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:25 smithi103 ceph-mon[18436]: pgmap v478: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:40:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:25 smithi190 ceph-mon[22032]: pgmap v478: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:40:28.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:27 smithi146 ceph-mon[22098]: pgmap v479: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:40:28.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:27 smithi103 ceph-mon[18436]: pgmap v479: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:40:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:27 smithi190 ceph-mon[22032]: pgmap v479: 97 pgs: 97 active+clean; 6.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:40:30.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:29 smithi146 ceph-mon[22098]: pgmap v480: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:40:30.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:29 smithi103 ceph-mon[18436]: pgmap v480: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.354+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/3084496487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc100770 msgr2=0x7f1fcc100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.354+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/3084496487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc100770 0x7f1fcc100b70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009900 tx=0x7f1fbc02a410 comp rx=0 tx=0).stop 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.355+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/3084496487 shutdown_connections 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.355+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/3084496487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fcc102320 0x7f1fcc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.355+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/3084496487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc101960 0x7f1fcc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.355+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/3084496487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc100770 0x7f1fcc100b70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.355+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/3084496487 >> 172.21.15.103:0/3084496487 conn(0x7f1fcc0fbf10 msgr2=0x7f1fcc0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.355+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/3084496487 shutdown_connections 2024-08-19T01:40:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.355+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/3084496487 wait complete. 2024-08-19T01:40:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 Processor -- start 2024-08-19T01:40:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 -- start start 2024-08-19T01:40:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc100770 0x7f1fcc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fcc101960 0x7f1fcc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc102320 0x7f1fcc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10c2d0 con 0x7f1fcc102320 2024-08-19T01:40:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10c150 con 0x7f1fcc100770 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fd0d4b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10c450 con 0x7f1fcc101960 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc102320 0x7f1fcc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc102320 0x7f1fcc19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55654/0 (socket says 172.21.15.103:55654) 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 -- 172.21.15.103:0/1197060163 learned_addr learned my addr 172.21.15.103:0/1197060163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fcc101960 0x7f1fcc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fca59c640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc100770 0x7f1fcc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 -- 172.21.15.103:0/1197060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fcc101960 msgr2=0x7f1fcc197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fcc101960 0x7f1fcc197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 -- 172.21.15.103:0/1197060163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc100770 msgr2=0x7f1fcc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc100770 0x7f1fcc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.356+0000 7f1fcad9d640 1 -- 172.21.15.103:0/1197060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc19f660 con 0x7f1fcc102320 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.357+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc102320 0x7f1fcc19eff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009fd0 tx=0x7f1fc400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.357+0000 7f1fbb7fe640 1 -- 172.21.15.103:0/1197060163 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4017070 con 0x7f1fcc102320 2024-08-19T01:40:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.357+0000 7f1fbb7fe640 1 -- 172.21.15.103:0/1197060163 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fc4003e20 con 0x7f1fcc102320 2024-08-19T01:40:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.357+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc19f950 con 0x7f1fcc102320 2024-08-19T01:40:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.357+0000 7f1fbb7fe640 1 -- 172.21.15.103:0/1197060163 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4010460 con 0x7f1fcc102320 2024-08-19T01:40:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.357+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc19fe50 con 0x7f1fcc102320 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.359+0000 7f1fbb7fe640 1 -- 172.21.15.103:0/1197060163 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1fc4003970 con 0x7f1fcc102320 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 Processor -- start 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 -- start start 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fbb7fe640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1fb40a2c30 0x7f1fb40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb1be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55efcb1199d0 con 0x55efcb1bc050 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55efcb0c1820 con 0x55efcb1b6680 2024-08-19T01:40:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55efcb115990 con 0x55efcb1b6fc0 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb1be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1fb40a2c30 0x7f1fb40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb1be510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55658/0 (socket says 172.21.15.103:55658) 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fcad9d640 1 -- 172.21.15.103:0/2701128655 learned_addr learned my addr 172.21.15.103:0/2701128655 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fca59c640 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.360+0000 7f1fbb7fe640 1 -- 172.21.15.103:0/1197060163 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1fc40cedb0 con 0x7f1fcc102320 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fca59c640 1 -- 172.21.15.103:0/2701128655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 msgr2=0x55efcb1bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fca59c640 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1bba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fca59c640 1 -- 172.21.15.103:0/2701128655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 msgr2=0x55efcb1be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fca59c640 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb1be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fca59c640 1 -- 172.21.15.103:0/2701128655 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efcae2d210 con 0x55efcb1b6680 2024-08-19T01:40:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb1be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1fb40a2c30 0x7f1fb40a50f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc198f50 tx=0x7f1fc0013880 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fca59c640 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1b6a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009230 tx=0x7f1fbc005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d987f16f7295b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fb8ff9640 1 -- 172.21.15.103:0/2701128655 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc035030 con 0x55efcb1b6680 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fb8ff9640 1 -- 172.21.15.103:0/2701128655 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fbc040070 con 0x55efcb1b6680 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.361+0000 7f1fb8ff9640 1 -- 172.21.15.103:0/2701128655 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc031870 con 0x55efcb1b6680 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 -- 172.21.15.103:0/2701128655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 msgr2=0x55efcb1b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1b6a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009230 tx=0x7f1fbc005c00 comp rx=0 tx=0).stop 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 -- 172.21.15.103:0/2701128655 shutdown_connections 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb1be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/2701128655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1b6a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 -- 172.21.15.103:0/2701128655 >> 172.21.15.103:0/2701128655 conn(0x55efcb1b1cf0 msgr2=0x55efcb1b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 -- 172.21.15.103:0/2701128655 shutdown_connections 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 -- 172.21.15.103:0/2701128655 wait complete. 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 -- start start 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.362+0000 7f1fd3be1740 1 Processor -- start 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6680 0x55efcb1c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1bc050 0x55efcb1c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55efcb1befc0 con 0x55efcb1b6680 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55efcb0d5520 con 0x55efcb1bc050 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55efcb1bf140 con 0x55efcb1b6fc0 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 10 client.0 _ll_register_callbacks cb 0x55efcae2ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6680 0x55efcb1c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6680 0x55efcb1c0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55668/0 (socket says 172.21.15.103:55668) 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 -- 172.21.15.103:0/1254662775 learned_addr learned my addr 172.21.15.103:0/1254662775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1bc050 0x55efcb1c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 -- 172.21.15.103:0/1254662775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 msgr2=0x55efcb1c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 -- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1bc050 msgr2=0x55efcb1c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1bc050 0x55efcb1c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 -- 172.21.15.103:0/1254662775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efcaef9620 con 0x55efcb1b6680 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fca59c640 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6680 0x55efcb1c0780 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc005ec0 tx=0x7f1fbc0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6817e9df53986a7a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/1254662775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc052060 con 0x55efcb1b6680 2024-08-19T01:40:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fbc040070 con 0x55efcb1b6680 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.363+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/1254662775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1fbc003a00 con 0x55efcb1b6680 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/1254662775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1fbc035050 con 0x55efcb1b6680 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 10 client.16569 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efcb1caaf0 con 0x55efcb1b6680 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/1254662775 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fbc040070 con 0x55efcb1b6680 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/1254662775 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1fbc059530 con 0x55efcb1b6680 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 10 client.16569 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 20 client.16569 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb1ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 4 client.16569 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efcb127100 con 0x55efcb1cbe40 2024-08-19T01:40:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.364+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb1ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.365+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb1ce230 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf44b38571e75d1 server_cookie=1970184936d4fd7e in_seq=0 out_seq=0 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.365+0000 7f1fb1ffb640 10 client.16569 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.365+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/1254662775 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efcb127100 con 0x55efcb1cbe40 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.365+0000 7f1fb1ffb640 10 client.16569 handle_command_reply: tid=0 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.365+0000 7f1fd3be1740 1 client.16569 shutdown 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.365+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6680 msgr2=0x55efcb1c0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.365+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6680 0x55efcb1c0780 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc005ec0 tx=0x7f1fbc0045b0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 shutdown_connections 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb1ce230 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fb1ffb640 0 client.16569 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1bc050 0x55efcb1c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/1254662775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6680 0x55efcb1c0780 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 >> 172.21.15.103:0/1254662775 conn(0x55efcb1b1cf0 msgr2=0x55efcb1b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 shutdown_connections 2024-08-19T01:40:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 1 -- 172.21.15.103:0/1254662775 wait complete. 2024-08-19T01:40:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.366+0000 7f1fd3be1740 20 client.16569 trim_cache size 0 max 16384 2024-08-19T01:40:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:29 smithi190 ceph-mon[22032]: pgmap v480: 97 pgs: 97 active+clean; 6.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:40:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 Processor -- start 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 -- start start 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1bc050 0x55efcb1b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1a68a0 con 0x55efcb1b6fc0 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1a8500 con 0x55efcb1b6680 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1a6f70 con 0x55efcb1bc050 2024-08-19T01:40:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55676/0 (socket says 172.21.15.103:55676) 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 -- 172.21.15.103:0/977323199 learned_addr learned my addr 172.21.15.103:0/977323199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1bc050 0x55efcb1b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fca59c640 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 -- 172.21.15.103:0/977323199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1bc050 msgr2=0x55efcb1b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1bc050 0x55efcb1b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 -- 172.21.15.103:0/977323199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 msgr2=0x55efcb1ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fc9d9b640 1 -- 172.21.15.103:0/977323199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efcb1b74a0 con 0x55efcb1b6fc0 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fca59c640 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.384+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1bc050 0x55efcb1b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.385+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c7c20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00096c0 tx=0x7f1fc001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97ecd6761c71810 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.385+0000 7f1fb3fff640 1 -- 172.21.15.103:0/977323199 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00044c0 con 0x55efcb1b6fc0 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.385+0000 7f1fb3fff640 1 -- 172.21.15.103:0/977323199 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fc002d070 con 0x55efcb1b6fc0 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.385+0000 7f1fb3fff640 1 -- 172.21.15.103:0/977323199 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0027310 con 0x55efcb1b6fc0 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- 172.21.15.103:0/977323199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6fc0 msgr2=0x55efcb1c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c7c20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00096c0 tx=0x7f1fc001f000 comp rx=0 tx=0).stop 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- 172.21.15.103:0/977323199 shutdown_connections 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1bc050 0x55efcb1b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c7c20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/977323199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- 172.21.15.103:0/977323199 >> 172.21.15.103:0/977323199 conn(0x55efcb1b1cf0 msgr2=0x55efcb1bfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- 172.21.15.103:0/977323199 shutdown_connections 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- 172.21.15.103:0/977323199 wait complete. 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- start start 2024-08-19T01:40:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 Processor -- start 2024-08-19T01:40:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb234a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1a7d70 con 0x55efcb1bc050 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1a7640 con 0x55efcb1b6680 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1a4880 con 0x55efcb1b6fc0 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fd3be1740 10 client.0 _ll_register_callbacks cb 0x7f1fc400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.386+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb234a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb234a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55688/0 (socket says 172.21.15.103:55688) 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 -- 172.21.15.103:0/339012373 learned_addr learned my addr 172.21.15.103:0/339012373 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fca59c640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 -- 172.21.15.103:0/339012373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 msgr2=0x55efcb1c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 -- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 msgr2=0x55efcb1c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 -- 172.21.15.103:0/339012373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efcb234f80 con 0x55efcb1bc050 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fc9d9b640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fca59c640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fcad9d640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb234a40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4004990 tx=0x7f1fc409b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e6b6bde6361d17 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/339012373 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4020040 con 0x55efcb1bc050 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/339012373 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1fc4004000 con 0x55efcb1bc050 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc002d070 con 0x55efcb1bc050 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/339012373 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1fc409f9f0 con 0x55efcb1bc050 2024-08-19T01:40:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fd3be1740 10 client.16575 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efcb236220 con 0x55efcb1bc050 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.387+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/339012373 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fc002d070 con 0x55efcb1bc050 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/339012373 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1fc4080810 con 0x55efcb1bc050 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fd3be1740 10 client.16575 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fd3be1740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fd3be1740 20 client.16575 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb238c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fd3be1740 4 client.16575 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efcb121450 con 0x55efcb1cbe40 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fca59c640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb238c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fb1ffb640 10 client.16575 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.388+0000 7f1fca59c640 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb238c50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c42904d3b16c423 server_cookie=1074e18d9134edef in_seq=0 out_seq=0 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fb1ffb640 1 -- 172.21.15.103:0/339012373 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efcb121450 con 0x55efcb1cbe40 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fb1ffb640 10 client.16575 handle_command_reply: tid=0 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 client.16575 shutdown 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 msgr2=0x55efcb234a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb234a40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4004990 tx=0x7f1fc409b180 comp rx=0 tx=0).stop 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 shutdown_connections 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55efcb1cbe40 0x55efcb238c50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efcb1bc050 0x55efcb234a40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55efcb1b6fc0 0x55efcb1c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 --2- 172.21.15.103:0/339012373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55efcb1b6680 0x55efcb1c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 >> 172.21.15.103:0/339012373 conn(0x55efcb1b1cf0 msgr2=0x55efcb1c05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fb1ffb640 0 client.16575 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 shutdown_connections 2024-08-19T01:40:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 1 -- 172.21.15.103:0/339012373 wait complete. 2024-08-19T01:40:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.395+0000 7f1fd3be1740 20 client.16575 trim_cache size 0 max 16384 2024-08-19T01:40:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:40:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:40:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:40:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:40:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1fb40a2c30 msgr2=0x7f1fb40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1fb40a2c30 0x7f1fb40a50f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc198f50 tx=0x7f1fc0013880 comp rx=0 tx=0).stop 2024-08-19T01:40:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc102320 msgr2=0x7f1fcc19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc102320 0x7f1fcc19eff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009fd0 tx=0x7f1fc400b260 comp rx=0 tx=0).stop 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 shutdown_connections 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1fb40a2c30 0x7f1fb40a50f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fcc102320 0x7f1fcc19eff0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1fcc101960 0x7f1fcc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 --2- 172.21.15.103:0/1197060163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fcc100770 0x7f1fcc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 >> 172.21.15.103:0/1197060163 conn(0x7f1fcc0fbf10 msgr2=0x7f1fcc0695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 shutdown_connections 2024-08-19T01:40:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:30.396+0000 7f1fd0d4b640 1 -- 172.21.15.103:0/1197060163 wait complete. 2024-08-19T01:40:31.130 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:40:31.421 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/1555283129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 msgr2=0x7fbb0410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/1555283129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb0410e790 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009cd0 tx=0x7fbafc02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/1555283129 shutdown_connections 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/1555283129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb0410e790 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/1555283129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb041018c0 0x7fbb04101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/1555283129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb041078c0 0x7fbb04107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/1555283129 >> 172.21.15.103:0/1555283129 conn(0x7fbb040fd620 msgr2=0x7fbb040ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/1555283129 shutdown_connections 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/1555283129 wait complete. 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.937+0000 7fbb0a5f9640 1 Processor -- start 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb0a5f9640 1 -- start start 2024-08-19T01:40:31.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb0a5f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb041018c0 0x7fbb04197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb0a5f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb04198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb0a5f9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb041078c0 0x7fbb0419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb0a5f9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04111b70 con 0x7fbb04102260 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb0a5f9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041119f0 con 0x7fbb041078c0 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb0a5f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04111cf0 con 0x7fbb041018c0 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb04198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb04198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39458/0 (socket says 172.21.15.103:39458) 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 -- 172.21.15.103:0/604159705 learned_addr learned my addr 172.21.15.103:0/604159705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb08b96640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb041078c0 0x7fbb0419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb03fff640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb041018c0 0x7fbb04197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 -- 172.21.15.103:0/604159705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb041018c0 msgr2=0x7fbb04197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb041018c0 0x7fbb04197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 -- 172.21.15.103:0/604159705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb041078c0 msgr2=0x7fbb0419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb041078c0 0x7fbb0419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.938+0000 7fbb037fe640 1 -- 172.21.15.103:0/604159705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0419cfa0 con 0x7fbb04102260 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.939+0000 7fbb037fe640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb04198530 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800c920 tx=0x7fbaf800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.939+0000 7fbb017fa640 1 -- 172.21.15.103:0/604159705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8007c70 con 0x7fbb04102260 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.939+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0419d290 con 0x7fbb04102260 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.939+0000 7fbb017fa640 1 -- 172.21.15.103:0/604159705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaf8007e10 con 0x7fbb04102260 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.939+0000 7fbb017fa640 1 -- 172.21.15.103:0/604159705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8018820 con 0x7fbb04102260 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.939+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb04074150 con 0x7fbb04102260 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.941+0000 7fbb017fa640 1 -- 172.21.15.103:0/604159705 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbaf80189c0 con 0x7fbb04102260 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 Processor -- start 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 -- start start 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb017fa640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbaec0a2b80 0x7fbaec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d690ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d69156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6917ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1d6867980 con 0x55d1d690aa20 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d1d67ffa70 con 0x55d1d6915c00 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d1d6863940 con 0x55d1d690b340 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.942+0000 7fbb08b96640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbaec0a2b80 0x7fbaec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d69156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb017fa640 1 -- 172.21.15.103:0/604159705 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbaf80cf090 con 0x7fbb04102260 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb08b96640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6917ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d69156c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:57578/0 (socket says 172.21.15.103:57578) 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 -- 172.21.15.103:0/1748606812 learned_addr learned my addr 172.21.15.103:0/1748606812 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb03fff640 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d690ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 -- 172.21.15.103:0/1748606812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 msgr2=0x55d1d6917ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6917ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 -- 172.21.15.103:0/1748606812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 msgr2=0x55d1d690ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d690ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 -- 172.21.15.103:0/1748606812 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d68626e0 con 0x55d1d690b340 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb08b96640 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6917ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb03fff640 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d690ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb08b96640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbaec0a2b80 0x7fbaec0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbafc005e10 tx=0x7fbafc0316e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.943+0000 7fbb037fe640 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d69156c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80982f0 tx=0x7fbaf809b340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983e086bc0405ca server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.946+0000 7fbaea7fc640 1 -- 172.21.15.103:0/1748606812 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf809b920 con 0x55d1d690b340 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.946+0000 7fbaea7fc640 1 -- 172.21.15.103:0/1748606812 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaf809fcd0 con 0x55d1d690b340 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.946+0000 7fbaea7fc640 1 -- 172.21.15.103:0/1748606812 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80c06a0 con 0x55d1d690b340 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1748606812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 msgr2=0x55d1d69156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d69156c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80982f0 tx=0x7fbaf809b340 comp rx=0 tx=0).stop 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1748606812 shutdown_connections 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6917ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d69156c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1748606812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d690ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1748606812 >> 172.21.15.103:0/1748606812 conn(0x55d1d6906fb0 msgr2=0x55d1d6907390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1748606812 shutdown_connections 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1748606812 wait complete. 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.947+0000 7fbb0c9f7740 1 -- start start 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 1 Processor -- start 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d6912b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d6913050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6913590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1d69187d0 con 0x55d1d690aa20 2024-08-19T01:40:31.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d1d6822aa0 con 0x55d1d6915c00 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d1d6918950 con 0x55d1d690b340 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb0c9f7740 10 client.0 _ll_register_callbacks cb 0x55d1d6811e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d6913050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d6913050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:57594/0 (socket says 172.21.15.103:57594) 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb037fe640 1 -- 172.21.15.103:0/1038965216 learned_addr learned my addr 172.21.15.103:0/1038965216 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb03fff640 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d6912b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.948+0000 7fbb08b96640 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6913590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb037fe640 1 -- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 msgr2=0x55d1d6913590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb037fe640 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6913590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb037fe640 1 -- 172.21.15.103:0/1038965216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 msgr2=0x55d1d6912b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb037fe640 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d6912b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb037fe640 1 -- 172.21.15.103:0/1038965216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d6806650 con 0x55d1d690b340 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb037fe640 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d6913050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80a4060 tx=0x7fbaf8099a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5be6cd15abd1e9d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:31.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbae37fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbae37fe640 1 -- 172.21.15.103:0/1038965216 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80a4590 con 0x55d1d690b340 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf809fcd0 con 0x55d1d690b340 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbae37fe640 1 -- 172.21.15.103:0/1038965216 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbaf80c0600 con 0x55d1d690b340 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbae37fe640 1 -- 172.21.15.103:0/1038965216 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbaf80c0800 con 0x55d1d690b340 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb0c9f7740 10 client.25945 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.949+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1d65d93c0 con 0x55d1d690b340 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbae37fe640 1 -- 172.21.15.103:0/1038965216 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaf809fcd0 con 0x55d1d690b340 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbae37fe640 1 -- 172.21.15.103:0/1038965216 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbaf80bbce0 con 0x55d1d690b340 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb0c9f7740 10 client.25945 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb0c9f7740 10 client.25945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb0c9f7740 20 client.25945 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6925580 0x55d1d6927970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb0c9f7740 4 client.25945 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d6874ef0 con 0x55d1d6925580 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb03fff640 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6925580 0x55d1d6927970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbb03fff640 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6925580 0x55d1d6927970 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1901356058aa2ab2 server_cookie=c38fdfd5a7244d4d in_seq=0 out_seq=0 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.950+0000 7fbae37fe640 10 client.25945 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbae37fe640 1 -- 172.21.15.103:0/1038965216 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1d6874ef0 con 0x55d1d6925580 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbae37fe640 10 client.25945 handle_command_reply: tid=0 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 client.25945 shutdown 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 msgr2=0x55d1d6913050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d6913050 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80a4060 tx=0x7fbaf8099a30 comp rx=0 tx=0).stop 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 shutdown_connections 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6925580 0x55d1d6927970 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d6913590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690b340 0x55d1d6913050 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbae37fe640 0 client.25945 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/1038965216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690aa20 0x55d1d6912b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 >> 172.21.15.103:0/1038965216 conn(0x55d1d6906fb0 msgr2=0x55d1d690cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 shutdown_connections 2024-08-19T01:40:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.951+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/1038965216 wait complete. 2024-08-19T01:40:31.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.952+0000 7fbb0c9f7740 20 client.25945 trim_cache size 0 max 16384 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 Processor -- start 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 -- start start 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690aa20 0x55d1d690d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d68fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d692cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041aaea0 con 0x55d1d690b340 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041acb00 con 0x55d1d6915c00 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.978+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041ab570 con 0x55d1d690aa20 2024-08-19T01:40:31.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d68fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d68fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39488/0 (socket says 172.21.15.103:39488) 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690aa20 0x55d1d690d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb037fe640 1 -- 172.21.15.103:0/216364430 learned_addr learned my addr 172.21.15.103:0/216364430 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb08b96640 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d692cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb08b96640 1 -- 172.21.15.103:0/216364430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690aa20 msgr2=0x55d1d690d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb08b96640 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690aa20 0x55d1d690d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb08b96640 1 -- 172.21.15.103:0/216364430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 msgr2=0x55d1d68fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb08b96640 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d68fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb08b96640 1 -- 172.21.15.103:0/216364430 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d6911060 con 0x55d1d6915c00 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb03fff640 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690aa20 0x55d1d690d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb037fe640 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d68fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:31.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbb08b96640 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d692cf70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009ca0 tx=0x7fbafc005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c2820bf0c8862d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:31.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.979+0000 7fbae9ffb640 1 -- 172.21.15.103:0/216364430 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc04f070 con 0x55d1d6915c00 2024-08-19T01:40:31.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbae9ffb640 1 -- 172.21.15.103:0/216364430 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbafc036040 con 0x55d1d6915c00 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbae9ffb640 1 -- 172.21.15.103:0/216364430 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc030d00 con 0x55d1d6915c00 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/216364430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 msgr2=0x55d1d692cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d692cf70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009ca0 tx=0x7fbafc005da0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/216364430 shutdown_connections 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d6915c00 0x55d1d692cf70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d68fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/216364430 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d690aa20 0x55d1d690d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/216364430 >> 172.21.15.103:0/216364430 conn(0x55d1d6906fb0 msgr2=0x55d1d68fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.980+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/216364430 shutdown_connections 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/216364430 wait complete. 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 -- start start 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 Processor -- start 2024-08-19T01:40:31.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d690aa20 0x55d1d691dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d69193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d6915c00 0x55d1d69198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041ac370 con 0x55d1d690b340 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041abc40 con 0x55d1d690aa20 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041a8e80 con 0x55d1d6915c00 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.981+0000 7fbb0c9f7740 10 client.0 _ll_register_callbacks cb 0x7fbaf80ce070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d69193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d69193b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39502/0 (socket says 172.21.15.103:39502) 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 -- 172.21.15.103:0/184875573 learned_addr learned my addr 172.21.15.103:0/184875573 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb08b96640 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d6915c00 0x55d1d69198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb03fff640 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d690aa20 0x55d1d691dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 -- 172.21.15.103:0/184875573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d6915c00 msgr2=0x55d1d69198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d6915c00 0x55d1d69198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 -- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d690aa20 msgr2=0x55d1d691dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d690aa20 0x55d1d691dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 -- 172.21.15.103:0/184875573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d6919e30 con 0x55d1d690b340 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb037fe640 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d69193b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80a4060 tx=0x7fbaf8097f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88d3e6cdfc94120 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbae37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbae37fe640 1 -- 172.21.15.103:0/184875573 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8097670 con 0x55d1d690b340 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbae37fe640 1 -- 172.21.15.103:0/184875573 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbaf80c0210 con 0x55d1d690b340 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc036040 con 0x55d1d690b340 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbae37fe640 1 -- 172.21.15.103:0/184875573 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbaf809a7e0 con 0x55d1d690b340 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.982+0000 7fbb0c9f7740 10 client.16593 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1d6919e30 con 0x55d1d690b340 2024-08-19T01:40:31.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbae37fe640 1 -- 172.21.15.103:0/184875573 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbafc036040 con 0x55d1d690b340 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbae37fe640 1 -- 172.21.15.103:0/184875573 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbaf80bbce0 con 0x55d1d690b340 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb0c9f7740 10 client.16593 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb0c9f7740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb0c9f7740 20 client.16593 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6990000 0x55d1d6992420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb0c9f7740 4 client.16593 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1d686f400 con 0x55d1d6990000 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.983+0000 7fbb03fff640 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6990000 0x55d1d6992420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb03fff640 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6990000 0x55d1d6992420 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162d18c67cb71547 server_cookie=785195df0748331f in_seq=0 out_seq=0 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbae37fe640 10 client.16593 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbae37fe640 1 -- 172.21.15.103:0/184875573 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d1d686f400 con 0x55d1d6990000 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbae37fe640 10 client.16593 handle_command_reply: tid=0 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 client.16593 shutdown 2024-08-19T01:40:31.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 msgr2=0x55d1d69193b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d69193b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80a4060 tx=0x7fbaf8097f90 comp rx=0 tx=0).stop 2024-08-19T01:40:31.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 shutdown_connections 2024-08-19T01:40:31.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55d1d6990000 0x55d1d6992420 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d1d6915c00 0x55d1d69198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1d690b340 0x55d1d69193b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbae37fe640 0 client.16593 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 --2- 172.21.15.103:0/184875573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d1d690aa20 0x55d1d691dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 >> 172.21.15.103:0/184875573 conn(0x55d1d6906fb0 msgr2=0x55d1d68fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 shutdown_connections 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 1 -- 172.21.15.103:0/184875573 wait complete. 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.984+0000 7fbb0c9f7740 20 client.16593 trim_cache size 0 max 16384 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:40:31.990 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:40:31.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbaec0a2b80 msgr2=0x7fbaec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbaec0a2b80 0x7fbaec0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbafc005e10 tx=0x7fbafc0316e0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 msgr2=0x7fbb04198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:31.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb04198530 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800c920 tx=0x7fbaf800cde0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 shutdown_connections 2024-08-19T01:40:31.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbaec0a2b80 0x7fbaec0a5040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb041078c0 0x7fbb0419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb04102260 0x7fbb04198530 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 --2- 172.21.15.103:0/604159705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb041018c0 0x7fbb04197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:31.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 >> 172.21.15.103:0/604159705 conn(0x7fbb040fd620 msgr2=0x7fbb040fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:31.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 shutdown_connections 2024-08-19T01:40:31.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:31.985+0000 7fbb0a5f9640 1 -- 172.21.15.103:0/604159705 wait complete. 2024-08-19T01:40:32.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:31 smithi146 ceph-mon[22098]: pgmap v481: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-08-19T01:40:32.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:31 smithi103 ceph-mon[18436]: pgmap v481: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-08-19T01:40:32.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:31 smithi190 ceph-mon[22032]: pgmap v481: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-08-19T01:40:32.911 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:40:32.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:40:33.189 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 -- 172.21.15.103:0/91346508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c100790 msgr2=0x7feb5c100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 --2- 172.21.15.103:0/91346508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c100790 0x7feb5c100bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009930 tx=0x7feb4c02a420 comp rx=0 tx=0).stop 2024-08-19T01:40:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 -- 172.21.15.103:0/91346508 shutdown_connections 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 --2- 172.21.15.103:0/91346508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5c101130 0x7feb5c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 --2- 172.21.15.103:0/91346508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c100790 0x7feb5c100bf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 --2- 172.21.15.103:0/91346508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c106790 0x7feb5c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 -- 172.21.15.103:0/91346508 >> 172.21.15.103:0/91346508 conn(0x7feb5c0fc4b0 msgr2=0x7feb5c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 -- 172.21.15.103:0/91346508 shutdown_connections 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.738+0000 7feb6384d640 1 -- 172.21.15.103:0/91346508 wait complete. 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 Processor -- start 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 -- start start 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5c100790 0x7feb5c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c101130 0x7feb5c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c106790 0x7feb5c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c110a00 con 0x7feb5c106790 2024-08-19T01:40:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c110880 con 0x7feb5c100790 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb6384d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c110b80 con 0x7feb5c101130 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb61dea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c106790 0x7feb5c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb61dea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c106790 0x7feb5c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39526/0 (socket says 172.21.15.103:39526) 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb61dea640 1 -- 172.21.15.103:0/1552416051 learned_addr learned my addr 172.21.15.103:0/1552416051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.739+0000 7feb615e9640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5c100790 0x7feb5c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb60de8640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c101130 0x7feb5c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb61dea640 1 -- 172.21.15.103:0/1552416051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c101130 msgr2=0x7feb5c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb61dea640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c101130 0x7feb5c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb61dea640 1 -- 172.21.15.103:0/1552416051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5c100790 msgr2=0x7feb5c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb61dea640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5c100790 0x7feb5c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb61dea640 1 -- 172.21.15.103:0/1552416051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5c19d0c0 con 0x7feb5c106790 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb61dea640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c106790 0x7feb5c19c9e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feb54009fd0 tx=0x7feb5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb527fc640 1 -- 172.21.15.103:0/1552416051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54017070 con 0x7feb5c106790 2024-08-19T01:40:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5c19d3b0 con 0x7feb5c106790 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb527fc640 1 -- 172.21.15.103:0/1552416051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb54003e20 con 0x7feb5c106790 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb527fc640 1 -- 172.21.15.103:0/1552416051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54010460 con 0x7feb5c106790 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.740+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5c1a4c90 con 0x7feb5c106790 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.741+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5c100bf0 con 0x7feb5c106790 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.742+0000 7feb527fc640 1 -- 172.21.15.103:0/1552416051 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feb54003970 con 0x7feb5c106790 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.743+0000 7feb527fc640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb480a2b80 0x7feb480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.743+0000 7feb615e9640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb480a2b80 0x7feb480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.743+0000 7feb527fc640 1 -- 172.21.15.103:0/1552416051 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7feb540ced70 con 0x7feb5c106790 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.744+0000 7feb615e9640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb480a2b80 0x7feb480a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb58005d00 tx=0x7feb5800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:33.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.748+0000 7feb527fc640 1 -- 172.21.15.103:0/1552416051 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feb5409b6b0 con 0x7feb5c106790 2024-08-19T01:40:33.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.994+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb5c1a4e90 con 0x7feb5c106790 2024-08-19T01:40:33.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.995+0000 7feb527fc640 1 -- 172.21.15.103:0/1552416051 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11999 (secure 0 0 0) 0x7feb540860c0 con 0x7feb5c106790 2024-08-19T01:40:33.996 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:40:33.996 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":32,"btime":"2024-08-19T01:39:31:934300+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:39:31.568876+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:40:33.996 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 32 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb480a2b80 msgr2=0x7feb480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb480a2b80 0x7feb480a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb58005d00 tx=0x7feb5800b410 comp rx=0 tx=0).stop 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c106790 msgr2=0x7feb5c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c106790 0x7feb5c19c9e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feb54009fd0 tx=0x7feb5400b260 comp rx=0 tx=0).stop 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 shutdown_connections 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb480a2b80 0x7feb480a5040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb5c106790 0x7feb5c19c9e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb5c101130 0x7feb5c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 --2- 172.21.15.103:0/1552416051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb5c100790 0x7feb5c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 >> 172.21.15.103:0/1552416051 conn(0x7feb5c0fc4b0 msgr2=0x7feb5c104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 shutdown_connections 2024-08-19T01:40:33.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:33.998+0000 7feb6384d640 1 -- 172.21.15.103:0/1552416051 wait complete. 2024-08-19T01:40:34.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:33 smithi146 ceph-mon[22098]: pgmap v482: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:40:34.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:33 smithi103 ceph-mon[18436]: pgmap v482: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:40:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:33 smithi190 ceph-mon[22032]: pgmap v482: 97 pgs: 97 active+clean; 6.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:40:34.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:40:34.958 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:35.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:34 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1552416051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:40:35.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:34 smithi103 ceph-mon[18436]: pgmap v483: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:40:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:34 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1552416051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:40:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:34 smithi190 ceph-mon[22032]: pgmap v483: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:40:35.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:34 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1552416051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:40:35.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:34 smithi146 ceph-mon[22098]: pgmap v483: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-08-19T01:40:35.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 -- 172.21.15.103:0/3305625815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 msgr2=0x7ff61410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/3305625815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 0x7ff61410d770 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009ca0 tx=0x7ff60c02a950 comp rx=0 tx=0).stop 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 -- 172.21.15.103:0/3305625815 shutdown_connections 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/3305625815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 0x7ff61410d770 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/3305625815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff6141022a0 0x7ff6141026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/3305625815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff614100b50 0x7ff614100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 -- 172.21.15.103:0/3305625815 >> 172.21.15.103:0/3305625815 conn(0x7ff6140fc500 msgr2=0x7ff6140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 -- 172.21.15.103:0/3305625815 shutdown_connections 2024-08-19T01:40:35.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.499+0000 7ff61a2dd640 1 -- 172.21.15.103:0/3305625815 wait complete. 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 Processor -- start 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 -- start start 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614100b50 0x7ff6140fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6141022a0 0x7ff6140fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 0x7ff6140fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff614110b60 con 0x7ff614102c20 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141109e0 con 0x7ff614100b50 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61a2dd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff614110ce0 con 0x7ff6141022a0 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61887a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 0x7ff6140fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61887a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 0x7ff6140fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39542/0 (socket says 172.21.15.103:39542) 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff61887a640 1 -- 172.21.15.103:0/951852981 learned_addr learned my addr 172.21.15.103:0/951852981 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:35.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.500+0000 7ff6137fe640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6141022a0 0x7ff6140fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff613fff640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614100b50 0x7ff6140fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6137fe640 1 -- 172.21.15.103:0/951852981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614100b50 msgr2=0x7ff6140fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6137fe640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614100b50 0x7ff6140fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6137fe640 1 -- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 msgr2=0x7ff6140fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6137fe640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 0x7ff6140fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6137fe640 1 -- 172.21.15.103:0/951852981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6141a6580 con 0x7ff6141022a0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6137fe640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6141022a0 0x7ff6140fcef0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff60800d6d0 tx=0x7ff60800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6117fa640 1 -- 172.21.15.103:0/951852981 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608017070 con 0x7ff6141022a0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6141a6870 con 0x7ff6141022a0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6117fa640 1 -- 172.21.15.103:0/951852981 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff608009820 con 0x7ff6141022a0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff6117fa640 1 -- 172.21.15.103:0/951852981 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608004c40 con 0x7ff6141022a0 2024-08-19T01:40:35.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.501+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6141a6e00 con 0x7ff6141022a0 2024-08-19T01:40:35.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.504+0000 7ff61c6db740 1 Processor -- start 2024-08-19T01:40:35.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.504+0000 7ff6117fa640 1 -- 172.21.15.103:0/951852981 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff6080099d0 con 0x7ff6141022a0 2024-08-19T01:40:35.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.504+0000 7ff61c6db740 1 -- start start 2024-08-19T01:40:35.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7eff0 0x560a1cc7f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc89c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc8a1d0 0x560a1cc8c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff61c6db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a1cbe4a40 con 0x560a1cc7eff0 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff61c6db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a1cb7c9d0 con 0x560a1cc7f910 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff61c6db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560a1cbe0a00 con 0x560a1cc8a1d0 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff6137fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc8a1d0 0x560a1cc8c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff613fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc89c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff613fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc89c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:54890/0 (socket says 172.21.15.103:54890) 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff613fff640 1 -- 172.21.15.103:0/1388653129 learned_addr learned my addr 172.21.15.103:0/1388653129 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff61887a640 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7eff0 0x560a1cc7f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff613fff640 1 -- 172.21.15.103:0/1388653129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc8a1d0 msgr2=0x560a1cc8c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff613fff640 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc8a1d0 0x560a1cc8c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.505+0000 7ff613fff640 1 -- 172.21.15.103:0/1388653129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7eff0 msgr2=0x560a1cc7f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff613fff640 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7eff0 0x560a1cc7f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff613fff640 1 -- 172.21.15.103:0/1388653129 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1cb83650 con 0x560a1cc7f910 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff61887a640 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7eff0 0x560a1cc7f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:35.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff6117fa640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff5fc0a2c00 0x7ff5fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff613fff640 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc89c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff604009d20 tx=0x7ff604011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b238d9d89bee584 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff61887a640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff5fc0a2c00 0x7ff5fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff6117fa640 1 -- 172.21.15.103:0/951852981 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff6080ce990 con 0x7ff6141022a0 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff5fa7fc640 1 -- 172.21.15.103:0/1388653129 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60401e070 con 0x560a1cc7f910 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.506+0000 7ff5fa7fc640 1 -- 172.21.15.103:0/1388653129 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff604009e10 con 0x560a1cc7f910 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff5fa7fc640 1 -- 172.21.15.103:0/1388653129 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6040187a0 con 0x560a1cc7f910 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61c6db740 1 -- 172.21.15.103:0/1388653129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 msgr2=0x560a1cc89c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61887a640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff5fc0a2c00 0x7ff5fc0a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009a60 tx=0x7ff60c0099f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc89c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff604009d20 tx=0x7ff604011810 comp rx=0 tx=0).stop 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61c6db740 1 -- 172.21.15.103:0/1388653129 shutdown_connections 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc8a1d0 0x560a1cc8c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc89c90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1388653129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7eff0 0x560a1cc7f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.507+0000 7ff61c6db740 1 -- 172.21.15.103:0/1388653129 >> 172.21.15.103:0/1388653129 conn(0x560a1cc7b580 msgr2=0x560a1cc7b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.508+0000 7ff61c6db740 1 -- 172.21.15.103:0/1388653129 shutdown_connections 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.508+0000 7ff61c6db740 1 -- 172.21.15.103:0/1388653129 wait complete. 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.508+0000 7ff61c6db740 1 -- start start 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.508+0000 7ff61c6db740 1 Processor -- start 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc89c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc86f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc87710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61c6db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a1cc8ce40 con 0x560a1cc8a1d0 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61c6db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a1cb9fb20 con 0x560a1cc7f910 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61c6db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560a1cc8cfc0 con 0x560a1cc7eff0 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61c6db740 10 client.0 _ll_register_callbacks cb 0x560a1cb9dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc87710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc87710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39568/0 (socket says 172.21.15.103:39568) 2024-08-19T01:40:35.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 -- 172.21.15.103:0/1723040651 learned_addr learned my addr 172.21.15.103:0/1723040651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff6137fe640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc86f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff613fff640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc89c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 -- 172.21.15.103:0/1723040651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 msgr2=0x560a1cc89c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc89c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 -- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 msgr2=0x560a1cc86f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc86f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 -- 172.21.15.103:0/1723040651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1c9563c0 con 0x560a1cc8a1d0 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff6137fe640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc86f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff61887a640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc87710 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff60c005fe0 tx=0x7ff60c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0cf43ff2bbb7e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.509+0000 7ff5f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1723040651 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c056060 con 0x560a1cc8a1d0 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1723040651 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff60c004250 con 0x560a1cc8a1d0 2024-08-19T01:40:35.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff604009e10 con 0x560a1cc8a1d0 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1723040651 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff60c034040 con 0x560a1cc8a1d0 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 10 client.16611 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a1cc98970 con 0x560a1cc8a1d0 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1723040651 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff604009e10 con 0x560a1cc8a1d0 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1723040651 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff60c0309c0 con 0x560a1cc8a1d0 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 10 client.16611 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 10 client.16611 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 20 client.16611 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cc99e30 0x560a1cc9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 4 client.16611 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.510+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a1cbf1fb0 con 0x560a1cc99e30 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.511+0000 7ff613fff640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cc99e30 0x560a1cc9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.511+0000 7ff613fff640 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cc99e30 0x560a1cc9c280 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a046a7547c24966 server_cookie=1e53b52319e3e786 in_seq=0 out_seq=0 2024-08-19T01:40:35.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.511+0000 7ff5f37fe640 10 client.16611 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1723040651 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a1cbf1fb0 con 0x560a1cc99e30 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff5f37fe640 10 client.16611 handle_command_reply: tid=0 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 client.16611 shutdown 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 msgr2=0x560a1cc87710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc87710 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff60c005fe0 tx=0x7ff60c0095c0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 shutdown_connections 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cc99e30 0x560a1cc9c280 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc87710 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc86f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1723040651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc89c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 >> 172.21.15.103:0/1723040651 conn(0x560a1cc7b580 msgr2=0x560a1cc812b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff5f37fe640 0 client.16611 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:40:35.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 shutdown_connections 2024-08-19T01:40:35.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.512+0000 7ff61c6db740 1 -- 172.21.15.103:0/1723040651 wait complete. 2024-08-19T01:40:35.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.513+0000 7ff61c6db740 20 client.16611 trim_cache size 0 max 16384 2024-08-19T01:40:35.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.525+0000 7ff61c6db740 1 Processor -- start 2024-08-19T01:40:35.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.525+0000 7ff61c6db740 1 -- start start 2024-08-19T01:40:35.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc86840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7f910 0x560a1cc82310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc8a1d0 0x560a1cc82850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff61c6db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141af2f0 con 0x560a1cc7f910 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff61c6db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141b0f50 con 0x560a1cc8a1d0 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff61c6db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141af9c0 con 0x560a1cc7eff0 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7f910 0x560a1cc82310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7f910 0x560a1cc82310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39578/0 (socket says 172.21.15.103:39578) 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 -- 172.21.15.103:0/3742600480 learned_addr learned my addr 172.21.15.103:0/3742600480 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff613fff640 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc86840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 -- 172.21.15.103:0/3742600480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 msgr2=0x560a1cc86840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc86840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 -- 172.21.15.103:0/3742600480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc8a1d0 msgr2=0x560a1cc82850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc8a1d0 0x560a1cc82850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 -- 172.21.15.103:0/3742600480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1cc7f740 con 0x560a1cc7f910 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff613fff640 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc86840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff6137fe640 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7f910 0x560a1cc82310 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cc4c0 tx=0x7ff6080d44d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e54b72f5e3c0c0f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.526+0000 7ff5f9ffb640 1 -- 172.21.15.103:0/3742600480 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608020040 con 0x560a1cc7f910 2024-08-19T01:40:35.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff5f9ffb640 1 -- 172.21.15.103:0/3742600480 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff608009d40 con 0x560a1cc7f910 2024-08-19T01:40:35.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff5f9ffb640 1 -- 172.21.15.103:0/3742600480 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080b9e60 con 0x560a1cc7f910 2024-08-19T01:40:35.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 -- 172.21.15.103:0/3742600480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7f910 msgr2=0x560a1cc82310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7f910 0x560a1cc82310 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cc4c0 tx=0x7ff6080d44d0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 -- 172.21.15.103:0/3742600480 shutdown_connections 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc8a1d0 0x560a1cc82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc7f910 0x560a1cc82310 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 --2- 172.21.15.103:0/3742600480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc86840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 -- 172.21.15.103:0/3742600480 >> 172.21.15.103:0/3742600480 conn(0x560a1cc7b580 msgr2=0x560a1cc89850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 -- 172.21.15.103:0/3742600480 shutdown_connections 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.527+0000 7ff61c6db740 1 -- 172.21.15.103:0/3742600480 wait complete. 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 -- start start 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 Processor -- start 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc83330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141b07c0 con 0x560a1cc8a1d0 2024-08-19T01:40:35.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141b0090 con 0x560a1cc7f910 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141ad2d0 con 0x560a1cc7eff0 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61c6db740 10 client.0 _ll_register_callbacks cb 0x7ff60c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.528+0000 7ff61887a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc83330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc83330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39590/0 (socket says 172.21.15.103:39590) 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 -- 172.21.15.103:0/1106522567 learned_addr learned my addr 172.21.15.103:0/1106522567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff6137fe640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff613fff640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 -- 172.21.15.103:0/1106522567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 msgr2=0x560a1cc92260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc92260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 -- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 msgr2=0x560a1cc927a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc927a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 -- 172.21.15.103:0/1106522567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1cc83870 con 0x560a1cc8a1d0 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff6137fe640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc927a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:35.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61887a640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc83330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff60c030040 tx=0x7ff60c004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78d07a239b02c89 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff5f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1106522567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c061030 con 0x560a1cc8a1d0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1106522567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff60c04e070 con 0x560a1cc8a1d0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a1cbec4c0 con 0x560a1cc8a1d0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.529+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1106522567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff60c056040 con 0x560a1cc8a1d0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 10 client.16617 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a1cd040c0 con 0x560a1cc8a1d0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1106522567 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560a1cbec4c0 con 0x560a1cc8a1d0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1106522567 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff60c030440 con 0x560a1cc8a1d0 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 10 client.16617 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 10 client.16617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 20 client.16617 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:35.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cd04830 0x560a1cd06c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 4 client.16617 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a1cb69170 con 0x560a1cd04830 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.530+0000 7ff613fff640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cd04830 0x560a1cd06c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff613fff640 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cd04830 0x560a1cd06c50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34b78503261db53d server_cookie=c90af802ba1856ee in_seq=0 out_seq=0 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff5f37fe640 10 client.16617 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff5f37fe640 1 -- 172.21.15.103:0/1106522567 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a1cb69170 con 0x560a1cd04830 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff5f37fe640 10 client.16617 handle_command_reply: tid=0 2024-08-19T01:40:35.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 client.16617 shutdown 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 msgr2=0x560a1cc83330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc83330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff60c030040 tx=0x7ff60c004480 comp rx=0 tx=0).stop 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 shutdown_connections 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x560a1cd04830 0x560a1cd06c50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a1cc8a1d0 0x560a1cc83330 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff5f37fe640 0 client.16617 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cc7f910 0x560a1cc927a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 --2- 172.21.15.103:0/1106522567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a1cc7eff0 0x560a1cc92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 >> 172.21.15.103:0/1106522567 conn(0x560a1cc7b580 msgr2=0x560a1cc82310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:35.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 shutdown_connections 2024-08-19T01:40:35.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 1 -- 172.21.15.103:0/1106522567 wait complete. 2024-08-19T01:40:35.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.531+0000 7ff61c6db740 20 client.16617 trim_cache size 0 max 16384 2024-08-19T01:40:35.537 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff5fc0a2c00 msgr2=0x7ff5fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff5fc0a2c00 0x7ff5fc0a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009a60 tx=0x7ff60c0099f0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6141022a0 msgr2=0x7ff6140fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6141022a0 0x7ff6140fcef0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff60800d6d0 tx=0x7ff60800db90 comp rx=0 tx=0).stop 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 shutdown_connections 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff5fc0a2c00 0x7ff5fc0a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff614102c20 0x7ff6140fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6141022a0 0x7ff6140fcef0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 --2- 172.21.15.103:0/951852981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff614100b50 0x7ff6140fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 >> 172.21.15.103:0/951852981 conn(0x7ff6140fc500 msgr2=0x7ff614069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 shutdown_connections 2024-08-19T01:40:35.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:35.532+0000 7ff61a2dd640 1 -- 172.21.15.103:0/951852981 wait complete. 2024-08-19T01:40:36.225 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:35 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:40:36.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:40:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:35 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:40:36.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:35 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:40:36.539 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:37.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 -- 172.21.15.103:0/222795395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c100b50 msgr2=0x7f864c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 --2- 172.21.15.103:0/222795395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c100b50 0x7f864c100f30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f863c009900 tx=0x7f863c02a480 comp rx=0 tx=0).stop 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 -- 172.21.15.103:0/222795395 shutdown_connections 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 --2- 172.21.15.103:0/222795395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c102c20 0x7f864c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 --2- 172.21.15.103:0/222795395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f864c1022a0 0x7f864c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 --2- 172.21.15.103:0/222795395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c100b50 0x7f864c100f30 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 -- 172.21.15.103:0/222795395 >> 172.21.15.103:0/222795395 conn(0x7f864c0fc500 msgr2=0x7f864c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 -- 172.21.15.103:0/222795395 shutdown_connections 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 -- 172.21.15.103:0/222795395 wait complete. 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.082+0000 7f8651af7640 1 Processor -- start 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f8651af7640 1 -- start start 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f8651af7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f864c100b50 0x7f864c1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f8651af7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c1022a0 0x7f864c103f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f8651af7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c102c20 0x7f864c104470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f8651af7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f864c110b60 con 0x7f864c102c20 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f8651af7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1109e0 con 0x7f864c100b50 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f8651af7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f864c110ce0 con 0x7f864c1022a0 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f864bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c102c20 0x7f864c104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f864b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f864c100b50 0x7f864c1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f864bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c102c20 0x7f864c104470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39604/0 (socket says 172.21.15.103:39604) 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f864bfff640 1 -- 172.21.15.103:0/1527578243 learned_addr learned my addr 172.21.15.103:0/1527578243 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.083+0000 7f864affd640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c1022a0 0x7f864c103f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f864bfff640 1 -- 172.21.15.103:0/1527578243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c1022a0 msgr2=0x7f864c103f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f864bfff640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c1022a0 0x7f864c103f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f864bfff640 1 -- 172.21.15.103:0/1527578243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f864c100b50 msgr2=0x7f864c1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f864bfff640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f864c100b50 0x7f864c1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f864bfff640 1 -- 172.21.15.103:0/1527578243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c1a65a0 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f864bfff640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c102c20 0x7f864c104470 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f864400cce0 tx=0x7f864400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f8648ff9640 1 -- 172.21.15.103:0/1527578243 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864400f450 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f8648ff9640 1 -- 172.21.15.103:0/1527578243 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f864400faa0 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c1a6890 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f8648ff9640 1 -- 172.21.15.103:0/1527578243 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644018b70 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.084+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c1a6dd0 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.087+0000 7f8648ff9640 1 -- 172.21.15.103:0/1527578243 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8644018d10 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.087+0000 7f8648ff9640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86340a2c00 0x7f86340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.087+0000 7f8648ff9640 1 -- 172.21.15.103:0/1527578243 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f86440cf2b0 con 0x7f864c102c20 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.087+0000 7f864b7fe640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86340a2c00 0x7f86340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 Processor -- start 2024-08-19T01:40:37.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 -- start start 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f864b7fe640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86340a2c00 0x7f86340a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f863c009230 tx=0x7f863c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550d51d0 0x5631550d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56315502fa40 con 0x5631550ca910 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563154fc79d0 con 0x5631550d51d0 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f8653ef5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56315502ba00 con 0x5631550c9ff0 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f864affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f864affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39606/0 (socket says 172.21.15.103:39606) 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f864affd640 1 -- 172.21.15.103:0/4280652748 learned_addr learned my addr 172.21.15.103:0/4280652748 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f864b7fe640 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.088+0000 7f864bfff640 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550d51d0 0x5631550d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f864b7fe640 1 -- 172.21.15.103:0/4280652748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550d51d0 msgr2=0x5631550d75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f864b7fe640 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550d51d0 0x5631550d75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f864b7fe640 1 -- 172.21.15.103:0/4280652748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 msgr2=0x5631550d4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f864b7fe640 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f864b7fe640 1 -- 172.21.15.103:0/4280652748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563154fce650 con 0x5631550c9ff0 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f864affd640 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f864b7fe640 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ca3d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f863c005940 tx=0x7f863c03d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc86be337281c9b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f8631ffb640 1 -- 172.21.15.103:0/4280652748 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c04c070 con 0x5631550c9ff0 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f8631ffb640 1 -- 172.21.15.103:0/4280652748 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f863c0035f0 con 0x5631550c9ff0 2024-08-19T01:40:37.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.089+0000 7f8631ffb640 1 -- 172.21.15.103:0/4280652748 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c035040 con 0x5631550c9ff0 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 -- 172.21.15.103:0/4280652748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 msgr2=0x5631550ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ca3d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f863c005940 tx=0x7f863c03d000 comp rx=0 tx=0).stop 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 -- 172.21.15.103:0/4280652748 shutdown_connections 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550d51d0 0x5631550d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 --2- 172.21.15.103:0/4280652748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ca3d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 -- 172.21.15.103:0/4280652748 >> 172.21.15.103:0/4280652748 conn(0x5631550c6580 msgr2=0x5631550c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.090+0000 7f8653ef5740 1 -- 172.21.15.103:0/4280652748 shutdown_connections 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 -- 172.21.15.103:0/4280652748 wait complete. 2024-08-19T01:40:37.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 -- start start 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 Processor -- start 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550c9ff0 0x5631550d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550d51d0 0x5631550d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5631550d7da0 con 0x5631550ca910 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563154feab20 con 0x5631550c9ff0 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.091+0000 7f8653ef5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5631550d7f20 con 0x5631550d51d0 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f8653ef5740 10 client.0 _ll_register_callbacks cb 0x563154fe8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39618/0 (socket says 172.21.15.103:39618) 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 -- 172.21.15.103:0/1793520550 learned_addr learned my addr 172.21.15.103:0/1793520550 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864b7fe640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550c9ff0 0x5631550d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864bfff640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550d51d0 0x5631550d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 -- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550d51d0 msgr2=0x5631550d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550d51d0 0x5631550d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 -- 172.21.15.103:0/1793520550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550c9ff0 msgr2=0x5631550d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550c9ff0 0x5631550d2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 -- 172.21.15.103:0/1793520550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563154da13c0 con 0x5631550ca910 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864b7fe640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550c9ff0 0x5631550d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f864affd640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d2550 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f864000baa0 tx=0x7f864000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7036dea084042bd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f862affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f862affd640 1 -- 172.21.15.103:0/1793520550 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640011440 con 0x5631550ca910 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f863c0035f0 con 0x5631550ca910 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.092+0000 7f862affd640 1 -- 172.21.15.103:0/1793520550 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f86400115e0 con 0x5631550ca910 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f862affd640 1 -- 172.21.15.103:0/1793520550 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f86400117e0 con 0x5631550ca910 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 10 client.16629 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:37.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631550e3970 con 0x5631550ca910 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f862affd640 1 -- 172.21.15.103:0/1793520550 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f863c0035f0 con 0x5631550ca910 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f862affd640 1 -- 172.21.15.103:0/1793520550 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f86400269c0 con 0x5631550ca910 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 10 client.16629 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 10 client.16629 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 20 client.16629 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x5631550e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 4 client.16629 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.093+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56315503cfb0 con 0x5631550e4e30 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.094+0000 7f864b7fe640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x5631550e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.094+0000 7f862affd640 10 client.16629 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.094+0000 7f864b7fe640 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x5631550e72d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3698252e23020498 server_cookie=bae1fd71aa0765d7 in_seq=0 out_seq=0 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f862affd640 1 -- 172.21.15.103:0/1793520550 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56315503cfb0 con 0x5631550e4e30 2024-08-19T01:40:37.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f862affd640 10 client.16629 handle_command_reply: tid=0 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 client.16629 shutdown 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 msgr2=0x5631550d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d2550 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f864000baa0 tx=0x7f864000bf60 comp rx=0 tx=0).stop 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 shutdown_connections 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x5631550e72d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f862affd640 0 client.16629 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550d51d0 0x5631550d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550ca910 0x5631550d2550 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 --2- 172.21.15.103:0/1793520550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550c9ff0 0x5631550d2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 >> 172.21.15.103:0/1793520550 conn(0x5631550c6580 msgr2=0x5631550cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 shutdown_connections 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 1 -- 172.21.15.103:0/1793520550 wait complete. 2024-08-19T01:40:37.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.095+0000 7f8653ef5740 20 client.16629 trim_cache size 0 max 16384 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 Processor -- start 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 -- start start 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1af280 con 0x5631550d51d0 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1b0ee0 con 0x5631550ca910 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f8653ef5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1af950 con 0x5631550c9ff0 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39624/0 (socket says 172.21.15.103:39624) 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 -- 172.21.15.103:0/3821119095 learned_addr learned my addr 172.21.15.103:0/3821119095 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864b7fe640 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864affd640 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 -- 172.21.15.103:0/3821119095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 msgr2=0x5631550ecba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ecba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 -- 172.21.15.103:0/3821119095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 msgr2=0x5631550ed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550ed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864bfff640 1 -- 172.21.15.103:0/3821119095 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631550c7630 con 0x5631550d51d0 2024-08-19T01:40:37.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.111+0000 7f864b7fe640 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:37.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.112+0000 7f864bfff640 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550c4d90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8644098510 tx=0x7f864400fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36415b9b20f8d08 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.112+0000 7f86317fa640 1 -- 172.21.15.103:0/3821119095 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440b8070 con 0x5631550d51d0 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.112+0000 7f86317fa640 1 -- 172.21.15.103:0/3821119095 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f864409fef0 con 0x5631550d51d0 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.112+0000 7f86317fa640 1 -- 172.21.15.103:0/3821119095 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86440ba2c0 con 0x5631550d51d0 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 -- 172.21.15.103:0/3821119095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 msgr2=0x5631550c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550c4d90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8644098510 tx=0x7f864400fd60 comp rx=0 tx=0).stop 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 -- 172.21.15.103:0/3821119095 shutdown_connections 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550c4d90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3821119095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 -- 172.21.15.103:0/3821119095 >> 172.21.15.103:0/3821119095 conn(0x5631550c6580 msgr2=0x5631550d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 -- 172.21.15.103:0/3821119095 shutdown_connections 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 -- 172.21.15.103:0/3821119095 wait complete. 2024-08-19T01:40:37.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 -- start start 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.113+0000 7f8653ef5740 1 Processor -- start 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f8653ef5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f8653ef5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1b0750 con 0x5631550d51d0 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f8653ef5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1b0020 con 0x5631550ca910 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f8653ef5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f864c1ad260 con 0x5631550c9ff0 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f8653ef5740 10 client.0 _ll_register_callbacks cb 0x7f8640004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:57716/0 (socket says 172.21.15.103:57716) 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 -- 172.21.15.103:0/3508626481 learned_addr learned my addr 172.21.15.103:0/3508626481 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864bfff640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864affd640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 -- 172.21.15.103:0/3508626481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 msgr2=0x5631550cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 -- 172.21.15.103:0/3508626481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 msgr2=0x5631550dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550dc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 -- 172.21.15.103:0/3508626481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631550dc7b0 con 0x5631550c9ff0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864affd640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f864b7fe640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ce9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f863c004650 tx=0x7f863c02ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c595474600a2c765 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.114+0000 7f862affd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f862affd640 1 -- 172.21.15.103:0/3508626481 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c04c040 con 0x5631550c9ff0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f864409fef0 con 0x5631550c9ff0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f862affd640 1 -- 172.21.15.103:0/3508626481 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f863c003800 con 0x5631550c9ff0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f862affd640 1 -- 172.21.15.103:0/3508626481 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f863c035040 con 0x5631550c9ff0 2024-08-19T01:40:37.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f8653ef5740 10 client.25987 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:37.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631550dda90 con 0x5631550c9ff0 2024-08-19T01:40:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f862affd640 1 -- 172.21.15.103:0/3508626481 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f864409fef0 con 0x5631550c9ff0 2024-08-19T01:40:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f862affd640 1 -- 172.21.15.103:0/3508626481 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f863c05a750 con 0x5631550c9ff0 2024-08-19T01:40:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f8653ef5740 10 client.25987 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f8653ef5740 10 client.25987 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:40:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f8653ef5740 20 client.25987 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.115+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x563155151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.116+0000 7f8653ef5740 4 client.25987 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.116+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631550374c0 con 0x5631550e4e30 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.116+0000 7f864affd640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x563155151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.116+0000 7f862affd640 10 client.25987 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.116+0000 7f864affd640 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x563155151b80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=239b03669d8d16f3 server_cookie=3218b5021fc50f43 in_seq=0 out_seq=0 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.116+0000 7f862affd640 1 -- 172.21.15.103:0/3508626481 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631550374c0 con 0x5631550e4e30 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.116+0000 7f862affd640 10 client.25987 handle_command_reply: tid=0 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 client.25987 shutdown 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 msgr2=0x5631550ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ce9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f863c004650 tx=0x7f863c02ac00 comp rx=0 tx=0).stop 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 shutdown_connections 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5631550e4e30 0x563155151b80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631550d51d0 0x5631550dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5631550ca910 0x5631550cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 --2- 172.21.15.103:0/3508626481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5631550c9ff0 0x5631550ce9e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 >> 172.21.15.103:0/3508626481 conn(0x5631550c6580 msgr2=0x5631550c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f862affd640 0 client.25987 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:40:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 shutdown_connections 2024-08-19T01:40:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 1 -- 172.21.15.103:0/3508626481 wait complete. 2024-08-19T01:40:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.117+0000 7f8653ef5740 20 client.25987 trim_cache size 0 max 16384 2024-08-19T01:40:37.123 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86340a2c00 msgr2=0x7f86340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86340a2c00 0x7f86340a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f863c009230 tx=0x7f863c0058d0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c102c20 msgr2=0x7f864c104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c102c20 0x7f864c104470 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f864400cce0 tx=0x7f864400a620 comp rx=0 tx=0).stop 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 shutdown_connections 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f86340a2c00 0x7f86340a50c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f864c102c20 0x7f864c104470 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c1022a0 0x7f864c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 --2- 172.21.15.103:0/1527578243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f864c100b50 0x7f864c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 >> 172.21.15.103:0/1527578243 conn(0x7f864c0fc500 msgr2=0x7f864c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:37.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.118+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 shutdown_connections 2024-08-19T01:40:37.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:37.119+0000 7f8651af7640 1 -- 172.21.15.103:0/1527578243 wait complete. 2024-08-19T01:40:37.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:40:37.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:40:37.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:40:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:36 smithi103 ceph-mon[18436]: pgmap v484: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 23 op/s 2024-08-19T01:40:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:40:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:40:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:40:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:36 smithi190 ceph-mon[22032]: pgmap v484: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 23 op/s 2024-08-19T01:40:37.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:40:37.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:40:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:40:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:36 smithi146 ceph-mon[22098]: pgmap v484: 97 pgs: 97 active+clean; 6.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 23 op/s 2024-08-19T01:40:37.963 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:40:38.257 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:38.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 -- 172.21.15.103:0/1618735937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 msgr2=0x7fb8f010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 --2- 172.21.15.103:0/1618735937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f010d770 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009ca0 tx=0x7fb8e802a950 comp rx=0 tx=0).stop 2024-08-19T01:40:38.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 -- 172.21.15.103:0/1618735937 shutdown_connections 2024-08-19T01:40:38.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 --2- 172.21.15.103:0/1618735937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f010d770 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 --2- 172.21.15.103:0/1618735937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8f01022a0 0x7fb8f01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 --2- 172.21.15.103:0/1618735937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8f0100b50 0x7fb8f0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 -- 172.21.15.103:0/1618735937 >> 172.21.15.103:0/1618735937 conn(0x7fb8f00fc500 msgr2=0x7fb8f00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 -- 172.21.15.103:0/1618735937 shutdown_connections 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 -- 172.21.15.103:0/1618735937 wait complete. 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 Processor -- start 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.768+0000 7fb8f7548640 1 -- start start 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f7548640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8f0100b50 0x7fb8f0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f7548640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8f01022a0 0x7fb8f0199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f7548640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f01a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f7548640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0110b60 con 0x7fb8f0102c20 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f7548640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01109e0 con 0x7fb8f01022a0 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f7548640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0110ce0 con 0x7fb8f0100b50 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f01a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f01a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39654/0 (socket says 172.21.15.103:39654) 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 -- 172.21.15.103:0/4061931348 learned_addr learned my addr 172.21.15.103:0/4061931348 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8f0100b50 0x7fb8f0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8f01022a0 0x7fb8f0199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 -- 172.21.15.103:0/4061931348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8f0100b50 msgr2=0x7fb8f0198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8f0100b50 0x7fb8f0198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 -- 172.21.15.103:0/4061931348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8f01022a0 msgr2=0x7fb8f0199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8f01022a0 0x7fb8f0199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 -- 172.21.15.103:0/4061931348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f01a4590 con 0x7fb8f0102c20 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8f0100b50 0x7fb8f0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.769+0000 7fb8f5ae5640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f01a4050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb8e802ae70 tx=0x7fb8e802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.770+0000 7fb8e67fc640 1 -- 172.21.15.103:0/4061931348 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8034040 con 0x7fb8f0102c20 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.770+0000 7fb8e67fc640 1 -- 172.21.15.103:0/4061931348 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e80308f0 con 0x7fb8f0102c20 2024-08-19T01:40:38.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.770+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f01a4760 con 0x7fb8f0102c20 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.770+0000 7fb8e67fc640 1 -- 172.21.15.103:0/4061931348 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8004780 con 0x7fb8f0102c20 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.770+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f01a4c10 con 0x7fb8f0102c20 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.772+0000 7fb8e67fc640 1 -- 172.21.15.103:0/4061931348 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb8e8037020 con 0x7fb8f0102c20 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.772+0000 7fb8f9946740 1 Processor -- start 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f9946740 1 -- start start 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff4ed0 0x5560a1ff52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a20040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a20045e0 0x5560a20069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f9946740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560a1f5aa40 con 0x5560a1ff4ed0 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f9946740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5560a1ef29d0 con 0x5560a1ff57f0 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f9946740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5560a1f56a00 con 0x5560a20045e0 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff4ed0 0x5560a1ff52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff4ed0 0x5560a1ff52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39664/0 (socket says 172.21.15.103:39664) 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/3343130242 learned_addr learned my addr 172.21.15.103:0/3343130242 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8e67fc640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb8dc0a2c00 0x7fb8dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a20040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a20045e0 0x5560a20069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/3343130242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a20045e0 msgr2=0x5560a20069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a20045e0 0x5560a20069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/3343130242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 msgr2=0x5560a20040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.773+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a20040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8f5ae5640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb8dc0a2c00 0x7fb8dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/3343130242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a1ef9650 con 0x5560a1ff4ed0 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8e67fc640 1 -- 172.21.15.103:0/4061931348 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fb8e80f0120 con 0x7fb8f0102c20 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff4ed0 0x5560a1ff52b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005e00 tx=0x7fb8e00094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4964ff2565c34339 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:38.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8db7fe640 1 -- 172.21.15.103:0/3343130242 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0017070 con 0x5560a1ff4ed0 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8db7fe640 1 -- 172.21.15.103:0/3343130242 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e00138b0 con 0x5560a1ff4ed0 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8f5ae5640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb8dc0a2c00 0x7fb8dc0a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8004af0 tx=0x7fb8e802a950 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.774+0000 7fb8db7fe640 1 -- 172.21.15.103:0/3343130242 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0012720 con 0x5560a1ff4ed0 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 -- 172.21.15.103:0/3343130242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff4ed0 msgr2=0x5560a1ff52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff4ed0 0x5560a1ff52b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005e00 tx=0x7fb8e00094c0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 -- 172.21.15.103:0/3343130242 shutdown_connections 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a20045e0 0x5560a20069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a20040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3343130242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff4ed0 0x5560a1ff52b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 -- 172.21.15.103:0/3343130242 >> 172.21.15.103:0/3343130242 conn(0x5560a1ff1580 msgr2=0x5560a1ff1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 -- 172.21.15.103:0/3343130242 shutdown_connections 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 -- 172.21.15.103:0/3343130242 wait complete. 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.775+0000 7fb8f9946740 1 -- start start 2024-08-19T01:40:38.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 1 Processor -- start 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a20019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a2001f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560a20071b0 con 0x5560a1ff57f0 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5560a1f15b20 con 0x5560a20045e0 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5560a2007330 con 0x5560a1ff4ed0 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f9946740 10 client.0 _ll_register_callbacks cb 0x5560a1f13bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f52e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a20019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f52e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a20019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39672/0 (socket says 172.21.15.103:39672) 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f52e4640 1 -- 172.21.15.103:0/1339918375 learned_addr learned my addr 172.21.15.103:0/1339918375 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a2001f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.776+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f52e4640 1 -- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 msgr2=0x5560a2001490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f52e4640 1 -- 172.21.15.103:0/1339918375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 msgr2=0x5560a2001f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a2001f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f52e4640 1 -- 172.21.15.103:0/1339918375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a1ccc3c0 con 0x5560a1ff57f0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a20019d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00bfb0 tx=0x7fb8ec0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d622c8d7b2d80294 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/1339918375 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0035f0 con 0x5560a1ff57f0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/1339918375 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fb8ec003790 con 0x5560a1ff57f0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e00138b0 con 0x5560a1ff57f0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/1339918375 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8ec003990 con 0x5560a1ff57f0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f9946740 10 client.16647 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560a2012e30 con 0x5560a1ff57f0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.777+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/1339918375 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e00138b0 con 0x5560a1ff57f0 2024-08-19T01:40:38.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/1339918375 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb8ec019070 con 0x5560a1ff57f0 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f9946740 10 client.16647 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f9946740 10 client.16647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f9946740 20 client.16647 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a2014390 0x5560a20167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f9946740 4 client.16647 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560a1f67fb0 con 0x5560a2014390 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a2014390 0x5560a20167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8d8ff9640 10 client.16647 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.778+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a2014390 0x5560a20167b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94563c8681e8b614 server_cookie=d0f846ef74147692 in_seq=0 out_seq=0 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.779+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/1339918375 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560a1f67fb0 con 0x5560a2014390 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.779+0000 7fb8d8ff9640 10 client.16647 handle_command_reply: tid=0 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.779+0000 7fb8f9946740 1 client.16647 shutdown 2024-08-19T01:40:38.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.779+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 msgr2=0x5560a20019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.779+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a20019d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00bfb0 tx=0x7fb8ec0128e0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.779+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 shutdown_connections 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a2014390 0x5560a20167b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a2001f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a20019d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1339918375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8d8ff9640 0 client.16647 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 >> 172.21.15.103:0/1339918375 conn(0x5560a1ff1580 msgr2=0x5560a1ff72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 shutdown_connections 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 1 -- 172.21.15.103:0/1339918375 wait complete. 2024-08-19T01:40:38.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.780+0000 7fb8f9946740 20 client.16647 trim_cache size 0 max 16384 2024-08-19T01:40:38.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 Processor -- start 2024-08-19T01:40:38.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 -- start start 2024-08-19T01:40:38.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a201c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a20045e0 0x5560a201c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01aaf00 con 0x5560a20045e0 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01acb60 con 0x5560a1ff57f0 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.795+0000 7fb8f9946740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01ab5d0 con 0x5560a1ff4ed0 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a20045e0 0x5560a201c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a20045e0 0x5560a201c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39688/0 (socket says 172.21.15.103:39688) 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/1506062574 learned_addr learned my addr 172.21.15.103:0/1506062574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a201c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/1506062574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 msgr2=0x5560a2001d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/1506062574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 msgr2=0x5560a201c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a201c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/1506062574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a1ff5620 con 0x5560a20045e0 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a20045e0 0x5560a201c5a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x5560a2007250 tx=0x7fb8e0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe42c6a4aaf7b1a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.796+0000 7fb8daffd640 1 -- 172.21.15.103:0/1506062574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0017040 con 0x5560a20045e0 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8daffd640 1 -- 172.21.15.103:0/1506062574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e0030070 con 0x5560a20045e0 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8daffd640 1 -- 172.21.15.103:0/1506062574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0025040 con 0x5560a20045e0 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 -- 172.21.15.103:0/1506062574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a20045e0 msgr2=0x5560a201c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a20045e0 0x5560a201c5a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x5560a2007250 tx=0x7fb8e0005c90 comp rx=0 tx=0).stop 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 -- 172.21.15.103:0/1506062574 shutdown_connections 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a20045e0 0x5560a201c5a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a1ff57f0 0x5560a201c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 --2- 172.21.15.103:0/1506062574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2001d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 -- 172.21.15.103:0/1506062574 >> 172.21.15.103:0/1506062574 conn(0x5560a1ff1580 msgr2=0x5560a2000580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 -- 172.21.15.103:0/1506062574 shutdown_connections 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.797+0000 7fb8f9946740 1 -- 172.21.15.103:0/1506062574 wait complete. 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 -- start start 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 Processor -- start 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2007b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a2008090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a207d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01ac3d0 con 0x5560a1ff57f0 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01abca0 con 0x5560a20045e0 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01a8ee0 con 0x5560a1ff4ed0 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f9946740 10 client.0 _ll_register_callbacks cb 0x7fb8ec00b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f4ae3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a207d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.798+0000 7fb8f52e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a2008090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f4ae3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a207d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:55046/0 (socket says 172.21.15.103:55046) 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f4ae3640 1 -- 172.21.15.103:0/3419155396 learned_addr learned my addr 172.21.15.103:0/3419155396 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2007b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f52e4640 1 -- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 msgr2=0x5560a2007b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2007b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f52e4640 1 -- 172.21.15.103:0/3419155396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 msgr2=0x5560a207d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a207d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f52e4640 1 -- 172.21.15.103:0/3419155396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a207d590 con 0x5560a1ff57f0 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f52e4640 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a2008090 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec010e70 tx=0x7fb8ec00bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae07995a31ae21b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/3419155396 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec02b040 con 0x5560a1ff57f0 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/3419155396 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fb8ec003b30 con 0x5560a1ff57f0 2024-08-19T01:40:38.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.799+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e0030070 con 0x5560a1ff57f0 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/3419155396 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8ec037070 con 0x5560a1ff57f0 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 10 client.16653 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5560a207e890 con 0x5560a1ff57f0 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/3419155396 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e0030070 con 0x5560a1ff57f0 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/3419155396 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb8ec019040 con 0x5560a1ff57f0 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 10 client.16653 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 10 client.16653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 20 client.16653 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a207efb0 0x5560a20813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 4 client.16653 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.800+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560a1f624c0 con 0x5560a207efb0 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a207efb0 0x5560a20813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:38.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8d8ff9640 10 client.16653 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8f4ae3640 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a207efb0 0x5560a20813d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7039f5be19b3f93 server_cookie=f8cdfd96d45363ff in_seq=0 out_seq=0 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8d8ff9640 1 -- 172.21.15.103:0/3419155396 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560a1f624c0 con 0x5560a207efb0 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8d8ff9640 10 client.16653 handle_command_reply: tid=0 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8f9946740 1 client.16653 shutdown 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 msgr2=0x5560a2008090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a2008090 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec010e70 tx=0x7fb8ec00bc70 comp rx=0 tx=0).stop 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 shutdown_connections 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.801+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5560a207efb0 0x5560a20813d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5560a20045e0 0x5560a207d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a1ff57f0 0x5560a2008090 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f9946740 1 --2- 172.21.15.103:0/3419155396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560a1ff4ed0 0x5560a2007b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 >> 172.21.15.103:0/3419155396 conn(0x5560a1ff1580 msgr2=0x5560a2002a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:38.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8d8ff9640 0 client.16653 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:40:38.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 shutdown_connections 2024-08-19T01:40:38.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f9946740 1 -- 172.21.15.103:0/3419155396 wait complete. 2024-08-19T01:40:38.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f9946740 20 client.16653 trim_cache size 0 max 16384 2024-08-19T01:40:38.815 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:40:38.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb8dc0a2c00 msgr2=0x7fb8dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f7548640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb8dc0a2c00 0x7fb8dc0a50c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8004af0 tx=0x7fb8e802a950 comp rx=0 tx=0).stop 2024-08-19T01:40:38.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 msgr2=0x7fb8f01a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:38.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.802+0000 7fb8f7548640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f01a4050 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb8e802ae70 tx=0x7fb8e802f5d0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 shutdown_connections 2024-08-19T01:40:38.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb8dc0a2c00 0x7fb8dc0a50c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f0102c20 0x7fb8f01a4050 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb8f01022a0 0x7fb8f0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 --2- 172.21.15.103:0/4061931348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8f0100b50 0x7fb8f0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:38.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 >> 172.21.15.103:0/4061931348 conn(0x7fb8f00fc500 msgr2=0x7fb8f00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:38.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 shutdown_connections 2024-08-19T01:40:38.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:38.803+0000 7fb8f7548640 1 -- 172.21.15.103:0/4061931348 wait complete. 2024-08-19T01:40:39.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:40:39.829 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:40.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:39 smithi146 ceph-mon[22098]: pgmap v485: 97 pgs: 97 active+clean; 6.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:40:40.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:39 smithi103 ceph-mon[18436]: pgmap v485: 97 pgs: 97 active+clean; 6.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 -- 172.21.15.103:0/1365690739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904107fd0 msgr2=0x7fa90410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/1365690739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904107fd0 0x7fa90410e970 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009cd0 tx=0x7fa8fc02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 -- 172.21.15.103:0/1365690739 shutdown_connections 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/1365690739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904107fd0 0x7fa90410e970 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/1365690739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa904103410 0x7fa9041078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/1365690739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa904102a60 0x7fa904102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 -- 172.21.15.103:0/1365690739 >> 172.21.15.103:0/1365690739 conn(0x7fa9040fc500 msgr2=0x7fa9040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 -- 172.21.15.103:0/1365690739 shutdown_connections 2024-08-19T01:40:40.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.342+0000 7fa90bb1a640 1 -- 172.21.15.103:0/1365690739 wait complete. 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 Processor -- start 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 -- start start 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa904102a60 0x7fa9041006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904103410 0x7fa904100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa904107fd0 0x7fa904101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111c80 con 0x7fa904103410 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111b00 con 0x7fa904102a60 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90bb1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111e00 con 0x7fa904107fd0 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa9090b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904103410 0x7fa904100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa9098b6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa904102a60 0x7fa9041006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa9090b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904103410 0x7fa904100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39706/0 (socket says 172.21.15.103:39706) 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa9090b5640 1 -- 172.21.15.103:0/654584827 learned_addr learned my addr 172.21.15.103:0/654584827 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:40.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.343+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa904107fd0 0x7fa904101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa9090b5640 1 -- 172.21.15.103:0/654584827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa904107fd0 msgr2=0x7fa904101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa9090b5640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa904107fd0 0x7fa904101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa9090b5640 1 -- 172.21.15.103:0/654584827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa904102a60 msgr2=0x7fa9041006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa9090b5640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa904102a60 0x7fa9041006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa9090b5640 1 -- 172.21.15.103:0/654584827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9041a65e0 con 0x7fa904103410 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa9098b6640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa904102a60 0x7fa9041006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa9090b5640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904103410 0x7fa904100c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c8c0 tx=0x7fa8f400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa8faffd640 1 -- 172.21.15.103:0/654584827 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4007bb0 con 0x7fa904103410 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa8faffd640 1 -- 172.21.15.103:0/654584827 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8f4007d50 con 0x7fa904103410 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9041a68d0 con 0x7fa904103410 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa8faffd640 1 -- 172.21.15.103:0/654584827 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f40187c0 con 0x7fa904103410 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.344+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9041a6e10 con 0x7fa904103410 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.347+0000 7fa8faffd640 1 -- 172.21.15.103:0/654584827 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa8f400f450 con 0x7fa904103410 2024-08-19T01:40:40.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.347+0000 7fa90df18740 1 Processor -- start 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.347+0000 7fa90df18740 1 -- start start 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90df18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90df18740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f2eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90df18740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2eb1d0 0x56430f2ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90df18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56430f245a40 con 0x56430f2dfff0 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90df18740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56430f1dd9d0 con 0x56430f2e0910 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90df18740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56430f241a00 con 0x56430f2eb1d0 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2e03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39714/0 (socket says 172.21.15.103:39714) 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 -- 172.21.15.103:0/1133164288 learned_addr learned my addr 172.21.15.103:0/1133164288 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa9090b5640 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2eb1d0 0x56430f2ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa9098b6640 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f2eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa8faffd640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8f00a2b80 0x7fa8f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 -- 172.21.15.103:0/1133164288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2eb1d0 msgr2=0x56430f2ed5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2eb1d0 0x56430f2ed5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 -- 172.21.15.103:0/1133164288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 msgr2=0x56430f2eac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f2eac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa90a0b7640 1 -- 172.21.15.103:0/1133164288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56430f1e4650 con 0x56430f2dfff0 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa8faffd640 1 -- 172.21.15.103:0/654584827 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa8f40cefc0 con 0x7fa904103410 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa9090b5640 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2eb1d0 0x56430f2ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.348+0000 7fa9098b6640 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f2eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.349+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8f00a2b80 0x7fa8f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.349+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8f00a2b80 0x7fa8f00a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc035000 tx=0x7fa8fc041000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.349+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2e03d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009cd0 tx=0x7fa8fc031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10376fb18e2e7018 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.349+0000 7fa8effff640 1 -- 172.21.15.103:0/1133164288 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc042040 con 0x56430f2dfff0 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa8effff640 1 -- 172.21.15.103:0/1133164288 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8fc044070 con 0x56430f2dfff0 2024-08-19T01:40:40.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa8effff640 1 -- 172.21.15.103:0/1133164288 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc030d20 con 0x56430f2dfff0 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa90df18740 1 -- 172.21.15.103:0/1133164288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 msgr2=0x56430f2e03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa90df18740 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2e03d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009cd0 tx=0x7fa8fc031500 comp rx=0 tx=0).stop 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa90df18740 1 -- 172.21.15.103:0/1133164288 shutdown_connections 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa90df18740 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2eb1d0 0x56430f2ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa90df18740 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f2eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.350+0000 7fa90df18740 1 --2- 172.21.15.103:0/1133164288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2e03d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.351+0000 7fa90df18740 1 -- 172.21.15.103:0/1133164288 >> 172.21.15.103:0/1133164288 conn(0x56430f2dc580 msgr2=0x56430f2dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.351+0000 7fa90df18740 1 -- 172.21.15.103:0/1133164288 shutdown_connections 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.351+0000 7fa90df18740 1 -- 172.21.15.103:0/1133164288 wait complete. 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.351+0000 7fa90df18740 1 -- start start 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.351+0000 7fa90df18740 1 Processor -- start 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90df18740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 0x56430f2eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90df18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2e0910 0x56430f2e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90df18740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90df18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56430f2edda0 con 0x56430f2e0910 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90df18740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56430f200b20 con 0x56430f2eb1d0 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90df18740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56430f2edf20 con 0x56430f2dfff0 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90df18740 10 client.0 _ll_register_callbacks cb 0x56430f1febb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2e0910 0x56430f2e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2e0910 0x56430f2e7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39728/0 (socket says 172.21.15.103:39728) 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 -- 172.21.15.103:0/2959430475 learned_addr learned my addr 172.21.15.103:0/2959430475 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9098b6640 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 0x56430f2eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 -- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 msgr2=0x56430f2eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 0x56430f2eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 -- 172.21.15.103:0/2959430475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 msgr2=0x56430f2e86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 -- 172.21.15.103:0/2959430475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56430efb73c0 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa9090b5640 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2e0910 0x56430f2e7f50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c890 tx=0x7fa8f409b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8b4c8f7c72e725 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa8ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.352+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/2959430475 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f409bf50 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/2959430475 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa8f4097ec0 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc044070 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/2959430475 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8f409b850 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 10 client.16665 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56430f2f98a0 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/2959430475 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8fc044070 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/2959430475 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa8f40d1160 con 0x56430f2e0910 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 10 client.16665 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 10 client.16665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 20 client.16665 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:40.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f2fabf0 0x56430f2fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 4 client.16665 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56430f252fb0 con 0x56430f2fabf0 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.353+0000 7fa9098b6640 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f2fabf0 0x56430f2fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.354+0000 7fa8ed7fa640 10 client.16665 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.354+0000 7fa9098b6640 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f2fabf0 0x56430f2fcfe0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87ff1fbb5cfe3418 server_cookie=2ea9399495625596 in_seq=0 out_seq=0 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.354+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/2959430475 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56430f252fb0 con 0x56430f2fabf0 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.354+0000 7fa8ed7fa640 10 client.16665 handle_command_reply: tid=0 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 client.16665 shutdown 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2e0910 msgr2=0x56430f2e7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2e0910 0x56430f2e7f50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c890 tx=0x7fa8f409b390 comp rx=0 tx=0).stop 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 shutdown_connections 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f2fabf0 0x56430f2fcfe0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2e0910 0x56430f2e7f50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 --2- 172.21.15.103:0/2959430475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 0x56430f2eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa8ed7fa640 0 client.16665 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 >> 172.21.15.103:0/2959430475 conn(0x56430f2dc580 msgr2=0x56430f2e2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 shutdown_connections 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 1 -- 172.21.15.103:0/2959430475 wait complete. 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.355+0000 7fa90df18740 20 client.16665 trim_cache size 0 max 16384 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 Processor -- start 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 -- start start 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 0x56430f3028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f302df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2eb1d0 0x56430f2e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041af290 con 0x56430f2eb1d0 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041b0ef0 con 0x56430f2e0910 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.368+0000 7fa90df18740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041af960 con 0x56430f2dfff0 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2eb1d0 0x56430f2e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2eb1d0 0x56430f2e51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39738/0 (socket says 172.21.15.103:39738) 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 -- 172.21.15.103:0/2763678259 learned_addr learned my addr 172.21.15.103:0/2763678259 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa9090b5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f302df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 -- 172.21.15.103:0/2763678259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 msgr2=0x56430f3028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 0x56430f3028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 -- 172.21.15.103:0/2763678259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 msgr2=0x56430f302df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f302df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 -- 172.21.15.103:0/2763678259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56430f2fd350 con 0x56430f2eb1d0 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa9090b5640 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f302df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:40.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2eb1d0 0x56430f2e51d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc030930 tx=0x7fa8fc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdf7dc0f3f8478 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa8ef7fe640 1 -- 172.21.15.103:0/2763678259 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc042040 con 0x56430f2eb1d0 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa8ef7fe640 1 -- 172.21.15.103:0/2763678259 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8fc036040 con 0x56430f2eb1d0 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.369+0000 7fa8ef7fe640 1 -- 172.21.15.103:0/2763678259 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc004410 con 0x56430f2eb1d0 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 -- 172.21.15.103:0/2763678259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2eb1d0 msgr2=0x56430f2e51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2eb1d0 0x56430f2e51d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc030930 tx=0x7fa8fc02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 -- 172.21.15.103:0/2763678259 shutdown_connections 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2eb1d0 0x56430f2e51d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2e0910 0x56430f302df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 --2- 172.21.15.103:0/2763678259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2dfff0 0x56430f3028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 -- 172.21.15.103:0/2763678259 >> 172.21.15.103:0/2763678259 conn(0x56430f2dc580 msgr2=0x56430f2e3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 -- 172.21.15.103:0/2763678259 shutdown_connections 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.370+0000 7fa90df18740 1 -- 172.21.15.103:0/2763678259 wait complete. 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 -- start start 2024-08-19T01:40:40.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 Processor -- start 2024-08-19T01:40:40.385 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:39 smithi190 ceph-mon[22032]: pgmap v485: 97 pgs: 97 active+clean; 6.6 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:40:40.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2e0910 0x56430f2e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041b0760 con 0x56430f2dfff0 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041b0030 con 0x56430f2eb1d0 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041ad270 con 0x56430f2e0910 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.371+0000 7fa90df18740 10 client.0 _ll_register_callbacks cb 0x7fa8f4007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39744/0 (socket says 172.21.15.103:39744) 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 -- 172.21.15.103:0/575181218 learned_addr learned my addr 172.21.15.103:0/575181218 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa90a0b7640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9090b5640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2e0910 0x56430f2e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 -- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2e0910 msgr2=0x56430f2e51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2e0910 0x56430f2e51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 -- 172.21.15.103:0/575181218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 msgr2=0x56430f2e5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:40.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 -- 172.21.15.103:0/575181218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56430f2e5c80 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9090b5640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2e0910 0x56430f2e51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa9098b6640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2f35d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x56430f2fd820 tx=0x7fa900011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2971d28f39d2d3b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa8ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/575181218 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900010c90 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/575181218 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa900010e30 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc036040 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.372+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/575181218 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa900005020 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 10 client.16671 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56430f2ecad0 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/575181218 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8fc036040 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/575181218 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa900031420 con 0x56430f2dfff0 2024-08-19T01:40:40.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 10 client.16671 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 10 client.16671 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 20 client.16671 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f365ed0 0x56430f3682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 4 client.16671 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56430f24d4c0 con 0x56430f365ed0 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.373+0000 7fa9090b5640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f365ed0 0x56430f3682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa9090b5640 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f365ed0 0x56430f3682f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2a3662d6087e825 server_cookie=1d42a1db8e16bd43 in_seq=0 out_seq=0 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa8ed7fa640 10 client.16671 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa8ed7fa640 1 -- 172.21.15.103:0/575181218 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56430f24d4c0 con 0x56430f365ed0 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa8ed7fa640 10 client.16671 handle_command_reply: tid=0 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 client.16671 shutdown 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 msgr2=0x56430f2f35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2f35d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x56430f2fd820 tx=0x7fa900011bf0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 shutdown_connections 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56430f365ed0 0x56430f3682f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56430f2eb1d0 0x56430f2e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56430f2e0910 0x56430f2e51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 --2- 172.21.15.103:0/575181218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56430f2dfff0 0x56430f2f35d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.374+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 >> 172.21.15.103:0/575181218 conn(0x56430f2dc580 msgr2=0x56430f2e7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa8ed7fa640 0 client.16671 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 shutdown_connections 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90df18740 1 -- 172.21.15.103:0/575181218 wait complete. 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90df18740 20 client.16671 trim_cache size 0 max 16384 2024-08-19T01:40:40.391 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:40:40.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8f00a2b80 msgr2=0x7fa8f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8f00a2b80 0x7fa8f00a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc035000 tx=0x7fa8fc041000 comp rx=0 tx=0).stop 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904103410 msgr2=0x7fa904100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904103410 0x7fa904100c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c8c0 tx=0x7fa8f400cd80 comp rx=0 tx=0).stop 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 shutdown_connections 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8f00a2b80 0x7fa8f00a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa904107fd0 0x7fa904101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa904103410 0x7fa904100c00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 --2- 172.21.15.103:0/654584827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa904102a60 0x7fa9041006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 >> 172.21.15.103:0/654584827 conn(0x7fa9040fc500 msgr2=0x7fa904105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.375+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 shutdown_connections 2024-08-19T01:40:40.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:40.376+0000 7fa90bb1a640 1 -- 172.21.15.103:0/654584827 wait complete. 2024-08-19T01:40:41.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:40:41.450 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:41.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 -- 172.21.15.103:0/2843558823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc108010 msgr2=0x7f37bc10e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:41.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 --2- 172.21.15.103:0/2843558823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc108010 0x7f37bc10e980 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009cd0 tx=0x7f37b402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 -- 172.21.15.103:0/2843558823 shutdown_connections 2024-08-19T01:40:41.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 --2- 172.21.15.103:0/2843558823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc108010 0x7f37bc10e980 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 --2- 172.21.15.103:0/2843558823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37bc103490 0x7f37bc1078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 --2- 172.21.15.103:0/2843558823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37bc102ae0 0x7f37bc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 -- 172.21.15.103:0/2843558823 >> 172.21.15.103:0/2843558823 conn(0x7f37bc0fc500 msgr2=0x7f37bc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 -- 172.21.15.103:0/2843558823 shutdown_connections 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.953+0000 7f37c3dce640 1 -- 172.21.15.103:0/2843558823 wait complete. 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 Processor -- start 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 -- start start 2024-08-19T01:40:41.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc102ae0 0x7f37bc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37bc103490 0x7f37bc1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37bc108010 0x7f37bc19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111c90 con 0x7f37bc102ae0 2024-08-19T01:40:41.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111b10 con 0x7f37bc103490 2024-08-19T01:40:41.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c3dce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111e10 con 0x7f37bc108010 2024-08-19T01:40:41.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc102ae0 0x7f37bc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc102ae0 0x7f37bc198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54730/0 (socket says 172.21.15.103:54730) 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c1b6a640 1 -- 172.21.15.103:0/642880650 learned_addr learned my addr 172.21.15.103:0/642880650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.954+0000 7f37c1369640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37bc103490 0x7f37bc1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c236b640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37bc108010 0x7f37bc19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c1b6a640 1 -- 172.21.15.103:0/642880650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37bc108010 msgr2=0x7f37bc19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37bc108010 0x7f37bc19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c1b6a640 1 -- 172.21.15.103:0/642880650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37bc103490 msgr2=0x7f37bc1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37bc103490 0x7f37bc1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c1b6a640 1 -- 172.21.15.103:0/642880650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc19d220 con 0x7f37bc102ae0 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c236b640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37bc108010 0x7f37bc19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c1369640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37bc103490 0x7f37bc1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc102ae0 0x7f37bc198270 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d770 tx=0x7f37b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37b2ffd640 1 -- 172.21.15.103:0/642880650 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8017070 con 0x7f37bc102ae0 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37b2ffd640 1 -- 172.21.15.103:0/642880650 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b80054d0 con 0x7f37bc102ae0 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37b2ffd640 1 -- 172.21.15.103:0/642880650 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8004cf0 con 0x7f37bc102ae0 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc19d510 con 0x7f37bc102ae0 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.955+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a4df0 con 0x7f37bc102ae0 2024-08-19T01:40:41.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.957+0000 7f37b2ffd640 1 -- 172.21.15.103:0/642880650 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f37b8004e90 con 0x7f37bc102ae0 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 Processor -- start 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 -- start start 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37b2ffd640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37a80a2c00 0x7f37a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a48ff0 0x559467a493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a49910 0x559467a53c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594679aea40 con 0x559467a49910 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5594679469d0 con 0x559467a541d0 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37c61cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5594679aaa00 con 0x559467a48ff0 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.958+0000 7f37b2ffd640 1 -- 172.21.15.103:0/642880650 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f37b80cea00 con 0x7f37bc102ae0 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c236b640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37a80a2c00 0x7f37a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a49910 0x559467a53c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a49910 0x559467a53c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54740/0 (socket says 172.21.15.103:54740) 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 -- 172.21.15.103:0/3084001629 learned_addr learned my addr 172.21.15.103:0/3084001629 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a48ff0 0x559467a493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c236b640 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 -- 172.21.15.103:0/3084001629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a48ff0 msgr2=0x559467a493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a48ff0 0x559467a493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 -- 172.21.15.103:0/3084001629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 msgr2=0x559467a565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 -- 172.21.15.103:0/3084001629 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946794d650 con 0x559467a49910 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c236b640 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:41.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c236b640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37a80a2c00 0x7f37a80a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009ca0 tx=0x7f37b40093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37c1369640 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a49910 0x559467a53c90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00bfa0 tx=0x7f37ac010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb0374a4d3d96b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.959+0000 7f37a7fff640 1 -- 172.21.15.103:0/3084001629 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac011700 con 0x559467a49910 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37a7fff640 1 -- 172.21.15.103:0/3084001629 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37ac011d50 con 0x559467a49910 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37a7fff640 1 -- 172.21.15.103:0/3084001629 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac01cd60 con 0x559467a49910 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37c61cc740 1 -- 172.21.15.103:0/3084001629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a49910 msgr2=0x559467a53c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37c61cc740 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a49910 0x559467a53c90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00bfa0 tx=0x7f37ac010860 comp rx=0 tx=0).stop 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37c61cc740 1 -- 172.21.15.103:0/3084001629 shutdown_connections 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37c61cc740 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37c61cc740 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a49910 0x559467a53c90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37c61cc740 1 --2- 172.21.15.103:0/3084001629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a48ff0 0x559467a493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.960+0000 7f37c61cc740 1 -- 172.21.15.103:0/3084001629 >> 172.21.15.103:0/3084001629 conn(0x559467a45580 msgr2=0x559467a45960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.961+0000 7f37c61cc740 1 -- 172.21.15.103:0/3084001629 shutdown_connections 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.961+0000 7f37c61cc740 1 -- 172.21.15.103:0/3084001629 wait complete. 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.961+0000 7f37c61cc740 1 -- start start 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.961+0000 7f37c61cc740 1 Processor -- start 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a51060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a515a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a51ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559467a56da0 con 0x559467a48ff0 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559467969b20 con 0x559467a541d0 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559467a56f20 con 0x559467a49910 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 10 client.0 _ll_register_callbacks cb 0x559467967bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a51060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a51060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54750/0 (socket says 172.21.15.103:54750) 2024-08-19T01:40:41.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2569405125 learned_addr learned my addr 172.21.15.103:0/2569405125 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c236b640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a51ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1369640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2569405125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 msgr2=0x559467a515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 msgr2=0x559467a51ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a51ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2569405125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594677203c0 con 0x559467a48ff0 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1369640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a51060 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f37b8097c60 tx=0x7f37b809acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebc41d9b8200562 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37a57fa640 1 -- 172.21.15.103:0/2569405125 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8020040 con 0x559467a48ff0 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37a57fa640 1 -- 172.21.15.103:0/2569405125 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f37b8097900 con 0x559467a48ff0 2024-08-19T01:40:41.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.962+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ac011d50 con 0x559467a48ff0 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37a57fa640 1 -- 172.21.15.103:0/2569405125 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f37b809b290 con 0x559467a48ff0 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 10 client.16683 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559467a628c0 con 0x559467a48ff0 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37a57fa640 1 -- 172.21.15.103:0/2569405125 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37ac011d50 con 0x559467a48ff0 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37a57fa640 1 -- 172.21.15.103:0/2569405125 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f37b805a780 con 0x559467a48ff0 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 10 client.16683 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 20 client.16683 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467a63da0 0x559467a66240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 4 client.16683 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594679bbfb0 con 0x559467a63da0 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.963+0000 7f37c1369640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467a63da0 0x559467a66240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.964+0000 7f37a57fa640 10 client.16683 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:41.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.964+0000 7f37c1369640 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467a63da0 0x559467a66240 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=106b249712b22b24 server_cookie=dcf7e5a63a3008c7 in_seq=0 out_seq=0 2024-08-19T01:40:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37a57fa640 1 -- 172.21.15.103:0/2569405125 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594679bbfb0 con 0x559467a63da0 2024-08-19T01:40:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37a57fa640 10 client.16683 handle_command_reply: tid=0 2024-08-19T01:40:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 client.16683 shutdown 2024-08-19T01:40:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 msgr2=0x559467a51060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a51060 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f37b8097c60 tx=0x7f37b809acb0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 shutdown_connections 2024-08-19T01:40:41.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467a63da0 0x559467a66240 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a51ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2569405125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a51060 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 >> 172.21.15.103:0/2569405125 conn(0x559467a45580 msgr2=0x559467a4b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37a57fa640 0 client.16683 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:41.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 shutdown_connections 2024-08-19T01:40:41.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 1 -- 172.21.15.103:0/2569405125 wait complete. 2024-08-19T01:40:41.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.965+0000 7f37c61cc740 20 client.16683 trim_cache size 0 max 16384 2024-08-19T01:40:41.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.979+0000 7f37c61cc740 1 Processor -- start 2024-08-19T01:40:41.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.979+0000 7f37c61cc740 1 -- start start 2024-08-19T01:40:41.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a44aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a4ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a6bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c61cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1aaef0 con 0x559467a48ff0 2024-08-19T01:40:41.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c61cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1acb50 con 0x559467a541d0 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c61cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1ab5c0 con 0x559467a49910 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a44aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a44aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54752/0 (socket says 172.21.15.103:54752) 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2224701522 learned_addr learned my addr 172.21.15.103:0/2224701522 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c236b640 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a6bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1369640 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a4ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2224701522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 msgr2=0x559467a4ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a4ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2224701522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 msgr2=0x559467a6bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a6bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2224701522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559467a53b10 con 0x559467a48ff0 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.980+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a44aa0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009fd0 tx=0x7f37b80997a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0028b3f6e24c9e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:41.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.981+0000 7f37a77fe640 1 -- 172.21.15.103:0/2224701522 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b809f640 con 0x559467a48ff0 2024-08-19T01:40:41.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.981+0000 7f37a77fe640 1 -- 172.21.15.103:0/2224701522 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b8015020 con 0x559467a48ff0 2024-08-19T01:40:41.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.981+0000 7f37a77fe640 1 -- 172.21.15.103:0/2224701522 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80bb0b0 con 0x559467a48ff0 2024-08-19T01:40:41.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 -- 172.21.15.103:0/2224701522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 msgr2=0x559467a44aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:41.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a44aa0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009fd0 tx=0x7f37b80997a0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 -- 172.21.15.103:0/2224701522 shutdown_connections 2024-08-19T01:40:41.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a6bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a4ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2224701522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a44aa0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 -- 172.21.15.103:0/2224701522 >> 172.21.15.103:0/2224701522 conn(0x559467a45580 msgr2=0x559467a50150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 -- 172.21.15.103:0/2224701522 shutdown_connections 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 -- 172.21.15.103:0/2224701522 wait complete. 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 -- start start 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.982+0000 7f37c61cc740 1 Processor -- start 2024-08-19T01:40:41.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a57130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a57670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c61cc740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a57bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c61cc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1ac3c0 con 0x559467a48ff0 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c61cc740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1abc90 con 0x559467a541d0 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c61cc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1a8ed0 con 0x559467a49910 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c61cc740 10 client.0 _ll_register_callbacks cb 0x7f37b800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a57130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c1b6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a57130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54762/0 (socket says 172.21.15.103:54762) 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2114221216 learned_addr learned my addr 172.21.15.103:0/2114221216 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c236b640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a57bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c1369640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a57670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2114221216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 msgr2=0x559467a57670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.983+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a57670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 msgr2=0x559467a57bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a57bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c1b6a640 1 -- 172.21.15.103:0/2114221216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559467acccd0 con 0x559467a48ff0 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c1369640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a57670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c236b640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a57bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c1b6a640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a57130 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f37b809af90 tx=0x7f37b80bb340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35163711b6cea80 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37a57fa640 1 -- 172.21.15.103:0/2114221216 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b809add0 con 0x559467a48ff0 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37a57fa640 1 -- 172.21.15.103:0/2114221216 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f37b8095e00 con 0x559467a48ff0 2024-08-19T01:40:41.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b8015020 con 0x559467a48ff0 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37a57fa640 1 -- 172.21.15.103:0/2114221216 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f37b80a3f00 con 0x559467a48ff0 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c61cc740 10 client.16689 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559467acdfb0 con 0x559467a48ff0 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.984+0000 7f37a57fa640 1 -- 172.21.15.103:0/2114221216 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b807c070 con 0x559467a48ff0 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37a57fa640 1 -- 172.21.15.103:0/2114221216 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f37b8096fe0 con 0x559467a48ff0 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c61cc740 10 client.16689 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c61cc740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c61cc740 20 client.16689 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467ace4d0 0x559467ad08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c61cc740 4 client.16689 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594679b64c0 con 0x559467ace4d0 2024-08-19T01:40:42.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c1369640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467ace4d0 0x559467ad08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.985+0000 7f37c1369640 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467ace4d0 0x559467ad08c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a427f5ade6626f5 server_cookie=100696f001289c4 in_seq=0 out_seq=0 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37a57fa640 10 client.16689 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37a57fa640 1 -- 172.21.15.103:0/2114221216 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594679b64c0 con 0x559467ace4d0 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37a57fa640 10 client.16689 handle_command_reply: tid=0 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 client.16689 shutdown 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 msgr2=0x559467a57130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a57130 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f37b809af90 tx=0x7f37b80bb340 comp rx=0 tx=0).stop 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 shutdown_connections 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x559467ace4d0 0x559467ad08c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559467a541d0 0x559467a57bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559467a49910 0x559467a57670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 --2- 172.21.15.103:0/2114221216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559467a48ff0 0x559467a57130 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 >> 172.21.15.103:0/2114221216 conn(0x559467a45580 msgr2=0x559467a526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:42.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37a57fa640 0 client.16689 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:42.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 shutdown_connections 2024-08-19T01:40:42.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 1 -- 172.21.15.103:0/2114221216 wait complete. 2024-08-19T01:40:42.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.986+0000 7f37c61cc740 20 client.16689 trim_cache size 0 max 16384 2024-08-19T01:40:42.002 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:40:42.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37a80a2c00 msgr2=0x7f37a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:42.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37a80a2c00 0x7f37a80a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009ca0 tx=0x7f37b40093f0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc102ae0 msgr2=0x7f37bc198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc102ae0 0x7f37bc198270 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f37b800d770 tx=0x7f37b800dc30 comp rx=0 tx=0).stop 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 shutdown_connections 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f37a80a2c00 0x7f37a80a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37bc108010 0x7f37bc19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37bc103490 0x7f37bc1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 --2- 172.21.15.103:0/642880650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc102ae0 0x7f37bc198270 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 >> 172.21.15.103:0/642880650 conn(0x7f37bc0fc500 msgr2=0x7f37bc105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 shutdown_connections 2024-08-19T01:40:42.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:41.987+0000 7f37c3dce640 1 -- 172.21.15.103:0/642880650 wait complete. 2024-08-19T01:40:42.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:41 smithi146 ceph-mon[22098]: pgmap v486: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:40:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:41 smithi103 ceph-mon[18436]: pgmap v486: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:40:42.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:41 smithi190 ceph-mon[22032]: pgmap v486: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:40:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.487+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3049819537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 msgr2=0x7f3df41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.487+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3049819537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df41099c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009ca0 tx=0x7f3dec02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3049819537 shutdown_connections 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3049819537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df41099c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3049819537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df4104c80 0x7f3df4107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3049819537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df4102350 0x7f3df4104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3049819537 >> 172.21.15.103:0/3049819537 conn(0x7f3df40fbf10 msgr2=0x7f3df40fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3049819537 shutdown_connections 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3049819537 wait complete. 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 Processor -- start 2024-08-19T01:40:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- start start 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df4102350 0x7f3df4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df4104c80 0x7f3df4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410d430 con 0x7f3df41075b0 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410d2b0 con 0x7f3df4104c80 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.488+0000 7f3dfc32c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410d5b0 con 0x7f3df4102350 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3dfa8c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3df98c7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df4104c80 0x7f3df4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3dfa8c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54778/0 (socket says 172.21.15.103:54778) 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3dfa8c9640 1 -- 172.21.15.103:0/3489180108 learned_addr learned my addr 172.21.15.103:0/3489180108 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df4102350 0x7f3df4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3df98c7640 1 -- 172.21.15.103:0/3489180108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df4102350 msgr2=0x7f3df4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3df98c7640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df4102350 0x7f3df4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3df98c7640 1 -- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 msgr2=0x7f3df419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3df98c7640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3df98c7640 1 -- 172.21.15.103:0/3489180108 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df419f6c0 con 0x7f3df4104c80 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3df98c7640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df4104c80 0x7f3df4197fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3df406a0d0 tx=0x7f3de400dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3deb7fe640 1 -- 172.21.15.103:0/3489180108 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4017070 con 0x7f3df4104c80 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df419f9b0 con 0x7f3df4104c80 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3deb7fe640 1 -- 172.21.15.103:0/3489180108 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3de40099d0 con 0x7f3df4104c80 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3deb7fe640 1 -- 172.21.15.103:0/3489180108 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de40047a0 con 0x7f3df4104c80 2024-08-19T01:40:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.489+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df419ff00 con 0x7f3df4104c80 2024-08-19T01:40:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.492+0000 7f3dff1cb740 1 Processor -- start 2024-08-19T01:40:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.492+0000 7f3deb7fe640 1 -- 172.21.15.103:0/3489180108 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3de4009b80 con 0x7f3df4104c80 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.492+0000 7f3dff1cb740 1 -- start start 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea54fc0 0x55b89ea59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea5a050 0x55b89ea5c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b89e9b79d0 con 0x55b89ea54680 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b89e95f820 con 0x55b89ea5a050 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b89e9b3990 con 0x55b89ea54fc0 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea54a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54794/0 (socket says 172.21.15.103:54794) 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 -- 172.21.15.103:0/2039686297 learned_addr learned my addr 172.21.15.103:0/2039686297 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea54fc0 0x55b89ea59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3df98c7640 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea5a050 0x55b89ea5c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3deb7fe640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3de00a2c30 0x7f3de00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 -- 172.21.15.103:0/2039686297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea54fc0 msgr2=0x55b89ea59a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea54fc0 0x55b89ea59a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 -- 172.21.15.103:0/2039686297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea5a050 msgr2=0x55b89ea5c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea5a050 0x55b89ea5c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 -- 172.21.15.103:0/2039686297 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b89e6cb210 con 0x55b89ea54680 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea54fc0 0x55b89ea59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3deb7fe640 1 -- 172.21.15.103:0/3489180108 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3de40cec10 con 0x7f3df4104c80 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.493+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea54a80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3dec02ade0 tx=0x7f3dec005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1baa7d5a3da2bd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.494+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3de00a2c30 0x7f3de00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.494+0000 7f3de8bfe640 1 -- 172.21.15.103:0/2039686297 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec04a040 con 0x55b89ea54680 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.494+0000 7f3de8bfe640 1 -- 172.21.15.103:0/2039686297 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dec036030 con 0x55b89ea54680 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.494+0000 7f3de8bfe640 1 -- 172.21.15.103:0/2039686297 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0305c0 con 0x55b89ea54680 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.494+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2039686297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 msgr2=0x55b89ea54a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.494+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3de00a2c30 0x7f3de00a50f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3dec045000 tx=0x7f3dec030d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.494+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea54a80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3dec02ade0 tx=0x7f3dec005fa0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2039686297 shutdown_connections 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea5a050 0x55b89ea5c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea54fc0 0x55b89ea59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2039686297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea54a80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2039686297 >> 172.21.15.103:0/2039686297 conn(0x55b89ea4fcf0 msgr2=0x55b89ea500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2039686297 shutdown_connections 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2039686297 wait complete. 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- start start 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 Processor -- start 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b89ea5ce40 con 0x55b89ea54680 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b89e973520 con 0x55b89ea54fc0 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b89ea5cfc0 con 0x55b89ea5a050 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dff1cb740 10 client.0 _ll_register_callbacks cb 0x55b89e6caa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dfa0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.495+0000 7f3dfa0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea5e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54800/0 (socket says 172.21.15.103:54800) 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/2384283656 learned_addr learned my addr 172.21.15.103:0/2384283656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3df98c7640 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/2384283656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 msgr2=0x55b89ea65b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea65b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 msgr2=0x55b89ea5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/2384283656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b89e797620 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3df98c7640 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea5e5c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3df000bad0 tx=0x7f3df000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966c4cd3e6a1171f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dddffb640 1 -- 172.21.15.103:0/2384283656 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0003e00 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dddffb640 1 -- 172.21.15.103:0/2384283656 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3df0011440 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dec036030 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dddffb640 1 -- 172.21.15.103:0/2384283656 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3df0011640 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dff1cb740 10 client.16701 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.496+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b89ea68a50 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dddffb640 1 -- 172.21.15.103:0/2384283656 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dec036030 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dddffb640 1 -- 172.21.15.103:0/2384283656 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3df00268b0 con 0x55b89ea54680 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dff1cb740 10 client.16701 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dff1cb740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dff1cb740 20 client.16701 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ea6a260 0x55b89ea6c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dff1cb740 4 client.16701 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b89e9c5100 con 0x55b89ea6a260 2024-08-19T01:40:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.497+0000 7f3df98c7640 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ea6a260 0x55b89ea6c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.498+0000 7f3dddffb640 10 client.16701 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.498+0000 7f3df98c7640 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ea6a260 0x55b89ea6c700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82bb9d00cb94f549 server_cookie=8fa68883be011f67 in_seq=0 out_seq=0 2024-08-19T01:40:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dddffb640 1 -- 172.21.15.103:0/2384283656 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b89e9c5100 con 0x55b89ea6a260 2024-08-19T01:40:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dddffb640 10 client.16701 handle_command_reply: tid=0 2024-08-19T01:40:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 client.16701 shutdown 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 msgr2=0x55b89ea5e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea5e5c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3df000bad0 tx=0x7f3df000bf90 comp rx=0 tx=0).stop 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 shutdown_connections 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ea6a260 0x55b89ea6c700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea65b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/2384283656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea5e5c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 >> 172.21.15.103:0/2384283656 conn(0x55b89ea4fcf0 msgr2=0x55b89ea51500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dddffb640 0 client.16701 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 shutdown_connections 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 1 -- 172.21.15.103:0/2384283656 wait complete. 2024-08-19T01:40:43.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.499+0000 7f3dff1cb740 20 client.16701 trim_cache size 0 max 16384 2024-08-19T01:40:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 Processor -- start 2024-08-19T01:40:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 -- start start 2024-08-19T01:40:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54680 0x55b89ea4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54fc0 0x55b89ea65ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41a8ad0 con 0x55b89ea54fc0 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41aa730 con 0x55b89ea54680 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41a91a0 con 0x55b89ea5a050 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3df98c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54fc0 0x55b89ea65ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3df98c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54fc0 0x55b89ea65ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54812/0 (socket says 172.21.15.103:54812) 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3df98c7640 1 -- 172.21.15.103:0/333644848 learned_addr learned my addr 172.21.15.103:0/333644848 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.521+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54680 0x55b89ea4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3df98c7640 1 -- 172.21.15.103:0/333644848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 msgr2=0x55b89ea500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3df98c7640 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3df98c7640 1 -- 172.21.15.103:0/333644848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54680 msgr2=0x55b89ea4ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3df98c7640 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54680 0x55b89ea4ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3df98c7640 1 -- 172.21.15.103:0/333644848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b89ea554a0 con 0x55b89ea54fc0 2024-08-19T01:40:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3dfa8c9640 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54680 0x55b89ea4ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3df98c7640 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54fc0 0x55b89ea65ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3de409b430 tx=0x7f3de40d26f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1001db3193de114 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3ddffff640 1 -- 172.21.15.103:0/333644848 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de409afe0 con 0x55b89ea54fc0 2024-08-19T01:40:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3ddffff640 1 -- 172.21.15.103:0/333644848 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3de409bba0 con 0x55b89ea54fc0 2024-08-19T01:40:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.522+0000 7f3ddffff640 1 -- 172.21.15.103:0/333644848 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de409ae00 con 0x55b89ea54fc0 2024-08-19T01:40:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 -- 172.21.15.103:0/333644848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54fc0 msgr2=0x55b89ea65ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54fc0 0x55b89ea65ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3de409b430 tx=0x7f3de40d26f0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 -- 172.21.15.103:0/333644848 shutdown_connections 2024-08-19T01:40:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ea500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54fc0 0x55b89ea65ad0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/333644848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54680 0x55b89ea4ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 -- 172.21.15.103:0/333644848 >> 172.21.15.103:0/333644848 conn(0x55b89ea4fcf0 msgr2=0x55b89ea4e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 -- 172.21.15.103:0/333644848 shutdown_connections 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 -- 172.21.15.103:0/333644848 wait complete. 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 -- start start 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.523+0000 7f3dff1cb740 1 Processor -- start 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea62950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea62e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ead2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41a9fa0 con 0x55b89ea54680 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41a9870 con 0x55b89ea54fc0 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41a6ab0 con 0x55b89ea5a050 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 10 client.0 _ll_register_callbacks cb 0x7f3df0004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea62950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea62950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54824/0 (socket says 172.21.15.103:54824) 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/3350277097 learned_addr learned my addr 172.21.15.103:0/3350277097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3df98c7640 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea62e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/3350277097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 msgr2=0x55b89ead2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ead2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 msgr2=0x55b89ea62e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea62e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 -- 172.21.15.103:0/3350277097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b89ead34d0 con 0x55b89ea54680 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dfa0c8640 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea62950 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3df0026f10 tx=0x7f3df0004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c30b9a8d518ed7c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dddffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dddffb640 1 -- 172.21.15.103:0/3350277097 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0033060 con 0x55b89ea54680 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dddffb640 1 -- 172.21.15.103:0/3350277097 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3df0017040 con 0x55b89ea54680 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de409bba0 con 0x55b89ea54680 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dddffb640 1 -- 172.21.15.103:0/3350277097 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3df0005030 con 0x55b89ea54680 2024-08-19T01:40:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 10 client.16707 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.524+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b89ead4730 con 0x55b89ea54680 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dddffb640 1 -- 172.21.15.103:0/3350277097 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3de409bba0 con 0x55b89ea54680 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dddffb640 1 -- 172.21.15.103:0/3350277097 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3df001b7a0 con 0x55b89ea54680 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dff1cb740 10 client.16707 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dff1cb740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dff1cb740 20 client.16707 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ead5140 0x55b89ead7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dff1cb740 4 client.16707 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b89e9bf450 con 0x55b89ead5140 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.525+0000 7f3df98c7640 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ead5140 0x55b89ead7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.526+0000 7f3dddffb640 10 client.16707 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.526+0000 7f3df98c7640 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ead5140 0x55b89ead7560 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91a808d9175630e server_cookie=8cf076e34a5ae2ba in_seq=0 out_seq=0 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dddffb640 1 -- 172.21.15.103:0/3350277097 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b89e9bf450 con 0x55b89ead5140 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dddffb640 10 client.16707 handle_command_reply: tid=0 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 client.16707 shutdown 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 msgr2=0x55b89ea62950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea62950 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3df0026f10 tx=0x7f3df0004440 comp rx=0 tx=0).stop 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 shutdown_connections 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b89ead5140 0x55b89ead7560 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dddffb640 0 client.16707 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b89ea5a050 0x55b89ead2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b89ea54fc0 0x55b89ea62e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 --2- 172.21.15.103:0/3350277097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b89ea54680 0x55b89ea62950 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 >> 172.21.15.103:0/3350277097 conn(0x55b89ea4fcf0 msgr2=0x55b89ea4eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 shutdown_connections 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 1 -- 172.21.15.103:0/3350277097 wait complete. 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.533+0000 7f3dff1cb740 20 client.16707 trim_cache size 0 max 16384 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:40:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:40:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:40:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3de00a2c30 msgr2=0x7f3de00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3de00a2c30 0x7f3de00a50f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3dec045000 tx=0x7f3dec030d90 comp rx=0 tx=0).stop 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df4104c80 msgr2=0x7f3df4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df4104c80 0x7f3df4197fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3df406a0d0 tx=0x7f3de400dbb0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 shutdown_connections 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f3de00a2c30 0x7f3de00a50f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df41075b0 0x7f3df419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3df4104c80 0x7f3df4197fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 --2- 172.21.15.103:0/3489180108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3df4102350 0x7f3df4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 >> 172.21.15.103:0/3489180108 conn(0x7f3df40fbf10 msgr2=0x7f3df4102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 shutdown_connections 2024-08-19T01:40:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:43.534+0000 7f3dfc32c640 1 -- 172.21.15.103:0/3489180108 wait complete. 2024-08-19T01:40:43.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:40:43.880 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:40:44.160 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:44.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:43 smithi146 ceph-mon[22098]: pgmap v487: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:40:44.284 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:43 smithi103 ceph-mon[18436]: pgmap v487: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:40:44.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:43 smithi190 ceph-mon[22032]: pgmap v487: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 -- 172.21.15.103:0/3498453181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 msgr2=0x7f045c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 --2- 172.21.15.103:0/3498453181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c10d770 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0454009ca0 tx=0x7f045402a950 comp rx=0 tx=0).stop 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 -- 172.21.15.103:0/3498453181 shutdown_connections 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 --2- 172.21.15.103:0/3498453181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c10d770 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 --2- 172.21.15.103:0/3498453181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c1022a0 0x7f045c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 --2- 172.21.15.103:0/3498453181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045c100b50 0x7f045c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 -- 172.21.15.103:0/3498453181 >> 172.21.15.103:0/3498453181 conn(0x7f045c0fc500 msgr2=0x7f045c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 -- 172.21.15.103:0/3498453181 shutdown_connections 2024-08-19T01:40:44.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.758+0000 7f046306c640 1 -- 172.21.15.103:0/3498453181 wait complete. 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 Processor -- start 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 -- start start 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c100b50 0x7f045c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045c1022a0 0x7f045c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f045c110b60 con 0x7f045c102c20 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1109e0 con 0x7f045c1022a0 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f046306c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f045c110ce0 con 0x7f045c100b50 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f0461609640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f0461609640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54846/0 (socket says 172.21.15.103:54846) 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f0461609640 1 -- 172.21.15.103:0/3607051111 learned_addr learned my addr 172.21.15.103:0/3607051111 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f045bfff640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045c1022a0 0x7f045c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.759+0000 7f0460e08640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c100b50 0x7f045c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0461609640 1 -- 172.21.15.103:0/3607051111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c100b50 msgr2=0x7f045c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0461609640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c100b50 0x7f045c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0461609640 1 -- 172.21.15.103:0/3607051111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045c1022a0 msgr2=0x7f045c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0461609640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045c1022a0 0x7f045c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0461609640 1 -- 172.21.15.103:0/3607051111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c19bee0 con 0x7f045c102c20 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0460e08640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c100b50 0x7f045c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0461609640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c19b800 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0454004af0 tx=0x7f045402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0459ffb640 1 -- 172.21.15.103:0/3607051111 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454034040 con 0x7f045c102c20 2024-08-19T01:40:44.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c19c170 con 0x7f045c102c20 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0459ffb640 1 -- 172.21.15.103:0/3607051111 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04540308f0 con 0x7f045c102c20 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f0459ffb640 1 -- 172.21.15.103:0/3607051111 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04540046a0 con 0x7f045c102c20 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.760+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c19c640 con 0x7f045c102c20 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.763+0000 7f0459ffb640 1 -- 172.21.15.103:0/3607051111 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0454037020 con 0x7f045c102c20 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.763+0000 7f046546a740 1 Processor -- start 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.763+0000 7f046546a740 1 -- start start 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f046546a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f046546a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b953ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f046546a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b9543e0 0x55614b9567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f046546a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55614b90ce40 con 0x55614b949b20 2024-08-19T01:40:44.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f046546a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55614b8ae7f0 con 0x55614b949200 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f046546a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55614b855fb0 con 0x55614b9543e0 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0459ffb640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f044c0a6f90 0x7f044c0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b953ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b953ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54852/0 (socket says 172.21.15.103:54852) 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 -- 172.21.15.103:0/3714606648 learned_addr learned my addr 172.21.15.103:0/3714606648 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f045bfff640 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b9543e0 0x55614b9567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0461609640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f044c0a6f90 0x7f044c0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f045bfff640 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0459ffb640 1 -- 172.21.15.103:0/3607051111 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f04540efb80 con 0x7f045c102c20 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 -- 172.21.15.103:0/3714606648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b9543e0 msgr2=0x55614b9567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b9543e0 0x55614b9567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 -- 172.21.15.103:0/3714606648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 msgr2=0x55614b9495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f0460e08640 1 -- 172.21.15.103:0/3714606648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614b6206a0 con 0x55614b949b20 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.764+0000 7f045bfff640 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.765+0000 7f0460e08640 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b953ea0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0450009d50 tx=0x7f04500117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12b2f77afc8918d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.765+0000 7f0461609640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f044c0a6f90 0x7f044c0a9450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f04540bc4c0 tx=0x7f0454087220 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.765+0000 7f0446ffd640 1 -- 172.21.15.103:0/3714606648 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045000f640 con 0x55614b949b20 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.765+0000 7f0446ffd640 1 -- 172.21.15.103:0/3714606648 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0450009e40 con 0x55614b949b20 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.765+0000 7f0446ffd640 1 -- 172.21.15.103:0/3714606648 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450022b50 con 0x55614b949b20 2024-08-19T01:40:44.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 -- 172.21.15.103:0/3714606648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 msgr2=0x55614b953ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b953ea0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0450009d50 tx=0x7f04500117b0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 -- 172.21.15.103:0/3714606648 shutdown_connections 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b9543e0 0x55614b9567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b953ea0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 --2- 172.21.15.103:0/3714606648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 -- 172.21.15.103:0/3714606648 >> 172.21.15.103:0/3714606648 conn(0x55614b945790 msgr2=0x55614b945b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 -- 172.21.15.103:0/3714606648 shutdown_connections 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 -- 172.21.15.103:0/3714606648 wait complete. 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 -- start start 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.766+0000 7f046546a740 1 Processor -- start 2024-08-19T01:40:44.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f046546a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b9512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f046546a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b9517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f046546a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b951d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f046546a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55614b869cb0 con 0x55614b949b20 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f046546a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55614b8aa7b0 con 0x55614b9543e0 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f046546a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55614b956fb0 con 0x55614b949200 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f046546a740 10 client.0 _ll_register_callbacks cb 0x55614b5bdba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b9517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b9517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54856/0 (socket says 172.21.15.103:54856) 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 -- 172.21.15.103:0/1767457249 learned_addr learned my addr 172.21.15.103:0/1767457249 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:44.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f045bfff640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b9512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f045bfff640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b951d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 -- 172.21.15.103:0/1767457249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 msgr2=0x55614b9512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b9512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 -- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 msgr2=0x55614b951d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b951d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 -- 172.21.15.103:0/1767457249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614b868bc0 con 0x55614b949b20 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f045bfff640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b951d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f0460e08640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b9517f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0450009a70 tx=0x7f0450009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25b57574c1cd087 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f043bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.767+0000 7f043bfff640 1 -- 172.21.15.103:0/1767457249 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045002f020 con 0x55614b949b20 2024-08-19T01:40:44.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f043bfff640 1 -- 172.21.15.103:0/1767457249 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0450022d40 con 0x55614b949b20 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55614b8bbd60 con 0x55614b949b20 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f043bfff640 1 -- 172.21.15.103:0/1767457249 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f04500035f0 con 0x55614b949b20 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 10 client.16719 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55614b962e50 con 0x55614b949b20 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f043bfff640 1 -- 172.21.15.103:0/1767457249 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55614b8bbd60 con 0x55614b949b20 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f043bfff640 1 -- 172.21.15.103:0/1767457249 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f045002b710 con 0x55614b949b20 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 10 client.16719 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:44.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 20 client.16719 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b966bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 4 client.16719 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.768+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55614b8b6270 con 0x55614b9647b0 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.769+0000 7f045bfff640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b966bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.769+0000 7f045bfff640 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b966bf0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c223d6c9754848ba server_cookie=20fbbeab5b57438f in_seq=0 out_seq=0 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.769+0000 7f043bfff640 10 client.16719 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f043bfff640 1 -- 172.21.15.103:0/1767457249 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55614b8b6270 con 0x55614b9647b0 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f043bfff640 10 client.16719 handle_command_reply: tid=0 2024-08-19T01:40:44.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 client.16719 shutdown 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 msgr2=0x55614b9517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b9517f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0450009a70 tx=0x7f0450009aa0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 shutdown_connections 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b966bf0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b951d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f043bfff640 0 client.16719 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b9517f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 --2- 172.21.15.103:0/1767457249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b9512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 >> 172.21.15.103:0/1767457249 conn(0x55614b945790 msgr2=0x55614b9448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 shutdown_connections 2024-08-19T01:40:44.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 1 -- 172.21.15.103:0/1767457249 wait complete. 2024-08-19T01:40:44.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.770+0000 7f046546a740 20 client.16719 trim_cache size 0 max 16384 2024-08-19T01:40:44.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.787+0000 7f046546a740 1 Processor -- start 2024-08-19T01:40:44.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f046546a740 1 -- start start 2024-08-19T01:40:44.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f046546a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f046546a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949b20 0x55614b96c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f046546a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b9543e0 0x55614b96ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f046546a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1aad90 con 0x55614b9543e0 2024-08-19T01:40:44.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f046546a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1ac9f0 con 0x55614b949200 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f046546a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1ab460 con 0x55614b949b20 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f045bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b9543e0 0x55614b96ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f045bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b9543e0 0x55614b96ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54860/0 (socket says 172.21.15.103:54860) 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f045bfff640 1 -- 172.21.15.103:0/3260132642 learned_addr learned my addr 172.21.15.103:0/3260132642 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f0460e08640 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949b20 0x55614b96c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f045bfff640 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f0460e08640 1 -- 172.21.15.103:0/3260132642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 msgr2=0x55614b9502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f0460e08640 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f0460e08640 1 -- 172.21.15.103:0/3260132642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b9543e0 msgr2=0x55614b96ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f0460e08640 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b9543e0 0x55614b96ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.788+0000 7f0460e08640 1 -- 172.21.15.103:0/3260132642 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614b949950 con 0x55614b949b20 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.789+0000 7f0460e08640 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949b20 0x55614b96c4c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f045002bcf0 tx=0x7f04500117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00684a777c9406 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.789+0000 7f04467fc640 1 -- 172.21.15.103:0/3260132642 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450043070 con 0x55614b949b20 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.789+0000 7f04467fc640 1 -- 172.21.15.103:0/3260132642 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f045002b440 con 0x55614b949b20 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.789+0000 7f04467fc640 1 -- 172.21.15.103:0/3260132642 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045002f040 con 0x55614b949b20 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 -- 172.21.15.103:0/3260132642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949b20 msgr2=0x55614b96c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949b20 0x55614b96c4c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f045002bcf0 tx=0x7f04500117b0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 -- 172.21.15.103:0/3260132642 shutdown_connections 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b9543e0 0x55614b96ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949b20 0x55614b96c4c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 --2- 172.21.15.103:0/3260132642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b949200 0x55614b9502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 -- 172.21.15.103:0/3260132642 >> 172.21.15.103:0/3260132642 conn(0x55614b945790 msgr2=0x55614b94fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 -- 172.21.15.103:0/3260132642 shutdown_connections 2024-08-19T01:40:44.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 -- 172.21.15.103:0/3260132642 wait complete. 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 -- start start 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.790+0000 7f046546a740 1 Processor -- start 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f046546a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b95c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f046546a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b94e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f046546a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b94eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f0460e08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b94e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f0460e08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b94e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54866/0 (socket says 172.21.15.103:54866) 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f045bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b95c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f045bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b95c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:59330/0 (socket says 172.21.15.103:59330) 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f0460e08640 1 -- 172.21.15.103:0/97273219 learned_addr learned my addr 172.21.15.103:0/97273219 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1ac260 con 0x55614b949b20 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1abb30 con 0x55614b9543e0 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1a8d90 con 0x55614b949200 2024-08-19T01:40:44.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f046546a740 10 client.0 _ll_register_callbacks cb 0x7f044800a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.791+0000 7f045bfff640 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b94eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f0460e08640 1 -- 172.21.15.103:0/97273219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 msgr2=0x55614b95c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f0460e08640 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b95c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f0460e08640 1 -- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 msgr2=0x55614b94eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f0460e08640 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b94eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f0460e08640 1 -- 172.21.15.103:0/97273219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614b94f0b0 con 0x55614b949b20 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f0460e08640 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b94e630 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0450011cb0 tx=0x7f0450009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9093314920e54 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f043bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f043bfff640 1 -- 172.21.15.103:0/97273219 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450043070 con 0x55614b949b20 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f043bfff640 1 -- 172.21.15.103:0/97273219 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f045002b5f0 con 0x55614b949b20 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045002b440 con 0x55614b949b20 2024-08-19T01:40:44.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f043bfff640 1 -- 172.21.15.103:0/97273219 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f045002f040 con 0x55614b949b20 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f046546a740 10 client.16725 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.792+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55614b9ce490 con 0x55614b949b20 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f043bfff640 1 -- 172.21.15.103:0/97273219 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f045004f070 con 0x55614b949b20 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f043bfff640 1 -- 172.21.15.103:0/97273219 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f045003d030 con 0x55614b949b20 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f046546a740 10 client.16725 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f046546a740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f046546a740 20 client.16725 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f046546a740 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b9d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:44.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f046546a740 4 client.16725 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:40:44.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55614b833110 con 0x55614b9647b0 2024-08-19T01:40:44.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.793+0000 7f045bfff640 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b9d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:44.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.794+0000 7f045bfff640 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b9d0b60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea041648bc994ca2 server_cookie=75a25448920f375d in_seq=0 out_seq=0 2024-08-19T01:40:44.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.794+0000 7f043bfff640 10 client.16725 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:44.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f043bfff640 1 -- 172.21.15.103:0/97273219 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55614b833110 con 0x55614b9647b0 2024-08-19T01:40:44.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f043bfff640 10 client.16725 handle_command_reply: tid=0 2024-08-19T01:40:44.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 client.16725 shutdown 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 msgr2=0x55614b94e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b94e630 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0450011cb0 tx=0x7f0450009d10 comp rx=0 tx=0).stop 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 shutdown_connections 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55614b9647b0 0x55614b9d0b60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55614b9543e0 0x55614b94eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614b949b20 0x55614b94e630 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 --2- 172.21.15.103:0/97273219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55614b949200 0x55614b95c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 >> 172.21.15.103:0/97273219 conn(0x55614b945790 msgr2=0x55614b9503c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f043bfff640 0 client.16725 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 shutdown_connections 2024-08-19T01:40:44.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.795+0000 7f046546a740 1 -- 172.21.15.103:0/97273219 wait complete. 2024-08-19T01:40:44.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046546a740 20 client.16725 trim_cache size 0 max 16384 2024-08-19T01:40:44.802 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:40:44.802 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:40:44.802 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "71406eaf-e402-4609-914e-4fc7b7ebe8f5", 2024-08-19T01:40:44.802 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:40:44.802 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:40:44.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f044c0a6f90 msgr2=0x7f044c0a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f044c0a6f90 0x7f044c0a9450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f04540bc4c0 tx=0x7f0454087220 comp rx=0 tx=0).stop 2024-08-19T01:40:44.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 msgr2=0x7f045c19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c19b800 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0454004af0 tx=0x7f045402f5d0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 shutdown_connections 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f044c0a6f90 0x7f044c0a9450 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f045c102c20 0x7f045c19b800 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f045c1022a0 0x7f045c197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 --2- 172.21.15.103:0/3607051111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c100b50 0x7f045c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.796+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 >> 172.21.15.103:0/3607051111 conn(0x7f045c0fc500 msgr2=0x7f045c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.797+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 shutdown_connections 2024-08-19T01:40:44.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:44.797+0000 7f046306c640 1 -- 172.21.15.103:0/3607051111 wait complete. 2024-08-19T01:40:44.950 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:44 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:44.950 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:44 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:45.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:44 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:45.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:44 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:45.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:44 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:45.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:44 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:45.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:40:46.113 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:45 smithi103 ceph-mon[18436]: pgmap v488: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:40:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:40:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:46.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:40:46.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:45 smithi190 ceph-mon[22032]: pgmap v488: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:40:46.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:40:46.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:45 smithi146 ceph-mon[22098]: pgmap v488: 97 pgs: 97 active+clean; 6.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:40:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.646+0000 7f342534a640 1 -- 172.21.15.103:0/1200913016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 msgr2=0x7f3420106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.646+0000 7f342534a640 1 --2- 172.21.15.103:0/1200913016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f3420106b90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3410009930 tx=0x7f341002a420 comp rx=0 tx=0).stop 2024-08-19T01:40:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.646+0000 7f342534a640 1 -- 172.21.15.103:0/1200913016 shutdown_connections 2024-08-19T01:40:46.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.646+0000 7f342534a640 1 --2- 172.21.15.103:0/1200913016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3420101150 0x7f342010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.646+0000 7f342534a640 1 --2- 172.21.15.103:0/1200913016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34201007b0 0x7f3420100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.647+0000 7f342534a640 1 --2- 172.21.15.103:0/1200913016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f3420106b90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.647+0000 7f342534a640 1 -- 172.21.15.103:0/1200913016 >> 172.21.15.103:0/1200913016 conn(0x7f34200fc4d0 msgr2=0x7f34200fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.647+0000 7f342534a640 1 -- 172.21.15.103:0/1200913016 shutdown_connections 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.647+0000 7f342534a640 1 -- 172.21.15.103:0/1200913016 wait complete. 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.647+0000 7f342534a640 1 Processor -- start 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f342534a640 1 -- start start 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f342534a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34201007b0 0x7f34201980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f342534a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3420101150 0x7f3420198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f342534a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f342019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f342534a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3420110a60 con 0x7f34201067b0 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f342534a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f34201108e0 con 0x7f3420101150 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f342534a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3420110be0 con 0x7f34201007b0 2024-08-19T01:40:46.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f342019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f342019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54896/0 (socket says 172.21.15.103:54896) 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 -- 172.21.15.103:0/380674801 learned_addr learned my addr 172.21.15.103:0/380674801 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341effd640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34201007b0 0x7f34201980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341e7fc640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3420101150 0x7f3420198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 -- 172.21.15.103:0/380674801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34201007b0 msgr2=0x7f34201980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34201007b0 0x7f34201980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 -- 172.21.15.103:0/380674801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3420101150 msgr2=0x7f3420198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3420101150 0x7f3420198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.648+0000 7f341f7fe640 1 -- 172.21.15.103:0/380674801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342019d0a0 con 0x7f34201067b0 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.649+0000 7f341f7fe640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f342019c9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3418009fd0 tx=0x7f341800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.649+0000 7f3407fff640 1 -- 172.21.15.103:0/380674801 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418017070 con 0x7f34201067b0 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.649+0000 7f3407fff640 1 -- 172.21.15.103:0/380674801 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3418003e20 con 0x7f34201067b0 2024-08-19T01:40:46.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.649+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342019d390 con 0x7f34201067b0 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.649+0000 7f3407fff640 1 -- 172.21.15.103:0/380674801 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418010460 con 0x7f34201067b0 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.649+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34200fcb40 con 0x7f34201067b0 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3407fff640 1 -- 172.21.15.103:0/380674801 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3418003970 con 0x7f34201067b0 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3427748740 1 Processor -- start 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3427748740 1 -- start start 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3427748740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e5085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3427748740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e512eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3427748740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e5133f0 0x56035e5157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3427748740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56035e46d980 con 0x56035e508b30 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.652+0000 7f3427748740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56035e405a70 con 0x56035e5133f0 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f3427748740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56035e469940 con 0x56035e508210 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e512eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e512eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54900/0 (socket says 172.21.15.103:54900) 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 -- 172.21.15.103:0/1421593100 learned_addr learned my addr 172.21.15.103:0/1421593100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e5085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e5133f0 0x56035e5157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 -- 172.21.15.103:0/1421593100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 msgr2=0x56035e5085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e5085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 -- 172.21.15.103:0/1421593100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e5133f0 msgr2=0x56035e5157e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e5133f0 0x56035e5157e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 -- 172.21.15.103:0/1421593100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035e4686e0 con 0x56035e508b30 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e5133f0 0x56035e5157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f3407fff640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f34080a6f90 0x7f34080a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341e7fc640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f34080a6f90 0x7f34080a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f341effd640 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e512eb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f341002a940 tx=0x7f3410031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2f9e97efef103b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f3407fff640 1 -- 172.21.15.103:0/380674801 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f34180cee10 con 0x7f34201067b0 2024-08-19T01:40:46.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.653+0000 7f34057fa640 1 -- 172.21.15.103:0/1421593100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410040070 con 0x56035e508b30 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f34057fa640 1 -- 172.21.15.103:0/1421593100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3410031e50 con 0x56035e508b30 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f34057fa640 1 -- 172.21.15.103:0/1421593100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410035040 con 0x56035e508b30 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 -- 172.21.15.103:0/1421593100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 msgr2=0x56035e512eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e512eb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f341002a940 tx=0x7f3410031ba0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f341e7fc640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f34080a6f90 0x7f34080a9450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56035e505b20 tx=0x7f3414005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 -- 172.21.15.103:0/1421593100 shutdown_connections 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e5133f0 0x56035e5157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e512eb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 --2- 172.21.15.103:0/1421593100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e5085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 -- 172.21.15.103:0/1421593100 >> 172.21.15.103:0/1421593100 conn(0x56035e5047a0 msgr2=0x56035e504b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 -- 172.21.15.103:0/1421593100 shutdown_connections 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.654+0000 7f3427748740 1 -- 172.21.15.103:0/1421593100 wait complete. 2024-08-19T01:40:46.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 -- start start 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 Processor -- start 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508210 0x56035e510290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e5107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e510d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56035e515fc0 con 0x56035e508210 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56035e428aa0 con 0x56035e508b30 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56035e516140 con 0x56035e5133f0 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f3427748740 10 client.0 _ll_register_callbacks cb 0x56035e417e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f341effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508210 0x56035e510290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f341effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508210 0x56035e510290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54914/0 (socket says 172.21.15.103:54914) 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f341effd640 1 -- 172.21.15.103:0/1192472314 learned_addr learned my addr 172.21.15.103:0/1192472314 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:46.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f341f7fe640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e510d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.655+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e5107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f341effd640 1 -- 172.21.15.103:0/1192472314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 msgr2=0x56035e510d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f341effd640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e510d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f341effd640 1 -- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 msgr2=0x56035e5107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f341effd640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e5107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f341effd640 1 -- 172.21.15.103:0/1192472314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035e40c650 con 0x56035e508210 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e5107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f341effd640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508210 0x56035e510290 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3410009a50 tx=0x7f3410005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c624d90021c80fc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3402ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3402ff5640 1 -- 172.21.15.103:0/1192472314 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341004d060 con 0x56035e508210 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3402ff5640 1 -- 172.21.15.103:0/1192472314 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3410004030 con 0x56035e508210 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3410031e50 con 0x56035e508210 2024-08-19T01:40:46.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3402ff5640 1 -- 172.21.15.103:0/1192472314 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3410040070 con 0x56035e508210 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3427748740 10 client.16737 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56035e1df3c0 con 0x56035e508210 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.656+0000 7f3402ff5640 1 -- 172.21.15.103:0/1192472314 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3410031e50 con 0x56035e508210 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3402ff5640 1 -- 172.21.15.103:0/1192472314 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f34100042e0 con 0x56035e508210 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3427748740 10 client.16737 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3427748740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3427748740 20 client.16737 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3427748740 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e522c80 0x56035e525070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3427748740 4 client.16737 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56035e47aef0 con 0x56035e522c80 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e522c80 0x56035e525070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e522c80 0x56035e525070 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e17b27ff347147 server_cookie=f8b870d1c70fe3f7 in_seq=0 out_seq=0 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.657+0000 7f3402ff5640 10 client.16737 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:46.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3402ff5640 1 -- 172.21.15.103:0/1192472314 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56035e47aef0 con 0x56035e522c80 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3402ff5640 10 client.16737 handle_command_reply: tid=0 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 client.16737 shutdown 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508210 msgr2=0x56035e510290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508210 0x56035e510290 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3410009a50 tx=0x7f3410005520 comp rx=0 tx=0).stop 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 shutdown_connections 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e522c80 0x56035e525070 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e510d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e5107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 --2- 172.21.15.103:0/1192472314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508210 0x56035e510290 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 >> 172.21.15.103:0/1192472314 conn(0x56035e5047a0 msgr2=0x56035e50a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3402ff5640 0 client.16737 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.658+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 shutdown_connections 2024-08-19T01:40:46.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.659+0000 7f3427748740 1 -- 172.21.15.103:0/1192472314 wait complete. 2024-08-19T01:40:46.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.659+0000 7f3427748740 20 client.16737 trim_cache size 0 max 16384 2024-08-19T01:40:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 Processor -- start 2024-08-19T01:40:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 -- start start 2024-08-19T01:40:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508210 0x56035e5129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e52a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e52aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34201aae80 con 0x56035e508b30 2024-08-19T01:40:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f34201acae0 con 0x56035e508210 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.678+0000 7f3427748740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f34201ab550 con 0x56035e5133f0 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e52a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e52a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54926/0 (socket says 172.21.15.103:54926) 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 -- 172.21.15.103:0/1062993456 learned_addr learned my addr 172.21.15.103:0/1062993456 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341f7fe640 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e52aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341effd640 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508210 0x56035e5129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 -- 172.21.15.103:0/1062993456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 msgr2=0x56035e52aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e52aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 -- 172.21.15.103:0/1062993456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508210 msgr2=0x56035e5129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508210 0x56035e5129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 -- 172.21.15.103:0/1062993456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035e50b080 con 0x56035e508b30 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f341e7fc640 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e52a760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3420104fd0 tx=0x7f3414016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2499a29a329e775b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f3404ff9640 1 -- 172.21.15.103:0/1062993456 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414005020 con 0x56035e508b30 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f3404ff9640 1 -- 172.21.15.103:0/1062993456 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34140047a0 con 0x56035e508b30 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.679+0000 7f3404ff9640 1 -- 172.21.15.103:0/1062993456 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341402a790 con 0x56035e508b30 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 -- 172.21.15.103:0/1062993456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 msgr2=0x56035e52a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e52a760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3420104fd0 tx=0x7f3414016000 comp rx=0 tx=0).stop 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 -- 172.21.15.103:0/1062993456 shutdown_connections 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e5133f0 0x56035e52aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e508b30 0x56035e52a760 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 --2- 172.21.15.103:0/1062993456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508210 0x56035e5129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 -- 172.21.15.103:0/1062993456 >> 172.21.15.103:0/1062993456 conn(0x56035e5047a0 msgr2=0x56035e50f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 -- 172.21.15.103:0/1062993456 shutdown_connections 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.680+0000 7f3427748740 1 -- 172.21.15.103:0/1062993456 wait complete. 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 -- start start 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 Processor -- start 2024-08-19T01:40:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e516ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e517010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e5133f0 0x56035e5856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34201ac350 con 0x56035e5133f0 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f34201abc20 con 0x56035e508b30 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f34201a8e80 con 0x56035e508210 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f3427748740 10 client.0 _ll_register_callbacks cb 0x7f3410031af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e5133f0 0x56035e5856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e5133f0 0x56035e5856f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54936/0 (socket says 172.21.15.103:54936) 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341f7fe640 1 -- 172.21.15.103:0/597988175 learned_addr learned my addr 172.21.15.103:0/597988175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341effd640 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e516ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341e7fc640 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e517010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341f7fe640 1 -- 172.21.15.103:0/597988175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 msgr2=0x56035e516ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341f7fe640 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e516ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341f7fe640 1 -- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 msgr2=0x56035e517010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.681+0000 7f341f7fe640 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e517010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f341f7fe640 1 -- 172.21.15.103:0/597988175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035e585d30 con 0x56035e5133f0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f341f7fe640 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e5133f0 0x56035e5856f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f34180cc990 tx=0x7f3418003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8597193e6f9a9c24 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3402ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3402ff5640 1 -- 172.21.15.103:0/597988175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418020040 con 0x56035e5133f0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3402ff5640 1 -- 172.21.15.103:0/597988175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f34180cce00 con 0x56035e5133f0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34140047a0 con 0x56035e5133f0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3402ff5640 1 -- 172.21.15.103:0/597988175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f341809b6a0 con 0x56035e5133f0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3427748740 10 client.16743 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56035e58d0f0 con 0x56035e5133f0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3402ff5640 1 -- 172.21.15.103:0/597988175 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34140047a0 con 0x56035e5133f0 2024-08-19T01:40:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.682+0000 7f3402ff5640 1 -- 172.21.15.103:0/597988175 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f34180a3e80 con 0x56035e5133f0 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3427748740 10 client.16743 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3427748740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3427748740 20 client.16743 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3427748740 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e58d850 0x56035e58fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3427748740 4 client.16743 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56035e475400 con 0x56035e58d850 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f341effd640 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e58d850 0x56035e58fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f341effd640 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e58d850 0x56035e58fc70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81618b5a509559 server_cookie=b6242cd7a96a4676 in_seq=0 out_seq=0 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3402ff5640 10 client.16743 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3402ff5640 1 -- 172.21.15.103:0/597988175 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56035e475400 con 0x56035e58d850 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.683+0000 7f3402ff5640 10 client.16743 handle_command_reply: tid=0 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 client.16743 shutdown 2024-08-19T01:40:46.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e5133f0 msgr2=0x56035e5856f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e5133f0 0x56035e5856f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f34180cc990 tx=0x7f3418003d90 comp rx=0 tx=0).stop 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 shutdown_connections 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56035e58d850 0x56035e58fc70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56035e5133f0 0x56035e5856f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56035e508b30 0x56035e517010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 --2- 172.21.15.103:0/597988175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56035e508210 0x56035e516ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 >> 172.21.15.103:0/597988175 conn(0x56035e5047a0 msgr2=0x56035e50ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3402ff5640 0 client.16743 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 shutdown_connections 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 1 -- 172.21.15.103:0/597988175 wait complete. 2024-08-19T01:40:46.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.684+0000 7f3427748740 20 client.16743 trim_cache size 0 max 16384 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: "71406eaf-e402-4609-914e-4fc7b7ebe8f5": { 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "71406eaf-e402-4609-914e-4fc7b7ebe8f5", 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:40:46.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:40:46.698 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:40:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f34080a6f90 msgr2=0x7f34080a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f34080a6f90 0x7f34080a9450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56035e505b20 tx=0x7f3414005f90 comp rx=0 tx=0).stop 2024-08-19T01:40:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 msgr2=0x7f342019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f342019c9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3418009fd0 tx=0x7f341800b260 comp rx=0 tx=0).stop 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 shutdown_connections 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f34080a6f90 0x7f34080a9450 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34201067b0 0x7f342019c9c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3420101150 0x7f3420198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 --2- 172.21.15.103:0/380674801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34201007b0 0x7f34201980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 >> 172.21.15.103:0/380674801 conn(0x7f34200fc4d0 msgr2=0x7f342006a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 shutdown_connections 2024-08-19T01:40:46.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:40:46.685+0000 7f342534a640 1 -- 172.21.15.103:0/380674801 wait complete. 2024-08-19T01:40:47.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:46 smithi103 ceph-mon[18436]: pgmap v489: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 32 op/s 2024-08-19T01:40:47.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:46 smithi190 ceph-mon[22032]: pgmap v489: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 32 op/s 2024-08-19T01:40:47.428 INFO:tasks.cephfs.filesystem:scrub status for tag:71406eaf-e402-4609-914e-4fc7b7ebe8f5 - {'path': '/', 'tag': '71406eaf-e402-4609-914e-4fc7b7ebe8f5', 'options': 'recursive,force'} 2024-08-19T01:40:47.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:46 smithi146 ceph-mon[22098]: pgmap v489: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 32 op/s 2024-08-19T01:40:50.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:49 smithi146 ceph-mon[22098]: pgmap v490: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:40:50.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:49 smithi103 ceph-mon[18436]: pgmap v490: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:40:50.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:49 smithi190 ceph-mon[22032]: pgmap v490: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:40:52.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:51 smithi146 ceph-mon[22098]: pgmap v491: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-08-19T01:40:52.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:51 smithi103 ceph-mon[18436]: pgmap v491: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-08-19T01:40:52.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:51 smithi190 ceph-mon[22032]: pgmap v491: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-08-19T01:40:54.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:53 smithi146 ceph-mon[22098]: pgmap v492: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:54.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:53 smithi103 ceph-mon[18436]: pgmap v492: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:54.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:53 smithi190 ceph-mon[22032]: pgmap v492: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:40:56.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:55 smithi146 ceph-mon[22098]: pgmap v493: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:40:56.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:55 smithi103 ceph-mon[18436]: pgmap v493: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:40:56.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:55 smithi190 ceph-mon[22032]: pgmap v493: 97 pgs: 97 active+clean; 7.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:40:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 -- 172.21.15.103:0/659535162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c104ce0 msgr2=0x7f8a4c1070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/659535162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c104ce0 0x7f8a4c1070d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009900 tx=0x7f8a3802a3f0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 -- 172.21.15.103:0/659535162 shutdown_connections 2024-08-19T01:40:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/659535162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a4c107610 0x7f8a4c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/659535162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c104ce0 0x7f8a4c1070d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/659535162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a4c1023b0 0x7f8a4c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 -- 172.21.15.103:0/659535162 >> 172.21.15.103:0/659535162 conn(0x7f8a4c0fbf10 msgr2=0x7f8a4c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 -- 172.21.15.103:0/659535162 shutdown_connections 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.634+0000 7f8a52bf1640 1 -- 172.21.15.103:0/659535162 wait complete. 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 Processor -- start 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 -- start start 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a4c1023b0 0x7f8a4c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a4c104ce0 0x7f8a4c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c107610 0x7f8a4c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10d3d0 con 0x7f8a4c107610 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10d250 con 0x7f8a4c104ce0 2024-08-19T01:40:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a52bf1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10d550 con 0x7f8a4c1023b0 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c107610 0x7f8a4c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5098d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a4c1023b0 0x7f8a4c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c107610 0x7f8a4c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39334/0 (socket says 172.21.15.103:39334) 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 -- 172.21.15.103:0/151940467 learned_addr learned my addr 172.21.15.103:0/151940467 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a4c104ce0 0x7f8a4c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 -- 172.21.15.103:0/151940467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a4c1023b0 msgr2=0x7f8a4c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a4c1023b0 0x7f8a4c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 -- 172.21.15.103:0/151940467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a4c104ce0 msgr2=0x7f8a4c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a4c104ce0 0x7f8a4c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 -- 172.21.15.103:0/151940467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c19f6e0 con 0x7f8a4c107610 2024-08-19T01:40:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.635+0000 7f8a5118e640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c107610 0x7f8a4c19f070 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400eaa0 tx=0x7f8a4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.636+0000 7f8a49ffb640 1 -- 172.21.15.103:0/151940467 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400cdb0 con 0x7f8a4c107610 2024-08-19T01:40:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.636+0000 7f8a49ffb640 1 -- 172.21.15.103:0/151940467 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44005020 con 0x7f8a4c107610 2024-08-19T01:40:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.636+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c19f9d0 con 0x7f8a4c107610 2024-08-19T01:40:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.636+0000 7f8a49ffb640 1 -- 172.21.15.103:0/151940467 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440116d0 con 0x7f8a4c107610 2024-08-19T01:40:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.636+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c19fef0 con 0x7f8a4c107610 2024-08-19T01:40:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.638+0000 7f8a49ffb640 1 -- 172.21.15.103:0/151940467 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8a440051d0 con 0x7f8a4c107610 2024-08-19T01:40:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 Processor -- start 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 -- start start 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a49ffb640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8a3c0a2cb0 0x7f8a3c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53c680 0x55e28e53ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e541a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e544510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e28e49f9d0 con 0x55e28e53c680 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e28e447820 con 0x55e28e542050 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e28e49b990 con 0x55e28e53cfc0 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a49ffb640 1 -- 172.21.15.103:0/151940467 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8a440cf1b0 con 0x7f8a4c107610 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8a3c0a2cb0 0x7f8a3c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a5098d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53c680 0x55e28e53ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e541a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a5098d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53c680 0x55e28e53ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39340/0 (socket says 172.21.15.103:39340) 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a5098d640 1 -- 172.21.15.103:0/637038963 learned_addr learned my addr 172.21.15.103:0/637038963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.639+0000 7f8a5118e640 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e544510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a5098d640 1 -- 172.21.15.103:0/637038963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 msgr2=0x55e28e541a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a5098d640 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e541a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a5098d640 1 -- 172.21.15.103:0/637038963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 msgr2=0x55e28e544510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a5098d640 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e544510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a5098d640 1 -- 172.21.15.103:0/637038963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e28e1b3210 con 0x55e28e53c680 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a5098d640 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53c680 0x55e28e53ca80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009cf0 tx=0x7f8a40011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8214cf37400f5bdf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8a3c0a2cb0 0x7f8a3c0a5170 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a38006010 tx=0x7f8a38005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a36ffd640 1 -- 172.21.15.103:0/637038963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40004220 con 0x55e28e53c680 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a36ffd640 1 -- 172.21.15.103:0/637038963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a400043c0 con 0x55e28e53c680 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.640+0000 7f8a36ffd640 1 -- 172.21.15.103:0/637038963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40018830 con 0x55e28e53c680 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 -- 172.21.15.103:0/637038963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53c680 msgr2=0x55e28e53ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53c680 0x55e28e53ca80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009cf0 tx=0x7f8a40011810 comp rx=0 tx=0).stop 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 -- 172.21.15.103:0/637038963 shutdown_connections 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e544510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e541a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/637038963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53c680 0x55e28e53ca80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 -- 172.21.15.103:0/637038963 >> 172.21.15.103:0/637038963 conn(0x55e28e537cf0 msgr2=0x55e28e5380f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 -- 172.21.15.103:0/637038963 shutdown_connections 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 -- 172.21.15.103:0/637038963 wait complete. 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 -- start start 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.641+0000 7f8a55a8e740 1 Processor -- start 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e546780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53cfc0 0x55e28e546cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e54dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e28e545020 con 0x55e28e53cfc0 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e28e45b520 con 0x55e28e542050 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e28e5451a0 con 0x55e28e53c680 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a55a8e740 10 client.0 _ll_register_callbacks cb 0x55e28e1b2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53cfc0 0x55e28e546cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53cfc0 0x55e28e546cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39346/0 (socket says 172.21.15.103:39346) 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 -- 172.21.15.103:0/761869041 learned_addr learned my addr 172.21.15.103:0/761869041 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a5098d640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e546780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a5118e640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e54dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 -- 172.21.15.103:0/761869041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 msgr2=0x55e28e546780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e546780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 -- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 msgr2=0x55e28e54dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e54dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 -- 172.21.15.103:0/761869041 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e28e27f620 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a5098d640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e546780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53cfc0 0x55e28e546cc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8a38033000 tx=0x7f8a3802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2e4409d2322e0b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.642+0000 7f8a2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a2bfff640 1 -- 172.21.15.103:0/761869041 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3804e070 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a2bfff640 1 -- 172.21.15.103:0/761869041 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8a38003a00 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a400043c0 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a2bfff640 1 -- 172.21.15.103:0/761869041 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a3805a050 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 10 client.16755 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e28e550b60 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a2bfff640 1 -- 172.21.15.103:0/761869041 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a400043c0 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a2bfff640 1 -- 172.21.15.103:0/761869041 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8a38032b60 con 0x55e28e53cfc0 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 10 client.16755 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 20 client.16755 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5523b0 0x55e28e554820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 4 client.16755 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.643+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e28e4ad100 con 0x55e28e5523b0 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.644+0000 7f8a5098d640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5523b0 0x55e28e554820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.644+0000 7f8a5098d640 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5523b0 0x55e28e554820 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3fabb66546264ef server_cookie=db255cf37760a89 in_seq=0 out_seq=0 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.644+0000 7f8a2bfff640 10 client.16755 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a2bfff640 1 -- 172.21.15.103:0/761869041 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e28e4ad100 con 0x55e28e5523b0 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a2bfff640 10 client.16755 handle_command_reply: tid=0 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 client.16755 shutdown 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53cfc0 msgr2=0x55e28e546cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53cfc0 0x55e28e546cc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8a38033000 tx=0x7f8a3802a3f0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 shutdown_connections 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5523b0 0x55e28e554820 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e542050 0x55e28e54dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e53cfc0 0x55e28e546cc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/761869041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e546780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 >> 172.21.15.103:0/761869041 conn(0x55e28e537cf0 msgr2=0x55e28e539500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a2bfff640 0 client.16755 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 shutdown_connections 2024-08-19T01:40:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 1 -- 172.21.15.103:0/761869041 wait complete. 2024-08-19T01:40:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.645+0000 7f8a55a8e740 20 client.16755 trim_cache size 0 max 16384 2024-08-19T01:40:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 Processor -- start 2024-08-19T01:40:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 -- start start 2024-08-19T01:40:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e538ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53cfc0 0x55e28e54dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e5390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1a8c10 con 0x55e28e542050 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1aa870 con 0x55e28e53cfc0 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.666+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1a92e0 con 0x55e28e53c680 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e5390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e5390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39354/0 (socket says 172.21.15.103:39354) 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 -- 172.21.15.103:0/3616395397 learned_addr learned my addr 172.21.15.103:0/3616395397 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53cfc0 0x55e28e54dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5098d640 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e538ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 -- 172.21.15.103:0/3616395397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 msgr2=0x55e28e538ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e538ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 -- 172.21.15.103:0/3616395397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53cfc0 msgr2=0x55e28e54dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53cfc0 0x55e28e54dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 -- 172.21.15.103:0/3616395397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e28e53d4a0 con 0x55e28e542050 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a5118e640 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e5390e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400efd0 tx=0x7f8a4409c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7afd389523daf8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a367fc640 1 -- 172.21.15.103:0/3616395397 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401f040 con 0x55e28e542050 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a367fc640 1 -- 172.21.15.103:0/3616395397 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44099d40 con 0x55e28e542050 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.667+0000 7f8a367fc640 1 -- 172.21.15.103:0/3616395397 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440979d0 con 0x55e28e542050 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 -- 172.21.15.103:0/3616395397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 msgr2=0x55e28e5390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e5390e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400efd0 tx=0x7f8a4409c280 comp rx=0 tx=0).stop 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 -- 172.21.15.103:0/3616395397 shutdown_connections 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e5390e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53cfc0 0x55e28e54dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/3616395397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53c680 0x55e28e538ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 -- 172.21.15.103:0/3616395397 >> 172.21.15.103:0/3616395397 conn(0x55e28e537cf0 msgr2=0x55e28e546780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 -- 172.21.15.103:0/3616395397 shutdown_connections 2024-08-19T01:40:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.668+0000 7f8a55a8e740 1 -- 172.21.15.103:0/3616395397 wait complete. 2024-08-19T01:40:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 -- start start 2024-08-19T01:40:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 Processor -- start 2024-08-19T01:40:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53c680 0x55e28e54b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e54b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e54c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1aa0e0 con 0x55e28e542050 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1a99b0 con 0x55e28e53c680 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1a6bf0 con 0x55e28e53cfc0 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a55a8e740 10 client.0 _ll_register_callbacks cb 0x7f8a38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a5118e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e54c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a5098d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53c680 0x55e28e54b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a5118e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e54c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39366/0 (socket says 172.21.15.103:39366) 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a5098d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53c680 0x55e28e54b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:42612/0 (socket says 172.21.15.103:42612) 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a5118e640 1 -- 172.21.15.103:0/1864024690 learned_addr learned my addr 172.21.15.103:0/1864024690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.669+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e54b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a5118e640 1 -- 172.21.15.103:0/1864024690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 msgr2=0x55e28e54b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a5118e640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e54b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a5118e640 1 -- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53c680 msgr2=0x55e28e54b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:40:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a5118e640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53c680 0x55e28e54b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a5118e640 1 -- 172.21.15.103:0/1864024690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e28e54cbf0 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a5098d640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53c680 0x55e28e54b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a4bfff640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e54b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a5118e640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e54c5e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8a440ccd30 tx=0x7f8a44099ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965bc005e9caf982 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a2bfff640 1 -- 172.21.15.103:0/1864024690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401f040 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a2bfff640 1 -- 172.21.15.103:0/1864024690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8a44097990 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a44099d40 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a2bfff640 1 -- 172.21.15.103:0/1864024690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8a4409a900 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a55a8e740 10 client.16761 fetch_fsmap learned FSMap version 32 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e28e54df70 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a2bfff640 1 -- 172.21.15.103:0/1864024690 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44099d40 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.670+0000 7f8a2bfff640 1 -- 172.21.15.103:0/1864024690 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8a4400f040 con 0x55e28e542050 2024-08-19T01:40:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a55a8e740 10 client.16761 fetch_fsmap finished waiting for FSMap version 32 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a55a8e740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a55a8e740 20 client.16761 populate_metadata read hostname 'smithi103' 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5bd7d0 0x55e28e5bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a55a8e740 4 client.16761 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e28e4a7450 con 0x55e28e5bd7d0 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a5098d640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5bd7d0 0x55e28e5bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a2bfff640 10 client.16761 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.671+0000 7f8a5098d640 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5bd7d0 0x55e28e5bfbf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d162988dfa942a50 server_cookie=1f800eb3a28abdff in_seq=0 out_seq=0 2024-08-19T01:40:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a2bfff640 1 -- 172.21.15.103:0/1864024690 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e28e4a7450 con 0x55e28e5bd7d0 2024-08-19T01:40:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a2bfff640 10 client.16761 handle_command_reply: tid=0 2024-08-19T01:40:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 client.16761 shutdown 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 msgr2=0x55e28e54c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e54c5e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8a440ccd30 tx=0x7f8a44099ef0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 shutdown_connections 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e28e5bd7d0 0x55e28e5bfbf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e28e542050 0x55e28e54c5e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a2bfff640 0 client.16761 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e28e53cfc0 0x55e28e54b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 --2- 172.21.15.103:0/1864024690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e28e53c680 0x55e28e54b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 >> 172.21.15.103:0/1864024690 conn(0x55e28e537cf0 msgr2=0x55e28e536680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 shutdown_connections 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.678+0000 7f8a55a8e740 1 -- 172.21.15.103:0/1864024690 wait complete. 2024-08-19T01:40:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a55a8e740 20 client.16761 trim_cache size 0 max 16384 2024-08-19T01:40:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:40:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:40:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:40:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8a3c0a2cb0 msgr2=0x7f8a3c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8a3c0a2cb0 0x7f8a3c0a5170 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a38006010 tx=0x7f8a38005d50 comp rx=0 tx=0).stop 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c107610 msgr2=0x7f8a4c19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c107610 0x7f8a4c19f070 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400eaa0 tx=0x7f8a4400ef60 comp rx=0 tx=0).stop 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 shutdown_connections 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8a3c0a2cb0 0x7f8a3c0a5170 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a4c107610 0x7f8a4c19f070 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a4c104ce0 0x7f8a4c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 --2- 172.21.15.103:0/151940467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a4c1023b0 0x7f8a4c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 >> 172.21.15.103:0/151940467 conn(0x7f8a4c0fbf10 msgr2=0x7f8a4c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:40:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 shutdown_connections 2024-08-19T01:40:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:40:56.679+0000 7f8a52bf1640 1 -- 172.21.15.103:0/151940467 wait complete. 2024-08-19T01:40:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:56 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:56 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:57.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:56 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:57.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:56 smithi103 ceph-mon[18436]: pgmap v494: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 MiB/s wr, 43 op/s 2024-08-19T01:40:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:56 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:56 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:56 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:56 smithi190 ceph-mon[22032]: pgmap v494: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 MiB/s wr, 43 op/s 2024-08-19T01:40:57.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:56 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:57.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:56 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:57.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:56 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:57.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:56 smithi146 ceph-mon[22098]: pgmap v494: 97 pgs: 97 active+clean; 7.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 MiB/s wr, 43 op/s 2024-08-19T01:40:58.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:57 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:40:58.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:57 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:40:58.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:57 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:40:59.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:40:58 smithi103 ceph-mon[18436]: pgmap v495: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 37 op/s 2024-08-19T01:40:59.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:40:58 smithi190 ceph-mon[22032]: pgmap v495: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 37 op/s 2024-08-19T01:40:59.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:40:58 smithi146 ceph-mon[22098]: pgmap v495: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 37 op/s 2024-08-19T01:41:02.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:01 smithi146 ceph-mon[22098]: pgmap v496: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:41:02.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:01 smithi103 ceph-mon[18436]: pgmap v496: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:41:02.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:01 smithi190 ceph-mon[22032]: pgmap v496: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:41:03.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:02 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:03.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:02 smithi146 ceph-mon[22098]: pgmap v497: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:41:03.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:02 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:03.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:02 smithi103 ceph-mon[18436]: pgmap v497: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:41:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:02 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:02 smithi190 ceph-mon[22032]: pgmap v497: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:41:06.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:05 smithi146 ceph-mon[22098]: pgmap v498: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:41:06.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:05 smithi103 ceph-mon[18436]: pgmap v498: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:41:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:05 smithi190 ceph-mon[22032]: pgmap v498: 97 pgs: 97 active+clean; 7.3 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:41:08.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:07 smithi146 ceph-mon[22098]: pgmap v499: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 23 op/s 2024-08-19T01:41:08.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:07 smithi103 ceph-mon[18436]: pgmap v499: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 23 op/s 2024-08-19T01:41:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:07 smithi190 ceph-mon[22032]: pgmap v499: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 23 op/s 2024-08-19T01:41:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 -- 172.21.15.103:0/3188799807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c102320 msgr2=0x7f900c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 --2- 172.21.15.103:0/3188799807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c102320 0x7f900c108bb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9004009ca0 tx=0x7f900402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 -- 172.21.15.103:0/3188799807 shutdown_connections 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 --2- 172.21.15.103:0/3188799807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c102320 0x7f900c108bb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 --2- 172.21.15.103:0/3188799807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c101960 0x7f900c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 --2- 172.21.15.103:0/3188799807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f900c100770 0x7f900c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 -- 172.21.15.103:0/3188799807 >> 172.21.15.103:0/3188799807 conn(0x7f900c0fbf10 msgr2=0x7f900c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 -- 172.21.15.103:0/3188799807 shutdown_connections 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 -- 172.21.15.103:0/3188799807 wait complete. 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.770+0000 7f9011f98640 1 Processor -- start 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f9011f98640 1 -- start start 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f9011f98640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f900c100770 0x7f900c06a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f9011f98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c101960 0x7f900c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f9011f98640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c102320 0x7f900c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f9011f98640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10c2a0 con 0x7f900c101960 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f9011f98640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10c120 con 0x7f900c102320 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f9011f98640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10c420 con 0x7f900c100770 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c101960 0x7f900c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c101960 0x7f900c19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53284/0 (socket says 172.21.15.103:53284) 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 -- 172.21.15.103:0/1205325631 learned_addr learned my addr 172.21.15.103:0/1205325631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f900c100770 0x7f900c06a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900bfff640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c102320 0x7f900c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 -- 172.21.15.103:0/1205325631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f900c100770 msgr2=0x7f900c06a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f900c100770 0x7f900c06a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 -- 172.21.15.103:0/1205325631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c102320 msgr2=0x7f900c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c102320 0x7f900c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900affd640 1 -- 172.21.15.103:0/1205325631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c19f670 con 0x7f900c101960 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.771+0000 7f900bfff640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c102320 0x7f900c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.772+0000 7f900affd640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c101960 0x7f900c19c740 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f900000c890 tx=0x7f900000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.772+0000 7f9008ff9640 1 -- 172.21.15.103:0/1205325631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000004220 con 0x7f900c101960 2024-08-19T01:41:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.772+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c19f900 con 0x7f900c101960 2024-08-19T01:41:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.772+0000 7f9008ff9640 1 -- 172.21.15.103:0/1205325631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90000043c0 con 0x7f900c101960 2024-08-19T01:41:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.772+0000 7f9008ff9640 1 -- 172.21.15.103:0/1205325631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000004a00 con 0x7f900c101960 2024-08-19T01:41:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.772+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c19fe50 con 0x7f900c101960 2024-08-19T01:41:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.774+0000 7f9014e27740 1 Processor -- start 2024-08-19T01:41:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.774+0000 7f9014e27740 1 -- start start 2024-08-19T01:41:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.774+0000 7f9008ff9640 1 -- 172.21.15.103:0/1205325631 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9000004ba0 con 0x7f900c101960 2024-08-19T01:41:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f9014e27740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a646ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f9014e27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a6471a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f9014e27740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a6472050 0x5649a6474510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f9014e27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5649a63cf9d0 con 0x5649a646cfc0 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f9014e27740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5649a6377820 con 0x5649a646c680 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f9014e27740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649a63cb990 con 0x5649a6472050 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a6471a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a6471a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53286/0 (socket says 172.21.15.103:53286) 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 -- 172.21.15.103:0/1992808647 learned_addr learned my addr 172.21.15.103:0/1992808647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900bfff640 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a646ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900affd640 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a6472050 0x5649a6474510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 -- 172.21.15.103:0/1992808647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a6472050 msgr2=0x5649a6474510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a6472050 0x5649a6474510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 -- 172.21.15.103:0/1992808647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 msgr2=0x5649a646ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a646ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 -- 172.21.15.103:0/1992808647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a60e3210 con 0x5649a646cfc0 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900bfff640 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a646ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a6471a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00bcb0 tx=0x7f8ffc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123ce2c76da5f887 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f9008ff9640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ff40a2c30 0x7f8ff40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.775+0000 7f8ff1ffb640 1 -- 172.21.15.103:0/1992808647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc005020 con 0x5649a646cfc0 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f900bfff640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ff40a2c30 0x7f8ff40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f8ff1ffb640 1 -- 172.21.15.103:0/1992808647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ffc0047a0 con 0x5649a646cfc0 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f8ff1ffb640 1 -- 172.21.15.103:0/1992808647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc011640 con 0x5649a646cfc0 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9008ff9640 1 -- 172.21.15.103:0/1205325631 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f90000cecd0 con 0x7f900c101960 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 -- 172.21.15.103:0/1992808647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 msgr2=0x5649a6471a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f900bfff640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ff40a2c30 0x7f8ff40a50f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9004009a60 tx=0x7f9004005fa0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a6471a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00bcb0 tx=0x7f8ffc0107a0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 -- 172.21.15.103:0/1992808647 shutdown_connections 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a6472050 0x5649a6474510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a6471a80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 --2- 172.21.15.103:0/1992808647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a646ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 -- 172.21.15.103:0/1992808647 >> 172.21.15.103:0/1992808647 conn(0x5649a6467cf0 msgr2=0x5649a64680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 -- 172.21.15.103:0/1992808647 shutdown_connections 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.776+0000 7f9014e27740 1 -- 172.21.15.103:0/1992808647 wait complete. 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 -- start start 2024-08-19T01:41:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 Processor -- start 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a6476810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646cfc0 0x5649a6476d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5649a6475020 con 0x5649a6472050 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5649a638b520 con 0x5649a646c680 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5649a64751a0 con 0x5649a646cfc0 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f9014e27740 10 client.0 _ll_register_callbacks cb 0x5649a60e2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f900bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f900bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53300/0 (socket says 172.21.15.103:53300) 2024-08-19T01:41:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f900bfff640 1 -- 172.21.15.103:0/1122419423 learned_addr learned my addr 172.21.15.103:0/1122419423 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.777+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a6476810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900affd640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646cfc0 0x5649a6476d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900bfff640 1 -- 172.21.15.103:0/1122419423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646cfc0 msgr2=0x5649a6476d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900bfff640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646cfc0 0x5649a6476d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900bfff640 1 -- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 msgr2=0x5649a6476810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900bfff640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a6476810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900bfff640 1 -- 172.21.15.103:0/1122419423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a61af620 con 0x5649a6472050 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a6476810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f900bfff640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647dd90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9004035000 tx=0x7f90040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3be41f4d0c33bfa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f8feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f8feaffd640 1 -- 172.21.15.103:0/1122419423 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900404f070 con 0x5649a6472050 2024-08-19T01:41:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f8feaffd640 1 -- 172.21.15.103:0/1122419423 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9004004790 con 0x5649a6472050 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ffc0047a0 con 0x5649a6472050 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f8feaffd640 1 -- 172.21.15.103:0/1122419423 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9004031720 con 0x5649a6472050 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f9014e27740 10 client.16773 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.778+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5649a6480b80 con 0x5649a6472050 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f8feaffd640 1 -- 172.21.15.103:0/1122419423 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ffc0047a0 con 0x5649a6472050 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f8feaffd640 1 -- 172.21.15.103:0/1122419423 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f9004044040 con 0x5649a6472050 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f9014e27740 10 client.16773 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f9014e27740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f9014e27740 20 client.16773 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f9014e27740 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a6482320 0x5649a64847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f9014e27740 4 client.16773 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649a63dd100 con 0x5649a6482320 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.779+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a6482320 0x5649a64847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f8feaffd640 10 client.16773 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f900b7fe640 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a6482320 0x5649a64847c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba714a6dde29a4 server_cookie=8427ba5e87dfa1c in_seq=0 out_seq=0 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f8feaffd640 1 -- 172.21.15.103:0/1122419423 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649a63dd100 con 0x5649a6482320 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f8feaffd640 10 client.16773 handle_command_reply: tid=0 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f9014e27740 1 client.16773 shutdown 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 msgr2=0x5649a647dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f9014e27740 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647dd90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9004035000 tx=0x7f90040095c0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 shutdown_connections 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.780+0000 7f9014e27740 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a6482320 0x5649a64847c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f9014e27740 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647dd90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f9014e27740 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646cfc0 0x5649a6476d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f8feaffd640 0 client.16773 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f9014e27740 1 --2- 172.21.15.103:0/1122419423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646c680 0x5649a6476810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 >> 172.21.15.103:0/1122419423 conn(0x5649a6467cf0 msgr2=0x5649a6469500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 shutdown_connections 2024-08-19T01:41:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f9014e27740 1 -- 172.21.15.103:0/1122419423 wait complete. 2024-08-19T01:41:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.781+0000 7f9014e27740 20 client.16773 trim_cache size 0 max 16384 2024-08-19T01:41:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 Processor -- start 2024-08-19T01:41:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 -- start start 2024-08-19T01:41:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a6466d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a647dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a6472050 0x5649a6489080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1a8b60 con 0x5649a646cfc0 2024-08-19T01:41:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1aa730 con 0x5649a6472050 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f9014e27740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1a9230 con 0x5649a646c680 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f900affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a647dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f900affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a647dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53308/0 (socket says 172.21.15.103:53308) 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f900affd640 1 -- 172.21.15.103:0/424826104 learned_addr learned my addr 172.21.15.103:0/424826104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.802+0000 7f900b7fe640 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a6466d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f900bfff640 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a6472050 0x5649a6489080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f900affd640 1 -- 172.21.15.103:0/424826104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 msgr2=0x5649a6466d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f900affd640 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a6466d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f900affd640 1 -- 172.21.15.103:0/424826104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a6472050 msgr2=0x5649a6489080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f900affd640 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a6472050 0x5649a6489080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f900affd640 1 -- 172.21.15.103:0/424826104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a6484e40 con 0x5649a646cfc0 2024-08-19T01:41:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f900affd640 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a647dcb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9000005fd0 tx=0x7f90000cefd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374e1fd276dea660 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f8ff17fa640 1 -- 172.21.15.103:0/424826104 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900001d040 con 0x5649a646cfc0 2024-08-19T01:41:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f8ff17fa640 1 -- 172.21.15.103:0/424826104 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90000d0040 con 0x5649a646cfc0 2024-08-19T01:41:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.803+0000 7f8ff17fa640 1 -- 172.21.15.103:0/424826104 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000099860 con 0x5649a646cfc0 2024-08-19T01:41:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 -- 172.21.15.103:0/424826104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 msgr2=0x5649a647dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a647dcb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9000005fd0 tx=0x7f90000cefd0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 -- 172.21.15.103:0/424826104 shutdown_connections 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a6472050 0x5649a6489080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a646cfc0 0x5649a647dcb0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 --2- 172.21.15.103:0/424826104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a6466d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 -- 172.21.15.103:0/424826104 >> 172.21.15.103:0/424826104 conn(0x5649a6467cf0 msgr2=0x5649a6466680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 -- 172.21.15.103:0/424826104 shutdown_connections 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 -- 172.21.15.103:0/424826104 wait complete. 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.804+0000 7f9014e27740 1 -- start start 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 1 Processor -- start 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a647b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646cfc0 0x5649a647bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1aa030 con 0x5649a6472050 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1a9900 con 0x5649a646cfc0 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1a6b40 con 0x5649a646c680 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f9014e27740 10 client.0 _ll_register_callbacks cb 0x7f9004009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53320/0 (socket says 172.21.15.103:53320) 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 -- 172.21.15.103:0/836085819 learned_addr learned my addr 172.21.15.103:0/836085819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900b7fe640 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a647b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900affd640 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646cfc0 0x5649a647bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 -- 172.21.15.103:0/836085819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 msgr2=0x5649a647b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a647b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 -- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646cfc0 msgr2=0x5649a647bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646cfc0 0x5649a647bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 -- 172.21.15.103:0/836085819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a647d510 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f900bfff640 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647cf00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9004046640 tx=0x7f90040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba4c53fee9381c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.805+0000 7f8feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f8feaffd640 1 -- 172.21.15.103:0/836085819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900404f040 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f8feaffd640 1 -- 172.21.15.103:0/836085819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9004046c40 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90000d0040 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f8feaffd640 1 -- 172.21.15.103:0/836085819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9004031450 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 10 client.16779 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5649a64ed5b0 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f8feaffd640 1 -- 172.21.15.103:0/836085819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90000d0040 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f8feaffd640 1 -- 172.21.15.103:0/836085819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f9004044070 con 0x5649a6472050 2024-08-19T01:41:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 10 client.16779 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 20 client.16779 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a64edfb0 0x5649a64f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 4 client.16779 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649a63d7450 con 0x5649a64edfb0 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.806+0000 7f900b7fe640 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a64edfb0 0x5649a64f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.807+0000 7f900b7fe640 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a64edfb0 0x5649a64f03d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b744eaef3e7472 server_cookie=85b1e50f2da53804 in_seq=0 out_seq=0 2024-08-19T01:41:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.807+0000 7f8feaffd640 10 client.16779 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f8feaffd640 1 -- 172.21.15.103:0/836085819 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5649a63d7450 con 0x5649a64edfb0 2024-08-19T01:41:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f8feaffd640 10 client.16779 handle_command_reply: tid=0 2024-08-19T01:41:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 client.16779 shutdown 2024-08-19T01:41:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 msgr2=0x5649a647cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647cf00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9004046640 tx=0x7f90040046c0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 shutdown_connections 2024-08-19T01:41:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5649a64edfb0 0x5649a64f03d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649a6472050 0x5649a647cf00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5649a646cfc0 0x5649a647bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 --2- 172.21.15.103:0/836085819 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a646c680 0x5649a647b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 >> 172.21.15.103:0/836085819 conn(0x5649a6467cf0 msgr2=0x5649a64693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f8feaffd640 0 client.16779 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 shutdown_connections 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 1 -- 172.21.15.103:0/836085819 wait complete. 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.814+0000 7f9014e27740 20 client.16779 trim_cache size 0 max 16384 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:41:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:41:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:41:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:41:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ff40a2c30 msgr2=0x7f8ff40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ff40a2c30 0x7f8ff40a50f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9004009a60 tx=0x7f9004005fa0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c101960 msgr2=0x7f900c19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c101960 0x7f900c19c740 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f900000c890 tx=0x7f900000cd50 comp rx=0 tx=0).stop 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 shutdown_connections 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ff40a2c30 0x7f8ff40a50f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f900c102320 0x7f900c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f900c101960 0x7f900c19c740 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 --2- 172.21.15.103:0/1205325631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f900c100770 0x7f900c06a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 >> 172.21.15.103:0/1205325631 conn(0x7f900c0fbf10 msgr2=0x7f900c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 shutdown_connections 2024-08-19T01:41:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:09.815+0000 7f9011f98640 1 -- 172.21.15.103:0/1205325631 wait complete. 2024-08-19T01:41:10.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:09 smithi146 ceph-mon[22098]: pgmap v500: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:41:10.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:09 smithi103 ceph-mon[18436]: pgmap v500: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:41:10.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:09 smithi190 ceph-mon[22032]: pgmap v500: 97 pgs: 97 active+clean; 7.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:41:11.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:10 smithi103 ceph-mon[18436]: pgmap v501: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:41:11.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:10 smithi190 ceph-mon[22032]: pgmap v501: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:41:11.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:10 smithi146 ceph-mon[22098]: pgmap v501: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:41:14.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:13 smithi146 ceph-mon[22098]: pgmap v502: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:41:14.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:13 smithi103 ceph-mon[18436]: pgmap v502: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:41:14.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:13 smithi190 ceph-mon[22032]: pgmap v502: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:41:16.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:15 smithi146 ceph-mon[22098]: pgmap v503: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:41:16.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:15 smithi103 ceph-mon[18436]: pgmap v503: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:41:16.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:15 smithi190 ceph-mon[22032]: pgmap v503: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:41:17.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:17 smithi103 ceph-mon[18436]: pgmap v504: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:41:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:17 smithi190 ceph-mon[22032]: pgmap v504: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:41:17.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:41:17.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:17 smithi146 ceph-mon[22098]: pgmap v504: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:41:17.717 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:18.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 -- 172.21.15.103:0/2317339698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380102a50 msgr2=0x7fd380102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 --2- 172.21.15.103:0/2317339698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380102a50 0x7fd380102e30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009900 tx=0x7fd37c02a480 comp rx=0 tx=0).stop 2024-08-19T01:41:18.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 -- 172.21.15.103:0/2317339698 shutdown_connections 2024-08-19T01:41:18.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 --2- 172.21.15.103:0/2317339698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd38010be80 0x7fd38010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 --2- 172.21.15.103:0/2317339698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd380103370 0x7fd38010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 --2- 172.21.15.103:0/2317339698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380102a50 0x7fd380102e30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 -- 172.21.15.103:0/2317339698 >> 172.21.15.103:0/2317339698 conn(0x7fd3800fc4b0 msgr2=0x7fd3800fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 -- 172.21.15.103:0/2317339698 shutdown_connections 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 -- 172.21.15.103:0/2317339698 wait complete. 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 Processor -- start 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.227+0000 7fd387e4b640 1 -- start start 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd387e4b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd380102a50 0x7fd380198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd387e4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380103370 0x7fd380198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd387e4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd38010be80 0x7fd38019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd387e4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd380111b30 con 0x7fd380103370 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd387e4b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801119b0 con 0x7fd380102a50 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd387e4b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd380111cb0 con 0x7fd38010be80 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380103370 0x7fd380198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380103370 0x7fd380198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41942/0 (socket says 172.21.15.103:41942) 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 -- 172.21.15.103:0/3980132753 learned_addr learned my addr 172.21.15.103:0/3980132753 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd385be7640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd380102a50 0x7fd380198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd38010be80 0x7fd38019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 -- 172.21.15.103:0/3980132753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd38010be80 msgr2=0x7fd38019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd38010be80 0x7fd38019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 -- 172.21.15.103:0/3980132753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd380102a50 msgr2=0x7fd380198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd380102a50 0x7fd380198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.228+0000 7fd3853e6640 1 -- 172.21.15.103:0/3980132753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38019cff0 con 0x7fd380103370 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.229+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380103370 0x7fd380198580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd37000d740 tx=0x7fd37000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.229+0000 7fd376ffd640 1 -- 172.21.15.103:0/3980132753 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370017070 con 0x7fd380103370 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.229+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38019d2e0 con 0x7fd380103370 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.229+0000 7fd376ffd640 1 -- 172.21.15.103:0/3980132753 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37000de50 con 0x7fd380103370 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.229+0000 7fd376ffd640 1 -- 172.21.15.103:0/3980132753 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37000f620 con 0x7fd380103370 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.229+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3801a4bc0 con 0x7fd380103370 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd376ffd640 1 -- 172.21.15.103:0/3980132753 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd370009c30 con 0x7fd380103370 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 Processor -- start 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 -- start start 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e7f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7fb30 0x5608f2e89eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608f2de4980 con 0x5608f2e7f210 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5608f2d7ca70 con 0x5608f2e7fb30 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd38a249740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608f2de0940 con 0x5608f2e8a3f0 2024-08-19T01:41:18.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.232+0000 7fd3863e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e7f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e7f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41954/0 (socket says 172.21.15.103:41954) 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 -- 172.21.15.103:0/3635640455 learned_addr learned my addr 172.21.15.103:0/3635640455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd376ffd640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd36c0a2b80 0x7fd36c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd385be7640 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7fb30 0x5608f2e89eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 -- 172.21.15.103:0/3635640455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 msgr2=0x5608f2e8c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 -- 172.21.15.103:0/3635640455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7fb30 msgr2=0x5608f2e89eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7fb30 0x5608f2e89eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 -- 172.21.15.103:0/3635640455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f2ddf6e0 con 0x5608f2e7f210 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd376ffd640 1 -- 172.21.15.103:0/3980132753 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd3700ceb20 con 0x7fd380103370 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:18.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd36c0a2b80 0x7fd36c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e7f5f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd3780072e0 tx=0x7fd37800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e8c560284c98a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.233+0000 7fd36bfff640 1 -- 172.21.15.103:0/3635640455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378019870 con 0x5608f2e7f210 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd36bfff640 1 -- 172.21.15.103:0/3635640455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd378010430 con 0x5608f2e7f210 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd36bfff640 1 -- 172.21.15.103:0/3635640455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37801a610 con 0x5608f2e7f210 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd38a249740 1 -- 172.21.15.103:0/3635640455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 msgr2=0x5608f2e7f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd36c0a2b80 0x7fd36c0a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd378020000 tx=0x7fd37801d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd38a249740 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e7f5f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd3780072e0 tx=0x7fd37800f850 comp rx=0 tx=0).stop 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd38a249740 1 -- 172.21.15.103:0/3635640455 shutdown_connections 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd38a249740 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd38a249740 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7fb30 0x5608f2e89eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd38a249740 1 --2- 172.21.15.103:0/3635640455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e7f5f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.234+0000 7fd38a249740 1 -- 172.21.15.103:0/3635640455 >> 172.21.15.103:0/3635640455 conn(0x5608f2e7b7a0 msgr2=0x5608f2e7bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 -- 172.21.15.103:0/3635640455 shutdown_connections 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 -- 172.21.15.103:0/3635640455 wait complete. 2024-08-19T01:41:18.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 -- start start 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 Processor -- start 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e87300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e87840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2e87d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608f2e8cfc0 con 0x5608f2e7f210 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5608f2d9faa0 con 0x5608f2e8a3f0 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5608f2e8d140 con 0x5608f2e7fb30 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.235+0000 7fd38a249740 10 client.0 _ll_register_callbacks cb 0x5608f2d8ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e87300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e87300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41968/0 (socket says 172.21.15.103:41968) 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 -- 172.21.15.103:0/1733229098 learned_addr learned my addr 172.21.15.103:0/1733229098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd3863e8640 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2e87d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd3853e6640 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e87840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 -- 172.21.15.103:0/1733229098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 msgr2=0x5608f2e87840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e87840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 -- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 msgr2=0x5608f2e87d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2e87d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 -- 172.21.15.103:0/1733229098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f2d83650 con 0x5608f2e7f210 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd385be7640 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e87300 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd37c033000 tx=0x7fd37c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb58946bfb90f894 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.236+0000 7fd3697fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd3697fa640 1 -- 172.21.15.103:0/1733229098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c035040 con 0x5608f2e7f210 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd378010430 con 0x5608f2e7f210 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd3697fa640 1 -- 172.21.15.103:0/1733229098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd37c004250 con 0x5608f2e7f210 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd3697fa640 1 -- 172.21.15.103:0/1733229098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd37c032e10 con 0x5608f2e7f210 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 10 client.16791 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5608f2b563c0 con 0x5608f2e7f210 2024-08-19T01:41:18.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd3697fa640 1 -- 172.21.15.103:0/1733229098 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd378010430 con 0x5608f2e7f210 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd3697fa640 1 -- 172.21.15.103:0/1733229098 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fd37c004b00 con 0x5608f2e7f210 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 10 client.16791 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 20 client.16791 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2e99df0 0x5608f2e9c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 4 client.16791 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.237+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608f2df1ef0 con 0x5608f2e99df0 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.238+0000 7fd3853e6640 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2e99df0 0x5608f2e9c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.238+0000 7fd3697fa640 10 client.16791 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.238+0000 7fd3853e6640 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2e99df0 0x5608f2e9c290 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c7fe4576f0e0e3 server_cookie=aef25f5718dbd4ca in_seq=0 out_seq=0 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd3697fa640 1 -- 172.21.15.103:0/1733229098 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608f2df1ef0 con 0x5608f2e99df0 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd3697fa640 10 client.16791 handle_command_reply: tid=0 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 client.16791 shutdown 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 msgr2=0x5608f2e87300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e87300 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd37c033000 tx=0x7fd37c0316d0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 shutdown_connections 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2e99df0 0x5608f2e9c290 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2e87d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e87840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 --2- 172.21.15.103:0/1733229098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e87300 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 >> 172.21.15.103:0/1733229098 conn(0x5608f2e7b7a0 msgr2=0x5608f2e814d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd3697fa640 0 client.16791 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 shutdown_connections 2024-08-19T01:41:18.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 1 -- 172.21.15.103:0/1733229098 wait complete. 2024-08-19T01:41:18.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.239+0000 7fd38a249740 20 client.16791 trim_cache size 0 max 16384 2024-08-19T01:41:18.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 Processor -- start 2024-08-19T01:41:18.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 -- start start 2024-08-19T01:41:18.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e86be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e88a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2ea1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801ad100 con 0x5608f2e7f210 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801aecd0 con 0x5608f2e8a3f0 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd38a249740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801ad7d0 con 0x5608f2e7fb30 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd385be7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e86be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd385be7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e86be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41984/0 (socket says 172.21.15.103:41984) 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.260+0000 7fd385be7640 1 -- 172.21.15.103:0/3742762427 learned_addr learned my addr 172.21.15.103:0/3742762427 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e88a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2ea1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd385be7640 1 -- 172.21.15.103:0/3742762427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 msgr2=0x5608f2e88a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd385be7640 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e88a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd385be7640 1 -- 172.21.15.103:0/3742762427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 msgr2=0x5608f2ea1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd385be7640 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2ea1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd385be7640 1 -- 172.21.15.103:0/3742762427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f2e89900 con 0x5608f2e7f210 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e88a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd385be7640 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e86be0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd37c004ed0 tx=0x7fd37c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f40c055cb244ea1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd36b7fe640 1 -- 172.21.15.103:0/3742762427 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c035030 con 0x5608f2e7f210 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd36b7fe640 1 -- 172.21.15.103:0/3742762427 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37c040070 con 0x5608f2e7f210 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.261+0000 7fd36b7fe640 1 -- 172.21.15.103:0/3742762427 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0055d0 con 0x5608f2e7f210 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 -- 172.21.15.103:0/3742762427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 msgr2=0x5608f2e86be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e86be0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd37c004ed0 tx=0x7fd37c02a480 comp rx=0 tx=0).stop 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 -- 172.21.15.103:0/3742762427 shutdown_connections 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e8a3f0 0x5608f2ea1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e7fb30 0x5608f2e88a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 --2- 172.21.15.103:0/3742762427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7f210 0x5608f2e86be0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 -- 172.21.15.103:0/3742762427 >> 172.21.15.103:0/3742762427 conn(0x5608f2e7b7a0 msgr2=0x5608f2e81480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 -- 172.21.15.103:0/3742762427 shutdown_connections 2024-08-19T01:41:18.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 -- 172.21.15.103:0/3742762427 wait complete. 2024-08-19T01:41:18.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.262+0000 7fd38a249740 1 -- start start 2024-08-19T01:41:18.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 1 Processor -- start 2024-08-19T01:41:18.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7f210 0x5608f2e92a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7fb30 0x5608f2e8dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801ae5d0 con 0x5608f2e7fb30 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801adea0 con 0x5608f2e7f210 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801ab0e0 con 0x5608f2e8a3f0 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd38a249740 10 client.0 _ll_register_callbacks cb 0x7fd37c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd3853e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7fb30 0x5608f2e8dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd3853e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7fb30 0x5608f2e8dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41992/0 (socket says 172.21.15.103:41992) 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd3853e6640 1 -- 172.21.15.103:0/3190163549 learned_addr learned my addr 172.21.15.103:0/3190163549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.263+0000 7fd3863e8640 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd385be7640 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7f210 0x5608f2e92a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3853e6640 1 -- 172.21.15.103:0/3190163549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 msgr2=0x5608f2e8e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3853e6640 1 -- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7f210 msgr2=0x5608f2e92a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:18.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7f210 0x5608f2e92a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3853e6640 1 -- 172.21.15.103:0/3190163549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f2e8eab0 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3853e6640 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7fb30 0x5608f2e8dd70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3700cc9c0 tx=0x7fd37006d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7233d1a18741fa92 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3697fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3697fa640 1 -- 172.21.15.103:0/3190163549 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370020040 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3697fa640 1 -- 172.21.15.103:0/3190163549 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd37006a0c0 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37c040070 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd3697fa640 1 -- 172.21.15.103:0/3190163549 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd37009f760 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd38a249740 10 client.16797 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.264+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5608f2f04ab0 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd3697fa640 1 -- 172.21.15.103:0/3190163549 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37c040070 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd3697fa640 1 -- 172.21.15.103:0/3190163549 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fd3700ccb10 con 0x5608f2e7fb30 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd38a249740 10 client.16797 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd38a249740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:18.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd38a249740 20 client.16797 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd38a249740 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2f05210 0x5608f2f07630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd38a249740 4 client.16797 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608f2dec400 con 0x5608f2f05210 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.265+0000 7fd385be7640 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2f05210 0x5608f2f07630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd3697fa640 10 client.16797 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd385be7640 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2f05210 0x5608f2f07630 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d102dc5be19a2d7a server_cookie=71c1ca678c993559 in_seq=0 out_seq=0 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd3697fa640 1 -- 172.21.15.103:0/3190163549 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5608f2dec400 con 0x5608f2f05210 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd3697fa640 10 client.16797 handle_command_reply: tid=0 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 client.16797 shutdown 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7fb30 msgr2=0x5608f2e8dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7fb30 0x5608f2e8dd70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3700cc9c0 tx=0x7fd37006d470 comp rx=0 tx=0).stop 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 shutdown_connections 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5608f2f05210 0x5608f2f07630 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5608f2e8a3f0 0x5608f2e8e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd3697fa640 0 client.16797 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f2e7fb30 0x5608f2e8dd70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 --2- 172.21.15.103:0/3190163549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5608f2e7f210 0x5608f2e92a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 >> 172.21.15.103:0/3190163549 conn(0x5608f2e7b7a0 msgr2=0x5608f2e81480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 shutdown_connections 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.266+0000 7fd38a249740 1 -- 172.21.15.103:0/3190163549 wait complete. 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd38a249740 20 client.16797 trim_cache size 0 max 16384 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:41:18.271 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:41:18.272 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:41:18.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd36c0a2b80 msgr2=0x7fd36c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd36c0a2b80 0x7fd36c0a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd378020000 tx=0x7fd37801d000 comp rx=0 tx=0).stop 2024-08-19T01:41:18.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380103370 msgr2=0x7fd380198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:18.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380103370 0x7fd380198580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd37000d740 tx=0x7fd37000dc00 comp rx=0 tx=0).stop 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 shutdown_connections 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd36c0a2b80 0x7fd36c0a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd38010be80 0x7fd38019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd380103370 0x7fd380198580 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 --2- 172.21.15.103:0/3980132753 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd380102a50 0x7fd380198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 >> 172.21.15.103:0/3980132753 conn(0x7fd3800fc4b0 msgr2=0x7fd3800fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 shutdown_connections 2024-08-19T01:41:18.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:18.267+0000 7fd387e4b640 1 -- 172.21.15.103:0/3980132753 wait complete. 2024-08-19T01:41:19.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:41:19.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:41:19.479 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:20.185 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:19 smithi103 ceph-mon[18436]: pgmap v505: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:41:20.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:19 smithi146 ceph-mon[22098]: pgmap v505: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:41:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 -- 172.21.15.103:0/2856566028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 msgr2=0x7fbd4c106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 --2- 172.21.15.103:0/2856566028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c106b70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009930 tx=0x7fbd4802a420 comp rx=0 tx=0).stop 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 -- 172.21.15.103:0/2856566028 shutdown_connections 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 --2- 172.21.15.103:0/2856566028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd4c101130 0x7fbd4c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 --2- 172.21.15.103:0/2856566028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd4c100790 0x7fbd4c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 --2- 172.21.15.103:0/2856566028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c106b70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 -- 172.21.15.103:0/2856566028 >> 172.21.15.103:0/2856566028 conn(0x7fbd4c0fc4b0 msgr2=0x7fbd4c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 -- 172.21.15.103:0/2856566028 shutdown_connections 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.184+0000 7fbd53641640 1 -- 172.21.15.103:0/2856566028 wait complete. 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 Processor -- start 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 -- start start 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd4c100790 0x7fbd4c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd4c101130 0x7fbd4c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06afc0 con 0x7fbd4c106790 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06ae40 con 0x7fbd4c101130 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd53641640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06b140 con 0x7fbd4c100790 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd51bde640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd51bde640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c19c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42002/0 (socket says 172.21.15.103:42002) 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd51bde640 1 -- 172.21.15.103:0/863874882 learned_addr learned my addr 172.21.15.103:0/863874882 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd50bdc640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd4c101130 0x7fbd4c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.185+0000 7fbd513dd640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd4c100790 0x7fbd4c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd51bde640 1 -- 172.21.15.103:0/863874882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd4c100790 msgr2=0x7fbd4c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd51bde640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd4c100790 0x7fbd4c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd51bde640 1 -- 172.21.15.103:0/863874882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd4c101130 msgr2=0x7fbd4c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd51bde640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd4c101130 0x7fbd4c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd51bde640 1 -- 172.21.15.103:0/863874882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c19d0a0 con 0x7fbd4c106790 2024-08-19T01:41:20.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd51bde640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c19c9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400eaa0 tx=0x7fbd4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:20.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd427fc640 1 -- 172.21.15.103:0/863874882 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400cdb0 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c19d390 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd427fc640 1 -- 172.21.15.103:0/863874882 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd44005020 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd427fc640 1 -- 172.21.15.103:0/863874882 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440116d0 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.186+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c1a4c70 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.187+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c100bf0 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.189+0000 7fbd427fc640 1 -- 172.21.15.103:0/863874882 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbd440051d0 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.189+0000 7fbd427fc640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd380a2b80 0x7fbd380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.189+0000 7fbd513dd640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd380a2b80 0x7fbd380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.189+0000 7fbd427fc640 1 -- 172.21.15.103:0/863874882 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbd440cf240 con 0x7fbd4c106790 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.190+0000 7fbd513dd640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd380a2b80 0x7fbd380a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48032ba0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:20.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.194+0000 7fbd427fc640 1 -- 172.21.15.103:0/863874882 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbd44016030 con 0x7fbd4c106790 2024-08-19T01:41:20.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:19 smithi190 ceph-mon[22032]: pgmap v505: 97 pgs: 97 active+clean; 7.7 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:41:20.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.433+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd4c02d5e0 con 0x7fbd4c106790 2024-08-19T01:41:20.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.434+0000 7fbd427fc640 1 -- 172.21.15.103:0/863874882 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11993 (secure 0 0 0) 0x7fbd4409c1d0 con 0x7fbd4c106790 2024-08-19T01:41:20.471 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:41:20.472 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":33,"btime":"2024-08-19T01:41:01:942779+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:41:01.347854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:41:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 33 2024-08-19T01:41:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd380a2b80 msgr2=0x7fbd380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd380a2b80 0x7fbd380a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48032ba0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 msgr2=0x7fbd4c19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c19c9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400eaa0 tx=0x7fbd4400ef60 comp rx=0 tx=0).stop 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 shutdown_connections 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd380a2b80 0x7fbd380a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd4c106790 0x7fbd4c19c9c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd4c101130 0x7fbd4c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 --2- 172.21.15.103:0/863874882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd4c100790 0x7fbd4c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 >> 172.21.15.103:0/863874882 conn(0x7fbd4c0fc4b0 msgr2=0x7fbd4c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 shutdown_connections 2024-08-19T01:41:20.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:20.436+0000 7fbd53641640 1 -- 172.21.15.103:0/863874882 wait complete. 2024-08-19T01:41:21.216 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:20 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/863874882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:41:21.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:41:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:20 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/863874882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:41:21.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:20 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/863874882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:41:21.529 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:22.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.036+0000 7f03c5e97640 1 -- 172.21.15.103:0/1952506871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c0102c20 msgr2=0x7f03c010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.036+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1952506871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c0102c20 0x7f03c010d770 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009ca0 tx=0x7f03b802a950 comp rx=0 tx=0).stop 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 -- 172.21.15.103:0/1952506871 shutdown_connections 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1952506871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c0102c20 0x7f03c010d770 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1952506871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03c01022a0 0x7f03c01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1952506871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03c0100b50 0x7f03c0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 -- 172.21.15.103:0/1952506871 >> 172.21.15.103:0/1952506871 conn(0x7f03c00fc500 msgr2=0x7f03c00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 -- 172.21.15.103:0/1952506871 shutdown_connections 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 -- 172.21.15.103:0/1952506871 wait complete. 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 Processor -- start 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.037+0000 7f03c5e97640 1 -- start start 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03c5e97640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03c0100b50 0x7f03c019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03c5e97640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c01022a0 0x7f03c019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03c5e97640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03c0102c20 0x7f03c019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03c5e97640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0110b60 con 0x7f03c01022a0 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03c5e97640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01109e0 con 0x7f03c0102c20 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03c5e97640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0110ce0 con 0x7f03c0100b50 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c01022a0 0x7f03c019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c01022a0 0x7f03c019a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55994/0 (socket says 172.21.15.103:55994) 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 -- 172.21.15.103:0/1704925842 learned_addr learned my addr 172.21.15.103:0/1704925842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03c0100b50 0x7f03c019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03bffff640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03c0102c20 0x7f03c019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 -- 172.21.15.103:0/1704925842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03c0100b50 msgr2=0x7f03c019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03c0100b50 0x7f03c019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 -- 172.21.15.103:0/1704925842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03c0102c20 msgr2=0x7f03c019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03c0102c20 0x7f03c019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 -- 172.21.15.103:0/1704925842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c019f2a0 con 0x7f03c01022a0 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03bffff640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03c0102c20 0x7f03c019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:22.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.038+0000 7f03beffd640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c01022a0 0x7f03c019a830 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f03b400d950 tx=0x7f03b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.039+0000 7f03bcff9640 1 -- 172.21.15.103:0/1704925842 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4017070 con 0x7f03c01022a0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.039+0000 7f03bcff9640 1 -- 172.21.15.103:0/1704925842 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b4009c30 con 0x7f03c01022a0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.039+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c019f590 con 0x7f03c01022a0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.039+0000 7f03bcff9640 1 -- 172.21.15.103:0/1704925842 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b400f620 con 0x7f03c01022a0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.039+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c01a6e70 con 0x7f03c01022a0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03bcff9640 1 -- 172.21.15.103:0/1704925842 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f03b40109a0 con 0x7f03c01022a0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 Processor -- start 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 -- start start 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d3910 0x55dc667ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667de1d0 0x55dc667e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc66738a40 con 0x55dc667d2ff0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55dc666d09d0 con 0x55dc667de1d0 2024-08-19T01:41:22.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03c8295740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dc66734a00 con 0x55dc667d3910 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56004/0 (socket says 172.21.15.103:56004) 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03bffff640 1 -- 172.21.15.103:0/4134102885 learned_addr learned my addr 172.21.15.103:0/4134102885 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03bf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d3910 0x55dc667ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.042+0000 7f03beffd640 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667de1d0 0x55dc667e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bf7fe640 1 -- 172.21.15.103:0/4134102885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667de1d0 msgr2=0x55dc667e05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bcff9640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f03a80a2b80 0x7f03a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667de1d0 0x55dc667e05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bf7fe640 1 -- 172.21.15.103:0/4134102885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 msgr2=0x55dc667d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bf7fe640 1 -- 172.21.15.103:0/4134102885 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc666d7650 con 0x55dc667d3910 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bffff640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f03a80a2b80 0x7f03a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bcff9640 1 -- 172.21.15.103:0/1704925842 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f03b40cec60 con 0x7f03c01022a0 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d3910 0x55dc667ddc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f03b0004630 tx=0x7f03b000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e874c0bef13c5c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03a5ffb640 1 -- 172.21.15.103:0/4134102885 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0003bd0 con 0x55dc667d3910 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03a5ffb640 1 -- 172.21.15.103:0/4134102885 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b0015070 con 0x55dc667d3910 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.043+0000 7f03a5ffb640 1 -- 172.21.15.103:0/4134102885 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0004d10 con 0x55dc667d3910 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 -- 172.21.15.103:0/4134102885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d3910 msgr2=0x55dc667ddc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03bffff640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f03a80a2b80 0x7f03a80a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005e30 tx=0x7f03b80095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d3910 0x55dc667ddc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f03b0004630 tx=0x7f03b000bb20 comp rx=0 tx=0).stop 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 -- 172.21.15.103:0/4134102885 shutdown_connections 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667de1d0 0x55dc667e05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d3910 0x55dc667ddc90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 --2- 172.21.15.103:0/4134102885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 -- 172.21.15.103:0/4134102885 >> 172.21.15.103:0/4134102885 conn(0x55dc667cf580 msgr2=0x55dc667cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 -- 172.21.15.103:0/4134102885 shutdown_connections 2024-08-19T01:41:22.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 -- 172.21.15.103:0/4134102885 wait complete. 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.044+0000 7f03c8295740 1 -- start start 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 1 Processor -- start 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc667e0da0 con 0x55dc667de1d0 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55dc666f3b20 con 0x55dc667d3910 2024-08-19T01:41:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dc667e0f20 con 0x55dc667d2ff0 2024-08-19T01:41:22.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03c8295740 10 client.0 _ll_register_callbacks cb 0x55dc666f1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667dba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56018/0 (socket says 172.21.15.103:56018) 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03beffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667db550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:36086/0 (socket says 172.21.15.103:36086) 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03bffff640 1 -- 172.21.15.103:0/683235336 learned_addr learned my addr 172.21.15.103:0/683235336 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.045+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03bffff640 1 -- 172.21.15.103:0/683235336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 msgr2=0x55dc667db010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03bffff640 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667db010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03bffff640 1 -- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 msgr2=0x55dc667db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03bffff640 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03bffff640 1 -- 172.21.15.103:0/683235336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc664aa3c0 con 0x55dc667de1d0 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03beffd640 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:22.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03bffff640 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667dba90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009ca0 tx=0x7f03b802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6be486832113059 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f039effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f039effd640 1 -- 172.21.15.103:0/683235336 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8034040 con 0x55dc667de1d0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f039effd640 1 -- 172.21.15.103:0/683235336 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f03b8030680 con 0x55dc667de1d0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b0015070 con 0x55dc667de1d0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f039effd640 1 -- 172.21.15.103:0/683235336 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f03b8057020 con 0x55dc667de1d0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03c8295740 10 client.16815 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.046+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc667ec940 con 0x55dc667de1d0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f039effd640 1 -- 172.21.15.103:0/683235336 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b0015070 con 0x55dc667de1d0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f039effd640 1 -- 172.21.15.103:0/683235336 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f03b8004460 con 0x55dc667de1d0 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f03c8295740 10 client.16815 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f03c8295740 10 client.16815 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f03c8295740 20 client.16815 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:22.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f03c8295740 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc667ede00 0x55dc667f0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f03c8295740 4 client.16815 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc66745fb0 con 0x55dc667ede00 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.047+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc667ede00 0x55dc667f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.048+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc667ede00 0x55dc667f0250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec3384e9a3ebac6a server_cookie=fe367f84df05c257 in_seq=0 out_seq=0 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.048+0000 7f039effd640 10 client.16815 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.048+0000 7f039effd640 1 -- 172.21.15.103:0/683235336 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc66745fb0 con 0x55dc667ede00 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.048+0000 7f039effd640 10 client.16815 handle_command_reply: tid=0 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.048+0000 7f03c8295740 1 client.16815 shutdown 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 msgr2=0x55dc667dba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667dba90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009ca0 tx=0x7f03b802a950 comp rx=0 tx=0).stop 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 shutdown_connections 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc667ede00 0x55dc667f0250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667dba90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 --2- 172.21.15.103:0/683235336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667db010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 >> 172.21.15.103:0/683235336 conn(0x55dc667cf580 msgr2=0x55dc667d52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f039effd640 0 client.16815 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 shutdown_connections 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 1 -- 172.21.15.103:0/683235336 wait complete. 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.049+0000 7f03c8295740 20 client.16815 trim_cache size 0 max 16384 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 Processor -- start 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 -- start start 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01af2f0 con 0x55dc667de1d0 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01b0ec0 con 0x55dc667d3910 2024-08-19T01:41:22.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03c8295740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01af9c0 con 0x55dc667d2ff0 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.064+0000 7f03bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667d98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56030/0 (socket says 172.21.15.103:56030) 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bffff640 1 -- 172.21.15.103:0/2991152426 learned_addr learned my addr 172.21.15.103:0/2991152426 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03beffd640 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bffff640 1 -- 172.21.15.103:0/2991152426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 msgr2=0x55dc667f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bffff640 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bffff640 1 -- 172.21.15.103:0/2991152426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 msgr2=0x55dc667f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bffff640 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bffff640 1 -- 172.21.15.103:0/2991152426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc667d9560 con 0x55dc667de1d0 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03bffff640 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667d98e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006010 tx=0x7f03b803a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e362dcd953bf24 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03a57fa640 1 -- 172.21.15.103:0/2991152426 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8061070 con 0x55dc667de1d0 2024-08-19T01:41:22.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03a57fa640 1 -- 172.21.15.103:0/2991152426 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b803add0 con 0x55dc667de1d0 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.065+0000 7f03a57fa640 1 -- 172.21.15.103:0/2991152426 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b805c6e0 con 0x55dc667de1d0 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 -- 172.21.15.103:0/2991152426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 msgr2=0x55dc667d98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667d98e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006010 tx=0x7f03b803a720 comp rx=0 tx=0).stop 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 -- 172.21.15.103:0/2991152426 shutdown_connections 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667de1d0 0x55dc667d98e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 --2- 172.21.15.103:0/2991152426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667d2ff0 0x55dc667f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 -- 172.21.15.103:0/2991152426 >> 172.21.15.103:0/2991152426 conn(0x55dc667cf580 msgr2=0x55dc667ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 -- 172.21.15.103:0/2991152426 shutdown_connections 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.066+0000 7f03c8295740 1 -- 172.21.15.103:0/2991152426 wait complete. 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 -- start start 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 Processor -- start 2024-08-19T01:41:22.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667de1d0 0x55dc667e1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01b07c0 con 0x55dc667d2ff0 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01b0090 con 0x55dc667d3910 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01ad2d0 con 0x55dc667de1d0 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03c8295740 10 client.0 _ll_register_callbacks cb 0x7f03b8009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667e1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56036/0 (socket says 172.21.15.103:56036) 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03bf7fe640 1 -- 172.21.15.103:0/580951760 learned_addr learned my addr 172.21.15.103:0/580951760 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03beffd640 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.067+0000 7f03bffff640 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667de1d0 0x55dc667e1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03bf7fe640 1 -- 172.21.15.103:0/580951760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667de1d0 msgr2=0x55dc667e1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667de1d0 0x55dc667e1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03bf7fe640 1 -- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 msgr2=0x55dc667e16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667e16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03bf7fe640 1 -- 172.21.15.103:0/580951760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc66856be0 con 0x55dc667d2ff0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03bf7fe640 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667e1160 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f03b0000c00 tx=0x7f03b0004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd75905a2cdd7d50 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f039effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f039effd640 1 -- 172.21.15.103:0/580951760 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0034060 con 0x55dc667d2ff0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f039effd640 1 -- 172.21.15.103:0/580951760 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f03b002f440 con 0x55dc667d2ff0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b803add0 con 0x55dc667d2ff0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f039effd640 1 -- 172.21.15.103:0/580951760 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f03b0003fe0 con 0x55dc667d2ff0 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03c8295740 10 client.16821 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:22.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.068+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc66857ec0 con 0x55dc667d2ff0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f039effd640 1 -- 172.21.15.103:0/580951760 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b803add0 con 0x55dc667d2ff0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f039effd640 1 -- 172.21.15.103:0/580951760 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f03b00116a0 con 0x55dc667d2ff0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f03c8295740 10 client.16821 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f03c8295740 10 client.16821 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f03c8295740 20 client.16821 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f03c8295740 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc668582b0 0x55dc6685a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f03c8295740 4 client.16821 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc667404c0 con 0x55dc668582b0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.069+0000 7f03beffd640 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc668582b0 0x55dc6685a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03beffd640 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc668582b0 0x55dc6685a6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d439b580e092e478 server_cookie=75fa533eb2f70e79 in_seq=0 out_seq=0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f039effd640 10 client.16821 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f039effd640 1 -- 172.21.15.103:0/580951760 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc667404c0 con 0x55dc668582b0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f039effd640 10 client.16821 handle_command_reply: tid=0 2024-08-19T01:41:22.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 client.16821 shutdown 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 msgr2=0x55dc667e1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667e1160 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f03b0000c00 tx=0x7f03b0004870 comp rx=0 tx=0).stop 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 shutdown_connections 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55dc668582b0 0x55dc6685a6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dc667de1d0 0x55dc667e1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dc667d3910 0x55dc667e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 --2- 172.21.15.103:0/580951760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc667d2ff0 0x55dc667e1160 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 >> 172.21.15.103:0/580951760 conn(0x55dc667cf580 msgr2=0x55dc667d98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f039effd640 0 client.16821 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 shutdown_connections 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.070+0000 7f03c8295740 1 -- 172.21.15.103:0/580951760 wait complete. 2024-08-19T01:41:22.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c8295740 20 client.16821 trim_cache size 0 max 16384 2024-08-19T01:41:22.083 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f03a80a2b80 msgr2=0x7f03a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f03a80a2b80 0x7f03a80a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005e30 tx=0x7f03b80095c0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c01022a0 msgr2=0x7f03c019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c01022a0 0x7f03c019a830 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f03b400d950 tx=0x7f03b400de10 comp rx=0 tx=0).stop 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 shutdown_connections 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f03a80a2b80 0x7f03a80a5040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03c0102c20 0x7f03c019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c01022a0 0x7f03c019a830 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 --2- 172.21.15.103:0/1704925842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03c0100b50 0x7f03c019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.071+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 >> 172.21.15.103:0/1704925842 conn(0x7f03c00fc500 msgr2=0x7f03c006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.072+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 shutdown_connections 2024-08-19T01:41:22.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:22.072+0000 7f03c5e97640 1 -- 172.21.15.103:0/1704925842 wait complete. 2024-08-19T01:41:22.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:21 smithi103 ceph-mon[18436]: pgmap v506: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:41:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:21 smithi190 ceph-mon[22032]: pgmap v506: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:41:22.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:21 smithi146 ceph-mon[22098]: pgmap v506: 97 pgs: 97 active+clean; 7.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:41:22.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 -- 172.21.15.103:0/2226474698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880101960 msgr2=0x7fa880101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 --2- 172.21.15.103:0/2226474698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880101960 0x7fa880101de0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa874009900 tx=0x7fa87402a410 comp rx=0 tx=0).stop 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 -- 172.21.15.103:0/2226474698 shutdown_connections 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 --2- 172.21.15.103:0/2226474698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa880102320 0x7fa880108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 --2- 172.21.15.103:0/2226474698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880101960 0x7fa880101de0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 --2- 172.21.15.103:0/2226474698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa880100770 0x7fa880100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 -- 172.21.15.103:0/2226474698 >> 172.21.15.103:0/2226474698 conn(0x7fa8800fbf10 msgr2=0x7fa8800fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 -- 172.21.15.103:0/2226474698 shutdown_connections 2024-08-19T01:41:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.915+0000 7fa8851e1640 1 -- 172.21.15.103:0/2226474698 wait complete. 2024-08-19T01:41:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 Processor -- start 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 -- start start 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880100770 0x7fa88006a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa880101960 0x7fa88019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa880102320 0x7fa88019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010c2a0 con 0x7fa880100770 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010c120 con 0x7fa880101960 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa8851e1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010c420 con 0x7fa880102320 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa87ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880100770 0x7fa88006a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa87ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880100770 0x7fa88006a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56056/0 (socket says 172.21.15.103:56056) 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa87ed9d640 1 -- 172.21.15.103:0/16637349 learned_addr learned my addr 172.21.15.103:0/16637349 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa87e59c640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa880101960 0x7fa88019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.916+0000 7fa87f59e640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa880102320 0x7fa88019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa87ed9d640 1 -- 172.21.15.103:0/16637349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa880102320 msgr2=0x7fa88019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa880102320 0x7fa88019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa87ed9d640 1 -- 172.21.15.103:0/16637349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa880101960 msgr2=0x7fa88019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa880101960 0x7fa88019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa87ed9d640 1 -- 172.21.15.103:0/16637349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88019f610 con 0x7fa880100770 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880100770 0x7fa88006a140 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c970 tx=0x7fa87000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa867fff640 1 -- 172.21.15.103:0/16637349 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870007c10 con 0x7fa880100770 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa867fff640 1 -- 172.21.15.103:0/16637349 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa870007db0 con 0x7fa880100770 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88019f8a0 con 0x7fa880100770 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa867fff640 1 -- 172.21.15.103:0/16637349 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000f870 con 0x7fa880100770 2024-08-19T01:41:22.920 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.917+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88019fe10 con 0x7fa880100770 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 Processor -- start 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 -- start start 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe3fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7fe42510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb7fd9d9d0 con 0x55fb7fe40050 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fb7fd45820 con 0x55fb7fe3a680 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa888078740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fb7fd99990 con 0x55fb7fe3afc0 2024-08-19T01:41:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa867fff640 1 -- 172.21.15.103:0/16637349 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa87000fa10 con 0x7fa880100770 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7fe42510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7fe42510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56064/0 (socket says 172.21.15.103:56064) 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 -- 172.21.15.103:0/2972050761 learned_addr learned my addr 172.21.15.103:0/2972050761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87f59e640 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 -- 172.21.15.103:0/2972050761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 msgr2=0x55fb7fe3fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe3fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 -- 172.21.15.103:0/2972050761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 msgr2=0x55fb7fe3aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.920+0000 7fa87e59c640 1 -- 172.21.15.103:0/2972050761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb7fab1210 con 0x55fb7fe40050 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.921+0000 7fa87e59c640 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7fe42510 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa874009900 tx=0x7fa874032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac1c8e985d43f27 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.921+0000 7fa8657fa640 1 -- 172.21.15.103:0/2972050761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874035070 con 0x55fb7fe40050 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.921+0000 7fa867fff640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8680a2cb0 0x7fa8680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.921+0000 7fa8657fa640 1 -- 172.21.15.103:0/2972050761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa874031780 con 0x55fb7fe40050 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.921+0000 7fa8657fa640 1 -- 172.21.15.103:0/2972050761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874045470 con 0x55fb7fe40050 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.921+0000 7fa87f59e640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8680a2cb0 0x7fa8680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.921+0000 7fa867fff640 1 -- 172.21.15.103:0/16637349 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa8700cf2a0 con 0x7fa880100770 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 -- 172.21.15.103:0/2972050761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 msgr2=0x55fb7fe42510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa87f59e640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8680a2cb0 0x7fa8680a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa878008e40 tx=0x7fa8780085f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7fe42510 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa874009900 tx=0x7fa874032e10 comp rx=0 tx=0).stop 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 -- 172.21.15.103:0/2972050761 shutdown_connections 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7fe42510 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe3fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 --2- 172.21.15.103:0/2972050761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 -- 172.21.15.103:0/2972050761 >> 172.21.15.103:0/2972050761 conn(0x55fb7fe35cf0 msgr2=0x55fb7fe360f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 -- 172.21.15.103:0/2972050761 shutdown_connections 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 -- 172.21.15.103:0/2972050761 wait complete. 2024-08-19T01:41:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 -- start start 2024-08-19T01:41:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.922+0000 7fa888078740 1 Processor -- start 2024-08-19T01:41:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa888078740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa888078740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe44e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa888078740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe4be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa888078740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb7fe43010 con 0x55fb7fe3afc0 2024-08-19T01:41:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa888078740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fb7fd59520 con 0x55fb7fe40050 2024-08-19T01:41:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa888078740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fb7fe43190 con 0x55fb7fe3a680 2024-08-19T01:41:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa888078740 10 client.0 _ll_register_callbacks cb 0x55fb7fab0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:60658/0 (socket says 172.21.15.103:60658) 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 -- 172.21.15.103:0/1194347608 learned_addr learned my addr 172.21.15.103:0/1194347608 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87e59c640 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe44e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 -- 172.21.15.103:0/1194347608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 msgr2=0x55fb7fe4be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe4be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 -- 172.21.15.103:0/1194347608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 msgr2=0x55fb7fe44e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe44e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87ed9d640 1 -- 172.21.15.103:0/1194347608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb7fb7d620 con 0x55fb7fe3a680 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.923+0000 7fa87e59c640 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe44e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe448c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c940 tx=0x7fa870018b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac652fe59cffc06 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa862ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:41:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa862ff5640 1 -- 172.21.15.103:0/1194347608 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87009b210 con 0x55fb7fe3a680 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa862ff5640 1 -- 172.21.15.103:0/1194347608 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa8700a2170 con 0x55fb7fe3a680 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa874031780 con 0x55fb7fe3a680 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa862ff5640 1 -- 172.21.15.103:0/1194347608 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa87009ff70 con 0x55fb7fe3a680 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa888078740 10 client.26146 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fb7fe4ed50 con 0x55fb7fe3a680 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa862ff5640 1 -- 172.21.15.103:0/1194347608 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa874031780 con 0x55fb7fe3a680 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.924+0000 7fa862ff5640 1 -- 172.21.15.103:0/1194347608 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fa870020030 con 0x55fb7fe3a680 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa888078740 10 client.26146 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa888078740 10 client.26146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa888078740 20 client.26146 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa888078740 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7fe50390 0x55fb7fe527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa888078740 4 client.26146 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb7fdab100 con 0x55fb7fe50390 2024-08-19T01:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa87e59c640 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7fe50390 0x55fb7fe527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa862ff5640 10 client.26146 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.925+0000 7fa87e59c640 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7fe50390 0x55fb7fe527a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5158564f37b4b9df server_cookie=a8fe605184d44698 in_seq=0 out_seq=0 2024-08-19T01:41:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa862ff5640 1 -- 172.21.15.103:0/1194347608 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb7fdab100 con 0x55fb7fe50390 2024-08-19T01:41:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa862ff5640 10 client.26146 handle_command_reply: tid=0 2024-08-19T01:41:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 client.26146 shutdown 2024-08-19T01:41:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 msgr2=0x55fb7fe448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe448c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c940 tx=0x7fa870018b50 comp rx=0 tx=0).stop 2024-08-19T01:41:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 shutdown_connections 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7fe50390 0x55fb7fe527a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe4be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe44e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 --2- 172.21.15.103:0/1194347608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe448c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 >> 172.21.15.103:0/1194347608 conn(0x55fb7fe35cf0 msgr2=0x55fb7fe37500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa862ff5640 0 client.26146 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 shutdown_connections 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 1 -- 172.21.15.103:0/1194347608 wait complete. 2024-08-19T01:41:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.926+0000 7fa888078740 20 client.26146 trim_cache size 0 max 16384 2024-08-19T01:41:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 Processor -- start 2024-08-19T01:41:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 -- start start 2024-08-19T01:41:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe4be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe44670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a89f0 con 0x55fb7fe3afc0 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801aa5c0 con 0x55fb7fe40050 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa888078740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a90c0 con 0x55fb7fe3a680 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa87e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe4be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa87e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe4be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56096/0 (socket says 172.21.15.103:56096) 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa87e59c640 1 -- 172.21.15.103:0/3911915089 learned_addr learned my addr 172.21.15.103:0/3911915089 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa87f59e640 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe44670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa87e59c640 1 -- 172.21.15.103:0/3911915089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 msgr2=0x55fb7fe3b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.942+0000 7fa87e59c640 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa87e59c640 1 -- 172.21.15.103:0/3911915089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 msgr2=0x55fb7fe44670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa87e59c640 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe44670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa87e59c640 1 -- 172.21.15.103:0/3911915089 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb7fe35b10 con 0x55fb7fe3afc0 2024-08-19T01:41:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa87f59e640 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe44670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa87e59c640 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe4be00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa874009900 tx=0x7fa87402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3a17500ebc7fee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa864ff9640 1 -- 172.21.15.103:0/3911915089 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874035070 con 0x55fb7fe3afc0 2024-08-19T01:41:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa864ff9640 1 -- 172.21.15.103:0/3911915089 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa87404b040 con 0x55fb7fe3afc0 2024-08-19T01:41:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.943+0000 7fa864ff9640 1 -- 172.21.15.103:0/3911915089 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874004920 con 0x55fb7fe3afc0 2024-08-19T01:41:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 -- 172.21.15.103:0/3911915089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 msgr2=0x55fb7fe4be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe4be00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa874009900 tx=0x7fa87402a410 comp rx=0 tx=0).stop 2024-08-19T01:41:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 -- 172.21.15.103:0/3911915089 shutdown_connections 2024-08-19T01:41:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe40050 0x55fb7fe44670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe4be00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 --2- 172.21.15.103:0/3911915089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3a680 0x55fb7fe3b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 -- 172.21.15.103:0/3911915089 >> 172.21.15.103:0/3911915089 conn(0x55fb7fe35cf0 msgr2=0x55fb7fe3c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 -- 172.21.15.103:0/3911915089 shutdown_connections 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 -- 172.21.15.103:0/3911915089 wait complete. 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 -- start start 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.944+0000 7fa888078740 1 Processor -- start 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe48ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe48ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7feb9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a9ec0 con 0x55fb7fe40050 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a9790 con 0x55fb7fe3a680 2024-08-19T01:41:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a69d0 con 0x55fb7fe3afc0 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 10 client.0 _ll_register_callbacks cb 0x7fa870018d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7feb9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7feb9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56112/0 (socket says 172.21.15.103:56112) 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 -- 172.21.15.103:0/2482906027 learned_addr learned my addr 172.21.15.103:0/2482906027 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87e59c640 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe48ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe48ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 -- 172.21.15.103:0/2482906027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 msgr2=0x55fb7fe48ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe48ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 -- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 msgr2=0x55fb7fe48ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe48ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 -- 172.21.15.103:0/2482906027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb7fe49630 con 0x55fb7fe40050 2024-08-19T01:41:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa87f59e640 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7feb9070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa87801ee20 tx=0x7fa878004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2254c16e4b7bda4f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa862ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa862ff5640 1 -- 172.21.15.103:0/2482906027 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87800edd0 con 0x55fb7fe40050 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa862ff5640 1 -- 172.21.15.103:0/2482906027 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa8780283f0 con 0x55fb7fe40050 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.945+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa87404b040 con 0x55fb7fe40050 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa862ff5640 1 -- 172.21.15.103:0/2482906027 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8780285f0 con 0x55fb7fe40050 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 10 client.16839 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fb7feba740 con 0x55fb7fe40050 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa862ff5640 1 -- 172.21.15.103:0/2482906027 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa87404b040 con 0x55fb7fe40050 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa862ff5640 1 -- 172.21.15.103:0/2482906027 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fa8780348b0 con 0x55fb7fe40050 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 10 client.16839 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 20 client.16839 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7febb150 0x55fb7febd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 4 client.16839 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:41:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb7fda5450 con 0x55fb7febb150 2024-08-19T01:41:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.946+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7febb150 0x55fb7febd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.947+0000 7fa862ff5640 10 client.16839 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.947+0000 7fa87ed9d640 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7febb150 0x55fb7febd570 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a6dadf303e1684 server_cookie=d528e29609c7ac7f in_seq=0 out_seq=0 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa862ff5640 1 -- 172.21.15.103:0/2482906027 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb7fda5450 con 0x55fb7febb150 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa862ff5640 10 client.16839 handle_command_reply: tid=0 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 client.16839 shutdown 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 msgr2=0x55fb7feb9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7feb9070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa87801ee20 tx=0x7fa878004ab0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 shutdown_connections 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fb7febb150 0x55fb7febd570 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb7fe40050 0x55fb7feb9070 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa862ff5640 0 client.16839 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fb7fe3afc0 0x55fb7fe48ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.954+0000 7fa888078740 1 --2- 172.21.15.103:0/2482906027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb7fe3a680 0x55fb7fe48ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 >> 172.21.15.103:0/2482906027 conn(0x55fb7fe35cf0 msgr2=0x55fb7fe34680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 shutdown_connections 2024-08-19T01:41:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa888078740 1 -- 172.21.15.103:0/2482906027 wait complete. 2024-08-19T01:41:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa888078740 20 client.16839 trim_cache size 0 max 16384 2024-08-19T01:41:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:41:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:41:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:41:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8680a2cb0 msgr2=0x7fa8680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8680a2cb0 0x7fa8680a5170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa878008e40 tx=0x7fa8780085f0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880100770 msgr2=0x7fa88006a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880100770 0x7fa88006a140 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c970 tx=0x7fa87000ce30 comp rx=0 tx=0).stop 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 shutdown_connections 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa8680a2cb0 0x7fa8680a5170 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa880102320 0x7fa88019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa880101960 0x7fa88019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 --2- 172.21.15.103:0/16637349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa880100770 0x7fa88006a140 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.955+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 >> 172.21.15.103:0/16637349 conn(0x7fa8800fbf10 msgr2=0x7fa880104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.956+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 shutdown_connections 2024-08-19T01:41:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:22.956+0000 7fa8851e1640 1 -- 172.21.15.103:0/16637349 wait complete. 2024-08-19T01:41:23.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:22 smithi103 ceph-mon[18436]: pgmap v507: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-08-19T01:41:23.119 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:22 smithi190 ceph-mon[22032]: pgmap v507: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-08-19T01:41:23.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:22 smithi146 ceph-mon[22098]: pgmap v507: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-08-19T01:41:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 -- 172.21.15.103:0/308965265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668102c20 msgr2=0x7f266810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 --2- 172.21.15.103:0/308965265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668102c20 0x7f266810d770 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2660009ca0 tx=0x7f266002a950 comp rx=0 tx=0).stop 2024-08-19T01:41:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 -- 172.21.15.103:0/308965265 shutdown_connections 2024-08-19T01:41:23.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 --2- 172.21.15.103:0/308965265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668102c20 0x7f266810d770 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 --2- 172.21.15.103:0/308965265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26681022a0 0x7f26681026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 --2- 172.21.15.103:0/308965265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2668100b50 0x7f2668100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 -- 172.21.15.103:0/308965265 >> 172.21.15.103:0/308965265 conn(0x7f26680fc500 msgr2=0x7f26680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 -- 172.21.15.103:0/308965265 shutdown_connections 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.665+0000 7f266e1e9640 1 -- 172.21.15.103:0/308965265 wait complete. 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 Processor -- start 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 -- start start 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668100b50 0x7f2668198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26681022a0 0x7f26681993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2668102c20 0x7f26681a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2668110b60 con 0x7f2668100b50 2024-08-19T01:41:23.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26681109e0 con 0x7f2668102c20 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.666+0000 7f266e1e9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2668110ce0 con 0x7f26681022a0 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668100b50 0x7f2668198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668100b50 0x7f2668198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56124/0 (socket says 172.21.15.103:56124) 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 -- 172.21.15.103:0/62114488 learned_addr learned my addr 172.21.15.103:0/62114488 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f2667fff640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2668102c20 0x7f26681a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f2666ffd640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26681022a0 0x7f26681993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 -- 172.21.15.103:0/62114488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26681022a0 msgr2=0x7f26681993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26681022a0 0x7f26681993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 -- 172.21.15.103:0/62114488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2668102c20 msgr2=0x7f26681a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2668102c20 0x7f26681a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 -- 172.21.15.103:0/62114488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26681a4470 con 0x7f2668100b50 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f26677fe640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668100b50 0x7f2668198eb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f265800e9f0 tx=0x7f265800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:23.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f2664ff9640 1 -- 172.21.15.103:0/62114488 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265800cd80 con 0x7f2668100b50 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f2664ff9640 1 -- 172.21.15.103:0/62114488 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26580108e0 con 0x7f2668100b50 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26681a4640 con 0x7f2668100b50 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f2664ff9640 1 -- 172.21.15.103:0/62114488 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658018920 con 0x7f2668100b50 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.667+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681a4b50 con 0x7f2668100b50 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.670+0000 7f2664ff9640 1 -- 172.21.15.103:0/62114488 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2658010a90 con 0x7f2668100b50 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.670+0000 7f26705e7740 1 Processor -- start 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.670+0000 7f26705e7740 1 -- start start 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f26705e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e18b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f26705e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e195c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f26705e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e1961d0 0x55a36e1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f26705e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a36e0f0a40 con 0x55a36e1961d0 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f26705e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a36e0889d0 con 0x55a36e18b910 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f26705e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a36e0eca00 con 0x55a36e18aff0 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e18b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e18b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:60688/0 (socket says 172.21.15.103:60688) 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 -- 172.21.15.103:0/917659079 learned_addr learned my addr 172.21.15.103:0/917659079 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2666ffd640 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e1961d0 0x55a36e1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f26677fe640 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2664ff9640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f26500a2b80 0x7f26500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 -- 172.21.15.103:0/917659079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 msgr2=0x55a36e195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 -- 172.21.15.103:0/917659079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e1961d0 msgr2=0x55a36e1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e1961d0 0x55a36e1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 -- 172.21.15.103:0/917659079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36e08f650 con 0x55a36e18aff0 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2664ff9640 1 -- 172.21.15.103:0/62114488 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f26580cf390 con 0x7f2668100b50 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.671+0000 7f2667fff640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f26500a2b80 0x7f26500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.672+0000 7f2667fff640 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e18b3d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2660035000 tx=0x7f266002fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b1b746aa2b6b34 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.672+0000 7f264dffb640 1 -- 172.21.15.103:0/917659079 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660034040 con 0x55a36e18aff0 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.672+0000 7f2667fff640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f26500a2b80 0x7f26500a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f266004d000 tx=0x7f266004a000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.672+0000 7f264dffb640 1 -- 172.21.15.103:0/917659079 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f266002fa60 con 0x55a36e18aff0 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.672+0000 7f264dffb640 1 -- 172.21.15.103:0/917659079 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660046760 con 0x55a36e18aff0 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 -- 172.21.15.103:0/917659079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 msgr2=0x55a36e18b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e18b3d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2660035000 tx=0x7f266002fe60 comp rx=0 tx=0).stop 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 -- 172.21.15.103:0/917659079 shutdown_connections 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e1961d0 0x55a36e1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 --2- 172.21.15.103:0/917659079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e18b3d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 -- 172.21.15.103:0/917659079 >> 172.21.15.103:0/917659079 conn(0x55a36e187580 msgr2=0x55a36e187960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 -- 172.21.15.103:0/917659079 shutdown_connections 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 -- 172.21.15.103:0/917659079 wait complete. 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 -- start start 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.673+0000 7f26705e7740 1 Processor -- start 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26705e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e193070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26705e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18b910 0x55a36e1935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26705e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e193af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26705e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a36e198da0 con 0x55a36e18b910 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26705e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a36e0abb20 con 0x55a36e1961d0 2024-08-19T01:41:23.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26705e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a36e198f20 con 0x55a36e18aff0 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26705e7740 10 client.0 _ll_register_callbacks cb 0x55a36e0a9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18b910 0x55a36e1935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18b910 0x55a36e1935b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56150/0 (socket says 172.21.15.103:56150) 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 -- 172.21.15.103:0/494185953 learned_addr learned my addr 172.21.15.103:0/494185953 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f26677fe640 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e193070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2667fff640 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e193af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 -- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 msgr2=0x55a36e193070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e193070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 -- 172.21.15.103:0/494185953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 msgr2=0x55a36e193af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e193af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 -- 172.21.15.103:0/494185953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36de623c0 con 0x55a36e18b910 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2666ffd640 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18b910 0x55a36e1935b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f265c00cd90 tx=0x7f265c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8172f12f3d3faa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:23.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.674+0000 7f2646ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f2646ffd640 1 -- 172.21.15.103:0/494185953 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c011910 con 0x55a36e18b910 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f2646ffd640 1 -- 172.21.15.103:0/494185953 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f265c011ab0 con 0x55a36e18b910 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a36e0fdfb0 con 0x55a36e18b910 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f2646ffd640 1 -- 172.21.15.103:0/494185953 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f265c011cb0 con 0x55a36e18b910 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f26705e7740 10 client.16851 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a36e1a49d0 con 0x55a36e18b910 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f2646ffd640 1 -- 172.21.15.103:0/494185953 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a36e0fdfb0 con 0x55a36e18b910 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f2646ffd640 1 -- 172.21.15.103:0/494185953 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f265c018070 con 0x55a36e18b910 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f26705e7740 10 client.16851 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.675+0000 7f26705e7740 10 client.16851 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.676+0000 7f26705e7740 20 client.16851 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.676+0000 7f26705e7740 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e1a5f30 0x55a36e1a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.676+0000 7f26705e7740 4 client.16851 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:23.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.676+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a36e0f84c0 con 0x55a36e1a5f30 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.676+0000 7f26677fe640 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e1a5f30 0x55a36e1a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.676+0000 7f26677fe640 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e1a5f30 0x55a36e1a8350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d154236690731656 server_cookie=3b8ede44d30a4f56 in_seq=0 out_seq=0 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.676+0000 7f2646ffd640 10 client.16851 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f2646ffd640 1 -- 172.21.15.103:0/494185953 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a36e0f84c0 con 0x55a36e1a5f30 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f2646ffd640 10 client.16851 handle_command_reply: tid=0 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 client.16851 shutdown 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18b910 msgr2=0x55a36e1935b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18b910 0x55a36e1935b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f265c00cd90 tx=0x7f265c0039c0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 shutdown_connections 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e1a5f30 0x55a36e1a8350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e193af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18b910 0x55a36e1935b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 --2- 172.21.15.103:0/494185953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18aff0 0x55a36e193070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 >> 172.21.15.103:0/494185953 conn(0x55a36e187580 msgr2=0x55a36e18d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:23.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.677+0000 7f2646ffd640 0 client.16851 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:41:23.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.678+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 shutdown_connections 2024-08-19T01:41:23.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.678+0000 7f26705e7740 1 -- 172.21.15.103:0/494185953 wait complete. 2024-08-19T01:41:23.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.678+0000 7f26705e7740 20 client.16851 trim_cache size 0 max 16384 2024-08-19T01:41:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 Processor -- start 2024-08-19T01:41:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 -- start start 2024-08-19T01:41:23.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e193940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e1adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e1961d0 0x55a36e1ae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26681ad0e0 con 0x55a36e18aff0 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26681aecb0 con 0x55a36e18b910 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26705e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f26681ad7b0 con 0x55a36e1961d0 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e193940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e193940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56156/0 (socket says 172.21.15.103:56156) 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f26677fe640 1 -- 172.21.15.103:0/3045035951 learned_addr learned my addr 172.21.15.103:0/3045035951 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f2667fff640 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e1961d0 0x55a36e1ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.698+0000 7f2666ffd640 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e1adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f26677fe640 1 -- 172.21.15.103:0/3045035951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e1961d0 msgr2=0x55a36e1ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f26677fe640 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e1961d0 0x55a36e1ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f26677fe640 1 -- 172.21.15.103:0/3045035951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 msgr2=0x55a36e1adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f26677fe640 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e1adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f26677fe640 1 -- 172.21.15.103:0/3045035951 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36e1873b0 con 0x55a36e18aff0 2024-08-19T01:41:23.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f26677fe640 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e193940 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f265800c6a0 tx=0x7f26580ccd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe84d4878b6599 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:23.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f264d7fa640 1 -- 172.21.15.103:0/3045035951 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265809ffd0 con 0x55a36e18aff0 2024-08-19T01:41:23.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f264d7fa640 1 -- 172.21.15.103:0/3045035951 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26580a4890 con 0x55a36e18aff0 2024-08-19T01:41:23.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.699+0000 7f264d7fa640 1 -- 172.21.15.103:0/3045035951 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265809b360 con 0x55a36e18aff0 2024-08-19T01:41:23.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 -- 172.21.15.103:0/3045035951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 msgr2=0x55a36e193940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e193940 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f265800c6a0 tx=0x7f26580ccd90 comp rx=0 tx=0).stop 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 -- 172.21.15.103:0/3045035951 shutdown_connections 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e1961d0 0x55a36e1ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e18b910 0x55a36e1adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 --2- 172.21.15.103:0/3045035951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e193940 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 -- 172.21.15.103:0/3045035951 >> 172.21.15.103:0/3045035951 conn(0x55a36e187580 msgr2=0x55a36e194fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 -- 172.21.15.103:0/3045035951 shutdown_connections 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 -- 172.21.15.103:0/3045035951 wait complete. 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.700+0000 7f26705e7740 1 -- start start 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 1 Processor -- start 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e208f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18b910 0x55a36e209470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e2099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26681ae5b0 con 0x55a36e18aff0 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f26681ade80 con 0x55a36e1961d0 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f26681ab0c0 con 0x55a36e18b910 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26705e7740 10 client.0 _ll_register_callbacks cb 0x7f265c004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e208f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e208f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56164/0 (socket says 172.21.15.103:56164) 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f26677fe640 1 -- 172.21.15.103:0/681144189 learned_addr learned my addr 172.21.15.103:0/681144189 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f2666ffd640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18b910 0x55a36e209470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.701+0000 7f2667fff640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e2099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26677fe640 1 -- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18b910 msgr2=0x55a36e209470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26677fe640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18b910 0x55a36e209470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26677fe640 1 -- 172.21.15.103:0/681144189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 msgr2=0x55a36e2099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:23.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26677fe640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e2099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26677fe640 1 -- 172.21.15.103:0/681144189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36e19d3d0 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f2667fff640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e2099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26677fe640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e208f30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f265800cfd0 tx=0x7f265809bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906910fe02134318 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f2646ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f2646ffd640 1 -- 172.21.15.103:0/681144189 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580b9060 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f2646ffd640 1 -- 172.21.15.103:0/681144189 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f2658098290 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26580a4890 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f2646ffd640 1 -- 172.21.15.103:0/681144189 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2658010de0 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26705e7740 10 client.16857 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a36e19e6d0 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.702+0000 7f2646ffd640 1 -- 172.21.15.103:0/681144189 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26580a4890 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f2646ffd640 1 -- 172.21.15.103:0/681144189 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f2658097970 con 0x55a36e18aff0 2024-08-19T01:41:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f26705e7740 10 client.16857 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f26705e7740 10 client.16857 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f26705e7740 20 client.16857 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f26705e7740 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e215000 0x55a36e2173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f26705e7740 4 client.16857 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a36e075170 con 0x55a36e215000 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f2666ffd640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e215000 0x55a36e2173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f2666ffd640 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e215000 0x55a36e2173f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4deafc35c4fde03 server_cookie=8a7c0eb5036538ea in_seq=0 out_seq=0 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.703+0000 7f2646ffd640 10 client.16857 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f2646ffd640 1 -- 172.21.15.103:0/681144189 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a36e075170 con 0x55a36e215000 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f2646ffd640 10 client.16857 handle_command_reply: tid=0 2024-08-19T01:41:23.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 client.16857 shutdown 2024-08-19T01:41:23.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 msgr2=0x55a36e208f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e208f30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f265800cfd0 tx=0x7f265809bda0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 shutdown_connections 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55a36e215000 0x55a36e2173f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a36e1961d0 0x55a36e2099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a36e18b910 0x55a36e209470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 --2- 172.21.15.103:0/681144189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a36e18aff0 0x55a36e208f30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 >> 172.21.15.103:0/681144189 conn(0x55a36e187580 msgr2=0x55a36e1946e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f2646ffd640 0 client.16857 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 shutdown_connections 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 1 -- 172.21.15.103:0/681144189 wait complete. 2024-08-19T01:41:23.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.704+0000 7f26705e7740 20 client.16857 trim_cache size 0 max 16384 2024-08-19T01:41:23.711 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:41:23.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f26500a2b80 msgr2=0x7f26500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f26500a2b80 0x7f26500a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f266004d000 tx=0x7f266004a000 comp rx=0 tx=0).stop 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668100b50 msgr2=0x7f2668198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668100b50 0x7f2668198eb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f265800e9f0 tx=0x7f265800eeb0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 shutdown_connections 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f26500a2b80 0x7f26500a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2668102c20 0x7f26681a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26681022a0 0x7f26681993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 --2- 172.21.15.103:0/62114488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2668100b50 0x7f2668198eb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 >> 172.21.15.103:0/62114488 conn(0x7f26680fc500 msgr2=0x7f2668106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 shutdown_connections 2024-08-19T01:41:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:23.705+0000 7f266e1e9640 1 -- 172.21.15.103:0/62114488 wait complete. 2024-08-19T01:41:24.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:41:24.927 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:25.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 -- 172.21.15.103:0/2039295363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874106810 msgr2=0x7f8874106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 --2- 172.21.15.103:0/2039295363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874106810 0x7f8874106bf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8870009900 tx=0x7f887002a480 comp rx=0 tx=0).stop 2024-08-19T01:41:25.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 -- 172.21.15.103:0/2039295363 shutdown_connections 2024-08-19T01:41:25.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 --2- 172.21.15.103:0/2039295363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8874101870 0x7f887410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 --2- 172.21.15.103:0/2039295363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8874100ed0 0x7f8874101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 --2- 172.21.15.103:0/2039295363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874106810 0x7f8874106bf0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 -- 172.21.15.103:0/2039295363 >> 172.21.15.103:0/2039295363 conn(0x7f8874076f50 msgr2=0x7f88740fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 -- 172.21.15.103:0/2039295363 shutdown_connections 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.460+0000 7f887bffa640 1 -- 172.21.15.103:0/2039295363 wait complete. 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 Processor -- start 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 -- start start 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874100ed0 0x7f8874198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8874101870 0x7f88741985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8874106810 0x7f887419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8874110a70 con 0x7f8874100ed0 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f88741108f0 con 0x7f8874106810 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f887bffa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8874110bf0 con 0x7f8874101870 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f8879d96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874100ed0 0x7f8874198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f8879d96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874100ed0 0x7f8874198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56170/0 (socket says 172.21.15.103:56170) 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.461+0000 7f8879d96640 1 -- 172.21.15.103:0/1167450025 learned_addr learned my addr 172.21.15.103:0/1167450025 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879595640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8874101870 0x7f88741985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f887a597640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8874106810 0x7f887419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879d96640 1 -- 172.21.15.103:0/1167450025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8874101870 msgr2=0x7f88741985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879d96640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8874101870 0x7f88741985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879d96640 1 -- 172.21.15.103:0/1167450025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8874106810 msgr2=0x7f887419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879d96640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8874106810 0x7f887419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879d96640 1 -- 172.21.15.103:0/1167450025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887419d040 con 0x7f8874100ed0 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879595640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8874101870 0x7f88741985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f887a597640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8874106810 0x7f887419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f8879d96640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874100ed0 0x7f8874198090 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8870005940 tx=0x7f8870004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f886affd640 1 -- 172.21.15.103:0/1167450025 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887003f030 con 0x7f8874100ed0 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f886affd640 1 -- 172.21.15.103:0/1167450025 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8870003aa0 con 0x7f8874100ed0 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f886affd640 1 -- 172.21.15.103:0/1167450025 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870035070 con 0x7f8874100ed0 2024-08-19T01:41:25.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887419d2d0 con 0x7f8874100ed0 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.462+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887419d760 con 0x7f8874100ed0 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.465+0000 7f887e3f8740 1 Processor -- start 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.465+0000 7f887e3f8740 1 -- start start 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.465+0000 7f886affd640 1 -- 172.21.15.103:0/1167450025 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8870042030 con 0x7f8874100ed0 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.465+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x5584197253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.465+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x55841972fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.465+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x5584197325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f887e3f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55841968aa40 con 0x5584197301d0 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f887e3f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5584196229d0 con 0x558419725910 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f887e3f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558419686a00 con 0x558419724ff0 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x5584197325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x5584197325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56184/0 (socket says 172.21.15.103:56184) 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 -- 172.21.15.103:0/1535857887 learned_addr learned my addr 172.21.15.103:0/1535857887 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f887a597640 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x5584197253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879d96640 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x55841972fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 -- 172.21.15.103:0/1535857887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 msgr2=0x5584197253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x5584197253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 -- 172.21.15.103:0/1535857887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 msgr2=0x55841972fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x55841972fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 -- 172.21.15.103:0/1535857887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558419629650 con 0x5584197301d0 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f8879595640 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x5584197325c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8864005fd0 tx=0x7f886400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2adbc828dd3a1b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.466+0000 7f886affd640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f88600a2b80 0x7f88600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.467+0000 7f887a597640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f88600a2b80 0x7f88600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.467+0000 7f885ffff640 1 -- 172.21.15.103:0/1535857887 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864016070 con 0x5584197301d0 2024-08-19T01:41:25.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.467+0000 7f886affd640 1 -- 172.21.15.103:0/1167450025 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f88700eef90 con 0x7f8874100ed0 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.467+0000 7f885ffff640 1 -- 172.21.15.103:0/1535857887 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88640054d0 con 0x5584197301d0 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.467+0000 7f885ffff640 1 -- 172.21.15.103:0/1535857887 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864011470 con 0x5584197301d0 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 -- 172.21.15.103:0/1535857887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 msgr2=0x5584197325c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887a597640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f88600a2b80 0x7f88600a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f886c005fd0 tx=0x7f886c009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x5584197325c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8864005fd0 tx=0x7f886400e6c0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 -- 172.21.15.103:0/1535857887 shutdown_connections 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x5584197325c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x55841972fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1535857887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x5584197253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 -- 172.21.15.103:0/1535857887 >> 172.21.15.103:0/1535857887 conn(0x558419721580 msgr2=0x558419721960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 -- 172.21.15.103:0/1535857887 shutdown_connections 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 -- 172.21.15.103:0/1535857887 wait complete. 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 -- start start 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.468+0000 7f887e3f8740 1 Processor -- start 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419724ff0 0x55841972d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x55841972d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584197301d0 0x55841972dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887e3f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558419732da0 con 0x558419725910 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887e3f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558419645b20 con 0x558419724ff0 2024-08-19T01:41:25.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887e3f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558419732f20 con 0x5584197301d0 2024-08-19T01:41:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x55841972d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887e3f8740 10 client.0 _ll_register_callbacks cb 0x558419643bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x55841972d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56186/0 (socket says 172.21.15.103:56186) 2024-08-19T01:41:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 -- 172.21.15.103:0/1827176542 learned_addr learned my addr 172.21.15.103:0/1827176542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879d96640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419724ff0 0x55841972d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887a597640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584197301d0 0x55841972dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 -- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584197301d0 msgr2=0x55841972dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584197301d0 0x55841972dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 -- 172.21.15.103:0/1827176542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419724ff0 msgr2=0x55841972d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419724ff0 0x55841972d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 -- 172.21.15.103:0/1827176542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584193fc3c0 con 0x558419725910 2024-08-19T01:41:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879d96640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419724ff0 0x55841972d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f887a597640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584197301d0 0x55841972dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f8879595640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x55841972d5a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8864004b30 tx=0x7f88640099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec38c9f957cf7df server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.469+0000 7f885d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f885d7fa640 1 -- 172.21.15.103:0/1827176542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864028060 con 0x558419725910 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f885d7fa640 1 -- 172.21.15.103:0/1827176542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8864004ea0 con 0x558419725910 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88640054d0 con 0x558419725910 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f885d7fa640 1 -- 172.21.15.103:0/1827176542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8864016040 con 0x558419725910 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 10 client.16869 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55841973e950 con 0x558419725910 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f885d7fa640 1 -- 172.21.15.103:0/1827176542 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88640054d0 con 0x558419725910 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f885d7fa640 1 -- 172.21.15.103:0/1827176542 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f886400f040 con 0x558419725910 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 10 client.16869 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 10 client.16869 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 20 client.16869 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55841973fd60 0x558419742150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 4 client.16869 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.470+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558419697fb0 con 0x55841973fd60 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.471+0000 7f8879d96640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55841973fd60 0x558419742150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.471+0000 7f885d7fa640 10 client.16869 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.471+0000 7f8879d96640 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55841973fd60 0x558419742150 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7710cce483b5e9ed server_cookie=43e024585164d5a8 in_seq=0 out_seq=0 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f885d7fa640 1 -- 172.21.15.103:0/1827176542 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558419697fb0 con 0x55841973fd60 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f885d7fa640 10 client.16869 handle_command_reply: tid=0 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 client.16869 shutdown 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 msgr2=0x55841972d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x55841972d5a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8864004b30 tx=0x7f88640099b0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 shutdown_connections 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55841973fd60 0x558419742150 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5584197301d0 0x55841972dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x55841972d5a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 --2- 172.21.15.103:0/1827176542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419724ff0 0x55841972d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f885d7fa640 0 client.16869 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:41:25.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 >> 172.21.15.103:0/1827176542 conn(0x558419721580 msgr2=0x558419727260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:25.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 shutdown_connections 2024-08-19T01:41:25.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 1 -- 172.21.15.103:0/1827176542 wait complete. 2024-08-19T01:41:25.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.472+0000 7f887e3f8740 20 client.16869 trim_cache size 0 max 16384 2024-08-19T01:41:25.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 Processor -- start 2024-08-19T01:41:25.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 -- start start 2024-08-19T01:41:25.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x55841972eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x5584197476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x558419747c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88741af3b0 con 0x5584197301d0 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f88741b0f80 con 0x558419725910 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.490+0000 7f887e3f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88741afa80 con 0x558419724ff0 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x558419747c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x558419747c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56194/0 (socket says 172.21.15.103:56194) 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 -- 172.21.15.103:0/2606843332 learned_addr learned my addr 172.21.15.103:0/2606843332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f8879595640 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x5584197476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f8879d96640 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x55841972eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 -- 172.21.15.103:0/2606843332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 msgr2=0x55841972eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x55841972eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 -- 172.21.15.103:0/2606843332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 msgr2=0x5584197476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x5584197476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 -- 172.21.15.103:0/2606843332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558419725740 con 0x5584197301d0 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f8879595640 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x5584197476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f887a597640 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x558419747c30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f886c012830 tx=0x7f886c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f44f4e453422677 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.491+0000 7f885f7fe640 1 -- 172.21.15.103:0/2606843332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c01e930 con 0x5584197301d0 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.492+0000 7f885f7fe640 1 -- 172.21.15.103:0/2606843332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f886c01f420 con 0x5584197301d0 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.492+0000 7f885f7fe640 1 -- 172.21.15.103:0/2606843332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c01c670 con 0x5584197301d0 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.492+0000 7f887e3f8740 1 -- 172.21.15.103:0/2606843332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 msgr2=0x558419747c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.492+0000 7f887e3f8740 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x558419747c30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f886c012830 tx=0x7f886c014000 comp rx=0 tx=0).stop 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 -- 172.21.15.103:0/2606843332 shutdown_connections 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5584197301d0 0x558419747c30 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558419725910 0x5584197476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 --2- 172.21.15.103:0/2606843332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x55841972eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 -- 172.21.15.103:0/2606843332 >> 172.21.15.103:0/2606843332 conn(0x558419721580 msgr2=0x55841972e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 -- 172.21.15.103:0/2606843332 shutdown_connections 2024-08-19T01:41:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 -- 172.21.15.103:0/2606843332 wait complete. 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 -- start start 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.493+0000 7f887e3f8740 1 Processor -- start 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x558419738710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x558419729bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x558419729bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f887e3f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584197301d0 0x55841972a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f887e3f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88741b0880 con 0x558419725910 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f887e3f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f88741b0150 con 0x5584197301d0 2024-08-19T01:41:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f887e3f8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88741ad390 con 0x558419724ff0 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x558419729bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56198/0 (socket says 172.21.15.103:56198) 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 -- 172.21.15.103:0/3170062594 learned_addr learned my addr 172.21.15.103:0/3170062594 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879d96640 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x558419738710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 -- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 msgr2=0x558419738710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x558419738710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 -- 172.21.15.103:0/3170062594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584197301d0 msgr2=0x55841972a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584197301d0 0x55841972a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 -- 172.21.15.103:0/3170062594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841972a750 con 0x558419725910 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f887e3f8740 10 client.0 _ll_register_callbacks cb 0x7f886c009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f8879595640 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x558419729bd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55841972c910 tx=0x7f8864004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1d0079cda30c54 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.494+0000 7f885d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f886c01f420 con 0x558419725910 2024-08-19T01:41:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f885d7fa640 1 -- 172.21.15.103:0/3170062594 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886403c060 con 0x558419725910 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f885d7fa640 1 -- 172.21.15.103:0/3170062594 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8864004ea0 con 0x558419725910 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f885d7fa640 1 -- 172.21.15.103:0/3170062594 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8864033070 con 0x558419725910 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f887e3f8740 10 client.16875 fetch_fsmap learned FSMap version 33 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=33}) -- 0x5584197aa340 con 0x558419725910 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f885d7fa640 1 -- 172.21.15.103:0/3170062594 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f88640050a0 con 0x558419725910 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f887e3f8740 10 client.16875 fetch_fsmap finished waiting for FSMap version 33 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f887e3f8740 10 client.16875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f887e3f8740 20 client.16875 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.495+0000 7f887e3f8740 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5584197aa9b0 0x5584197ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.496+0000 7f887e3f8740 4 client.16875 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.496+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584196924c0 con 0x5584197aa9b0 2024-08-19T01:41:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.496+0000 7f8879d96640 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5584197aa9b0 0x5584197ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:25.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f885d7fa640 10 client.16875 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f8879d96640 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5584197aa9b0 0x5584197ace10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e92a88f52a64b720 server_cookie=b3323c1c014a7918 in_seq=0 out_seq=0 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f885d7fa640 1 -- 172.21.15.103:0/3170062594 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584196924c0 con 0x5584197aa9b0 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f885d7fa640 10 client.16875 handle_command_reply: tid=0 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 client.16875 shutdown 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 msgr2=0x558419729bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x558419729bd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55841972c910 tx=0x7f8864004c20 comp rx=0 tx=0).stop 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 shutdown_connections 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x5584197aa9b0 0x5584197ace10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5584197301d0 0x55841972a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558419725910 0x558419729bd0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 --2- 172.21.15.103:0/3170062594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558419724ff0 0x558419738710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 >> 172.21.15.103:0/3170062594 conn(0x558419721580 msgr2=0x558419727260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f885d7fa640 0 client.16875 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:41:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 shutdown_connections 2024-08-19T01:41:25.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.497+0000 7f887e3f8740 1 -- 172.21.15.103:0/3170062594 wait complete. 2024-08-19T01:41:25.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887e3f8740 20 client.16875 trim_cache size 0 max 16384 2024-08-19T01:41:25.502 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f88600a2b80 msgr2=0x7f88600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f88600a2b80 0x7f88600a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f886c005fd0 tx=0x7f886c009680 comp rx=0 tx=0).stop 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874100ed0 msgr2=0x7f8874198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874100ed0 0x7f8874198090 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8870005940 tx=0x7f8870004270 comp rx=0 tx=0).stop 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 shutdown_connections 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f88600a2b80 0x7f88600a5040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8874106810 0x7f887419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8874101870 0x7f88741985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 --2- 172.21.15.103:0/1167450025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874100ed0 0x7f8874198090 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 >> 172.21.15.103:0/1167450025 conn(0x7f8874076f50 msgr2=0x7f88740fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:25.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 shutdown_connections 2024-08-19T01:41:25.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:25.498+0000 7f887bffa640 1 -- 172.21.15.103:0/1167450025 wait complete. 2024-08-19T01:41:26.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:25 smithi146 ceph-mon[22098]: pgmap v508: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-08-19T01:41:26.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:25 smithi103 ceph-mon[18436]: pgmap v508: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-08-19T01:41:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:25 smithi190 ceph-mon[22032]: pgmap v508: 97 pgs: 97 active+clean; 7.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-08-19T01:41:26.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:41:26.852 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:27 smithi103 ceph-mon[18436]: pgmap v509: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-08-19T01:41:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:27 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:27.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:27 smithi190 ceph-mon[22032]: pgmap v509: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-08-19T01:41:27.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:27 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:27.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.414+0000 7f77fb207640 1 -- 172.21.15.103:0/2080544175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 msgr2=0x7f77f410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.414+0000 7f77fb207640 1 --2- 172.21.15.103:0/2080544175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f410e960 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009cd0 tx=0x7f77ec02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.414+0000 7f77fb207640 1 -- 172.21.15.103:0/2080544175 shutdown_connections 2024-08-19T01:41:27.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.414+0000 7f77fb207640 1 --2- 172.21.15.103:0/2080544175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f410e960 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.414+0000 7f77fb207640 1 --2- 172.21.15.103:0/2080544175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f4103470 0x7f77f41078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.414+0000 7f77fb207640 1 --2- 172.21.15.103:0/2080544175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f4102ac0 0x7f77f4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.414+0000 7f77fb207640 1 -- 172.21.15.103:0/2080544175 >> 172.21.15.103:0/2080544175 conn(0x7f77f40fc500 msgr2=0x7f77f40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 -- 172.21.15.103:0/2080544175 shutdown_connections 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 -- 172.21.15.103:0/2080544175 wait complete. 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 Processor -- start 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 -- start start 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f4102ac0 0x7f77f4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f4103470 0x7f77f4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111cb0 con 0x7f77f4107fc0 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111b30 con 0x7f77f4102ac0 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.415+0000 7f77fb207640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111e30 con 0x7f77f4103470 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56220/0 (socket says 172.21.15.103:56220) 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 -- 172.21.15.103:0/816507764 learned_addr learned my addr 172.21.15.103:0/816507764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f4102ac0 0x7f77f4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f3fff640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f4103470 0x7f77f4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 -- 172.21.15.103:0/816507764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f4103470 msgr2=0x7f77f4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f4103470 0x7f77f4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 -- 172.21.15.103:0/816507764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f4102ac0 msgr2=0x7f77f4198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f4102ac0 0x7f77f4198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 -- 172.21.15.103:0/816507764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f419d1b0 con 0x7f77f4107fc0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f97a4640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f419cad0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f77ec005cc0 tx=0x7f77ec003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f1ffb640 1 -- 172.21.15.103:0/816507764 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec004240 con 0x7f77f4107fc0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f1ffb640 1 -- 172.21.15.103:0/816507764 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77ec0305a0 con 0x7f77f4107fc0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77f1ffb640 1 -- 172.21.15.103:0/816507764 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0466c0 con 0x7f77f4107fc0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.416+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f419d440 con 0x7f77f4107fc0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.417+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f419d910 con 0x7f77f4107fc0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.419+0000 7f77f1ffb640 1 -- 172.21.15.103:0/816507764 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f77ec030ac0 con 0x7f77f4107fc0 2024-08-19T01:41:27.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.419+0000 7f77fd605740 1 Processor -- start 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77fd605740 1 -- start start 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77fd605740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca4049ed0 0x559ca404a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77fd605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77fd605740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca40595e0 0x559ca405b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77fd605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ca3fafa40 con 0x559ca404a7f0 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77fd605740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559ca3f479d0 con 0x559ca4049ed0 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77fd605740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559ca3faba00 con 0x559ca40595e0 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56222/0 (socket says 172.21.15.103:56222) 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 -- 172.21.15.103:0/3324851537 learned_addr learned my addr 172.21.15.103:0/3324851537 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:27.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f3fff640 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca4049ed0 0x559ca404a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f3fff640 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca40595e0 0x559ca405b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f1ffb640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e40a2c00 0x7f77e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 -- 172.21.15.103:0/3324851537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca40595e0 msgr2=0x559ca405b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca40595e0 0x559ca405b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 -- 172.21.15.103:0/3324851537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca4049ed0 msgr2=0x559ca404a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca4049ed0 0x559ca404a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.420+0000 7f77f8fa3640 1 -- 172.21.15.103:0/3324851537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca3f4e650 con 0x559ca404a7f0 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.421+0000 7f77f97a4640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e40a2c00 0x7f77e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.421+0000 7f77f1ffb640 1 -- 172.21.15.103:0/816507764 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f77ec02e030 con 0x7f77f4107fc0 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.421+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40590a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009a70 tx=0x7f77e8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fdd4cc0d45a31d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.421+0000 7f77deffd640 1 -- 172.21.15.103:0/3324851537 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8017070 con 0x559ca404a7f0 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.421+0000 7f77deffd640 1 -- 172.21.15.103:0/3324851537 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77e8013470 con 0x559ca404a7f0 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.421+0000 7f77deffd640 1 -- 172.21.15.103:0/3324851537 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8012630 con 0x559ca404a7f0 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 -- 172.21.15.103:0/3324851537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 msgr2=0x559ca40590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77f97a4640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e40a2c00 0x7f77e40a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77ec031e90 tx=0x7f77ec08ee20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40590a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009a70 tx=0x7f77e8009f30 comp rx=0 tx=0).stop 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 -- 172.21.15.103:0/3324851537 shutdown_connections 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca40595e0 0x559ca405b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40590a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 --2- 172.21.15.103:0/3324851537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca4049ed0 0x559ca404a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 -- 172.21.15.103:0/3324851537 >> 172.21.15.103:0/3324851537 conn(0x559ca4046580 msgr2=0x559ca4046960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 -- 172.21.15.103:0/3324851537 shutdown_connections 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 -- 172.21.15.103:0/3324851537 wait complete. 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 -- start start 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.422+0000 7f77fd605740 1 Processor -- start 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77fd605740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca40564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77fd605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77fd605740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca4056f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77fd605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ca405c1b0 con 0x559ca404a7f0 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77fd605740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559ca3f6ab20 con 0x559ca40595e0 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77fd605740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559ca405c330 con 0x559ca4049ed0 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77fd605740 10 client.0 _ll_register_callbacks cb 0x559ca3f68bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:27.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f8fa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f8fa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56236/0 (socket says 172.21.15.103:56236) 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f8fa3640 1 -- 172.21.15.103:0/1189443732 learned_addr learned my addr 172.21.15.103:0/1189443732 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca4056f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca40564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f3fff640 1 -- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 msgr2=0x559ca40564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca40564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f3fff640 1 -- 172.21.15.103:0/1189443732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 msgr2=0x559ca40569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f3fff640 1 -- 172.21.15.103:0/1189443732 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca3d213c0 con 0x559ca40595e0 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.423+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:27.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca4056f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x559ca4047900 tx=0x7f77e000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1025250b2cb1a864 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77d3fff640 1 -- 172.21.15.103:0/1189443732 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0018070 con 0x559ca40595e0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e8013470 con 0x559ca40595e0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77d3fff640 1 -- 172.21.15.103:0/1189443732 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f77e0005220 con 0x559ca40595e0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77d3fff640 1 -- 172.21.15.103:0/1189443732 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77e0028070 con 0x559ca40595e0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77fd605740 10 client.26234 fetch_fsmap learned FSMap version 34 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559ca4067eb0 con 0x559ca40595e0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.424+0000 7f77d3fff640 1 -- 172.21.15.103:0/1189443732 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77e8013470 con 0x559ca40595e0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77d3fff640 1 -- 172.21.15.103:0/1189443732 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f77e0005420 con 0x559ca40595e0 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77fd605740 10 client.26234 fetch_fsmap finished waiting for FSMap version 34 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77fd605740 10 client.26234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77fd605740 20 client.26234 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77fd605740 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40692c0 0x559ca406b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77fd605740 4 client.26234 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ca3fbcfb0 con 0x559ca40692c0 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.425+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40692c0 0x559ca406b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.426+0000 7f77d3fff640 10 client.26234 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.426+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40692c0 0x559ca406b6b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0c1a1486d841f14 server_cookie=9a5614e6299667fc in_seq=0 out_seq=0 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.426+0000 7f77d3fff640 1 -- 172.21.15.103:0/1189443732 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ca3fbcfb0 con 0x559ca40692c0 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.426+0000 7f77d3fff640 10 client.26234 handle_command_reply: tid=0 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 client.26234 shutdown 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 msgr2=0x559ca4056f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca4056f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x559ca4047900 tx=0x7f77e000c2d0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 shutdown_connections 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40692c0 0x559ca406b6b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca4056f20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca40569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 --2- 172.21.15.103:0/1189443732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca40564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77d3fff640 0 client.26234 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:41:27.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 >> 172.21.15.103:0/1189443732 conn(0x559ca4046580 msgr2=0x559ca404c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:27.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 shutdown_connections 2024-08-19T01:41:27.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 1 -- 172.21.15.103:0/1189443732 wait complete. 2024-08-19T01:41:27.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.427+0000 7f77fd605740 20 client.26234 trim_cache size 0 max 16384 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 Processor -- start 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 -- start start 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4056d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca404a7f0 0x559ca4070f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca40595e0 0x559ca4071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41aaf00 con 0x559ca40595e0 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41acaa0 con 0x559ca404a7f0 2024-08-19T01:41:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77fd605740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41ab5d0 con 0x559ca4049ed0 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca40595e0 0x559ca4071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca40595e0 0x559ca4071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56240/0 (socket says 172.21.15.103:56240) 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77f3fff640 1 -- 172.21.15.103:0/1706539839 learned_addr learned my addr 172.21.15.103:0/1706539839 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4056d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.441+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca404a7f0 0x559ca4070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77f3fff640 1 -- 172.21.15.103:0/1706539839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 msgr2=0x559ca4056d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4056d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77f3fff640 1 -- 172.21.15.103:0/1706539839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca404a7f0 msgr2=0x559ca4070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca404a7f0 0x559ca4070f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77f3fff640 1 -- 172.21.15.103:0/1706539839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca4047710 con 0x559ca40595e0 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77f3fff640 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca40595e0 0x559ca4071480 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f77e000c6f0 tx=0x7f77e0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d67efdbafcb5699 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77de7fc640 1 -- 172.21.15.103:0/1706539839 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0018070 con 0x559ca40595e0 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77de7fc640 1 -- 172.21.15.103:0/1706539839 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77e00120e0 con 0x559ca40595e0 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.442+0000 7f77de7fc640 1 -- 172.21.15.103:0/1706539839 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e001b040 con 0x559ca40595e0 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 -- 172.21.15.103:0/1706539839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca40595e0 msgr2=0x559ca4071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca40595e0 0x559ca4071480 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f77e000c6f0 tx=0x7f77e0014040 comp rx=0 tx=0).stop 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 -- 172.21.15.103:0/1706539839 shutdown_connections 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca40595e0 0x559ca4071480 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca404a7f0 0x559ca4070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 --2- 172.21.15.103:0/1706539839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4056d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 -- 172.21.15.103:0/1706539839 >> 172.21.15.103:0/1706539839 conn(0x559ca4046580 msgr2=0x559ca4054d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 -- 172.21.15.103:0/1706539839 shutdown_connections 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 -- 172.21.15.103:0/1706539839 wait complete. 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 -- start start 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.443+0000 7f77fd605740 1 Processor -- start 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77fd605740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4061ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77fd605740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca405c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77fd605740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca405cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77fd605740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41ac3a0 con 0x559ca404a7f0 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77fd605740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41abca0 con 0x559ca40595e0 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77fd605740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41a8ee0 con 0x559ca4049ed0 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77fd605740 10 client.0 _ll_register_callbacks cb 0x7f77e000ccd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f8fa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca405c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca405cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca405cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:36318/0 (socket says 172.21.15.103:36318) 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f8fa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca405c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56252/0 (socket says 172.21.15.103:56252) 2024-08-19T01:41:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f3fff640 1 -- 172.21.15.103:0/3333271083 learned_addr learned my addr 172.21.15.103:0/3333271083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:27.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f3fff640 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4061ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f8fa3640 1 -- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 msgr2=0x559ca4061ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4061ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f8fa3640 1 -- 172.21.15.103:0/3333271083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 msgr2=0x559ca405cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca405cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f8fa3640 1 -- 172.21.15.103:0/3333271083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca40d1f00 con 0x559ca404a7f0 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.444+0000 7f77f3fff640 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca405cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77f8fa3640 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca405c580 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f77e8004810 tx=0x7f77e8013c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a9a6d35569df70 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77d3fff640 1 -- 172.21.15.103:0/3333271083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8017040 con 0x559ca404a7f0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77d3fff640 1 -- 172.21.15.103:0/3333271083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f77e8009660 con 0x559ca404a7f0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e00120e0 con 0x559ca404a7f0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77d3fff640 1 -- 172.21.15.103:0/3333271083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77e8013ca0 con 0x559ca404a7f0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 10 client.16893 fetch_fsmap learned FSMap version 34 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559ca40d3040 con 0x559ca404a7f0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77d3fff640 1 -- 172.21.15.103:0/3333271083 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77e00120e0 con 0x559ca404a7f0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77d3fff640 1 -- 172.21.15.103:0/3333271083 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f77e8026510 con 0x559ca404a7f0 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 10 client.16893 fetch_fsmap finished waiting for FSMap version 34 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 10 client.16893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 20 client.16893 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40d3430 0x559ca40d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 4 client.16893 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.445+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ca3fb74c0 con 0x559ca40d3430 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77f3fff640 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40d3430 0x559ca40d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77f3fff640 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40d3430 0x559ca40d5850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8513291d45d250d server_cookie=e3ad9221a1f71ddc in_seq=0 out_seq=0 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77d3fff640 10 client.16893 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77d3fff640 1 -- 172.21.15.103:0/3333271083 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ca3fb74c0 con 0x559ca40d3430 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77d3fff640 10 client.16893 handle_command_reply: tid=0 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77fd605740 1 client.16893 shutdown 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 msgr2=0x559ca405c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.446+0000 7f77fd605740 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca405c580 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f77e8004810 tx=0x7f77e8013c30 comp rx=0 tx=0).stop 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 shutdown_connections 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x559ca40d3430 0x559ca40d5850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559ca40595e0 0x559ca405cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ca404a7f0 0x559ca405c580 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 --2- 172.21.15.103:0/3333271083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca4049ed0 0x559ca4061ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77d3fff640 0 client.16893 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:41:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 >> 172.21.15.103:0/3333271083 conn(0x559ca4046580 msgr2=0x559ca4057ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 shutdown_connections 2024-08-19T01:41:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 1 -- 172.21.15.103:0/3333271083 wait complete. 2024-08-19T01:41:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fd605740 20 client.16893 trim_cache size 0 max 16384 2024-08-19T01:41:27.454 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:41:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e40a2c00 msgr2=0x7f77e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fb207640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e40a2c00 0x7f77e40a50c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77ec031e90 tx=0x7f77ec08ee20 comp rx=0 tx=0).stop 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 msgr2=0x7f77f419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.447+0000 7f77fb207640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f419cad0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f77ec005cc0 tx=0x7f77ec003ba0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 shutdown_connections 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f77e40a2c00 0x7f77e40a50c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77f4107fc0 0x7f77f419cad0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77f4103470 0x7f77f4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 --2- 172.21.15.103:0/816507764 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77f4102ac0 0x7f77f4198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 >> 172.21.15.103:0/816507764 conn(0x7f77f40fc500 msgr2=0x7f77f4105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 shutdown_connections 2024-08-19T01:41:27.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:27.448+0000 7f77fb207640 1 -- 172.21.15.103:0/816507764 wait complete. 2024-08-19T01:41:27.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:27 smithi146 ceph-mon[22098]: pgmap v509: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-08-19T01:41:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:27 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:28.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:41:28.672 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 -- 172.21.15.103:0/9827762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01078a0 msgr2=0x7f7bf0107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/9827762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01078a0 0x7f7bf0107c80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009930 tx=0x7f7bec02a420 comp rx=0 tx=0).stop 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 -- 172.21.15.103:0/9827762 shutdown_connections 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/9827762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf0102240 0x7f7bf010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/9827762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf01018a0 0x7f7bf0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/9827762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01078a0 0x7f7bf0107c80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 -- 172.21.15.103:0/9827762 >> 172.21.15.103:0/9827762 conn(0x7f7bf00fd5e0 msgr2=0x7f7bf00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:29.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 -- 172.21.15.103:0/9827762 shutdown_connections 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.213+0000 7f7bf75dc640 1 -- 172.21.15.103:0/9827762 wait complete. 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 Processor -- start 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 -- start start 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01018a0 0x7f7bf01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf0102240 0x7f7bf0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf01078a0 0x7f7bf019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0111b50 con 0x7f7bf01018a0 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01119d0 con 0x7f7bf01078a0 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf75dc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0111cd0 con 0x7f7bf0102240 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01018a0 0x7f7bf01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01018a0 0x7f7bf01980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56266/0 (socket says 172.21.15.103:56266) 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.214+0000 7f7bf5378640 1 -- 172.21.15.103:0/1490335600 learned_addr learned my addr 172.21.15.103:0/1490335600 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:29.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf01078a0 0x7f7bf019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf0102240 0x7f7bf0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf5378640 1 -- 172.21.15.103:0/1490335600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf0102240 msgr2=0x7f7bf0198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf5378640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf0102240 0x7f7bf0198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf5378640 1 -- 172.21.15.103:0/1490335600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf01078a0 msgr2=0x7f7bf019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf5378640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf01078a0 0x7f7bf019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf5378640 1 -- 172.21.15.103:0/1490335600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf019d080 con 0x7f7bf01018a0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf5378640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01018a0 0x7f7bf01980d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005940 tx=0x7f7bec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7be67fc640 1 -- 172.21.15.103:0/1490335600 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec03f030 con 0x7f7bf01018a0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf019d310 con 0x7f7bf01018a0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7be67fc640 1 -- 172.21.15.103:0/1490335600 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bec0318a0 con 0x7f7bf01018a0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7be67fc640 1 -- 172.21.15.103:0/1490335600 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec035070 con 0x7f7bf01018a0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.215+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf01a4c50 con 0x7f7bf01018a0 2024-08-19T01:41:29.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.218+0000 7f7be67fc640 1 -- 172.21.15.103:0/1490335600 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7bec042030 con 0x7f7bf01018a0 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.218+0000 7f7bf99da740 1 Processor -- start 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.218+0000 7f7bf99da740 1 -- start start 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b847d0 0x555571b930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b935e0 0x555571b959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf99da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555571ae9a40 con 0x555571b847d0 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf99da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555571a819d0 con 0x555571b935e0 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf99da740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555571ae5a00 con 0x555571b83eb0 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b847d0 0x555571b930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b847d0 0x555571b930a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56270/0 (socket says 172.21.15.103:56270) 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 -- 172.21.15.103:0/3646967064 learned_addr learned my addr 172.21.15.103:0/3646967064 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b935e0 0x555571b959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7be67fc640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7bdc0a2b80 0x7f7bdc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 -- 172.21.15.103:0/3646967064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 msgr2=0x555571b84290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b84290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 -- 172.21.15.103:0/3646967064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b935e0 msgr2=0x555571b959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b935e0 0x555571b959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7be67fc640 1 -- 172.21.15.103:0/1490335600 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7bec0ef080 con 0x7f7bf01018a0 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5378640 1 -- 172.21.15.103:0/3646967064 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555571a88650 con 0x555571b847d0 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b84290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b935e0 0x555571b959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.219+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7bdc0a2b80 0x7f7bdc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.220+0000 7f7bf5378640 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b847d0 0x555571b930a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0bb610 tx=0x7f7bec02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15af9feca39a97c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.220+0000 7f7bdb7fe640 1 -- 172.21.15.103:0/3646967064 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec049070 con 0x555571b847d0 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.220+0000 7f7bdb7fe640 1 -- 172.21.15.103:0/3646967064 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bec003e20 con 0x555571b847d0 2024-08-19T01:41:29.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.220+0000 7f7bdb7fe640 1 -- 172.21.15.103:0/3646967064 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0bb910 con 0x555571b847d0 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.220+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7bdc0a2b80 0x7f7bdc0a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7be80092f0 tx=0x7f7be8004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 -- 172.21.15.103:0/3646967064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b847d0 msgr2=0x555571b930a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b847d0 0x555571b930a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0bb610 tx=0x7f7bec02a420 comp rx=0 tx=0).stop 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 -- 172.21.15.103:0/3646967064 shutdown_connections 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b935e0 0x555571b959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b847d0 0x555571b930a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3646967064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b84290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 -- 172.21.15.103:0/3646967064 >> 172.21.15.103:0/3646967064 conn(0x555571b80580 msgr2=0x555571b80960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 -- 172.21.15.103:0/3646967064 shutdown_connections 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 -- 172.21.15.103:0/3646967064 wait complete. 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 -- start start 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.221+0000 7f7bf99da740 1 Processor -- start 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b90a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571b90f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf99da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555571b95f70 con 0x555571b935e0 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf99da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555571aa4b20 con 0x555571b847d0 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf99da740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x555571b960f0 con 0x555571b83eb0 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf99da740 10 client.0 _ll_register_callbacks cb 0x555571aa2bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b904c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:60870/0 (socket says 172.21.15.103:60870) 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 -- 172.21.15.103:0/202538919 learned_addr learned my addr 172.21.15.103:0/202538919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571b90f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b90a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 -- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 msgr2=0x555571b90a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b90a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 -- 172.21.15.103:0/202538919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 msgr2=0x555571b90f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571b90f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.222+0000 7f7bf5378640 1 -- 172.21.15.103:0/202538919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557185b3c0 con 0x555571b83eb0 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bf5378640 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b904c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0c4050 tx=0x7f7bec03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac99cf7999a3e3db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/202538919 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec049040 con 0x555571b83eb0 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555571af6fb0 con 0x555571b83eb0 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/202538919 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7bec0a6110 con 0x555571b83eb0 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/202538919 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7bec0bc010 con 0x555571b83eb0 2024-08-19T01:41:29.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bf99da740 10 client.26206 fetch_fsmap learned FSMap version 34 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555571b95f70 con 0x555571b83eb0 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/202538919 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555571af6fb0 con 0x555571b83eb0 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/202538919 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7bec073080 con 0x555571b83eb0 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bf99da740 10 client.26206 fetch_fsmap finished waiting for FSMap version 34 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bf99da740 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.223+0000 7f7bf99da740 20 client.26206 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.224+0000 7f7bf99da740 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571ba2c20 0x555571ba5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.224+0000 7f7bf99da740 4 client.26206 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.224+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555571af14c0 con 0x555571ba2c20 2024-08-19T01:41:29.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.224+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571ba2c20 0x555571ba5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.224+0000 7f7bd8ff9640 10 client.26206 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.224+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571ba2c20 0x555571ba5010 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7f8ac14adf8058 server_cookie=e5c5c39d9e47e1da in_seq=0 out_seq=0 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/202538919 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555571af14c0 con 0x555571ba2c20 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bd8ff9640 10 client.26206 handle_command_reply: tid=0 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 client.26206 shutdown 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 msgr2=0x555571b904c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b904c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0c4050 tx=0x7f7bec03b460 comp rx=0 tx=0).stop 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 shutdown_connections 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571ba2c20 0x555571ba5010 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571b90f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b90a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bd8ff9640 0 client.26206 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:29.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 --2- 172.21.15.103:0/202538919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b904c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 >> 172.21.15.103:0/202538919 conn(0x555571b80580 msgr2=0x555571b862e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:29.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 shutdown_connections 2024-08-19T01:41:29.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 1 -- 172.21.15.103:0/202538919 wait complete. 2024-08-19T01:41:29.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.225+0000 7f7bf99da740 20 client.26206 trim_cache size 0 max 16384 2024-08-19T01:41:29.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 Processor -- start 2024-08-19T01:41:29.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 -- start start 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571baa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571baadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01aafe0 con 0x555571b935e0 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01acb80 con 0x555571b847d0 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.242+0000 7f7bf99da740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ab6b0 con 0x555571b83eb0 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571baadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571baadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56278/0 (socket says 172.21.15.103:56278) 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 -- 172.21.15.103:0/3466890748 learned_addr learned my addr 172.21.15.103:0/3466890748 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571baa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5378640 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 -- 172.21.15.103:0/3466890748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 msgr2=0x555571b90d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b90d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 -- 172.21.15.103:0/3466890748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 msgr2=0x555571baa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571baa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 -- 172.21.15.103:0/3466890748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555571b7fdc0 con 0x555571b935e0 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571baadd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7be8013880 tx=0x7f7be8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28ee92ddea507b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bdaffd640 1 -- 172.21.15.103:0/3466890748 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8003970 con 0x555571b935e0 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bdaffd640 1 -- 172.21.15.103:0/3466890748 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be8017070 con 0x555571b935e0 2024-08-19T01:41:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.243+0000 7f7bdaffd640 1 -- 172.21.15.103:0/3466890748 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004410 con 0x555571b935e0 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 -- 172.21.15.103:0/3466890748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 msgr2=0x555571baadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571baadd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7be8013880 tx=0x7f7be8015000 comp rx=0 tx=0).stop 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 -- 172.21.15.103:0/3466890748 shutdown_connections 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571baadd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571baa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3466890748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 -- 172.21.15.103:0/3466890748 >> 172.21.15.103:0/3466890748 conn(0x555571b80580 msgr2=0x555571b92430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 -- 172.21.15.103:0/3466890748 shutdown_connections 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.244+0000 7f7bf99da740 1 -- 172.21.15.103:0/3466890748 wait complete. 2024-08-19T01:41:29.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 -- start start 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 Processor -- start 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b96560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b96aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571c0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ac480 con 0x555571b935e0 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01abd80 con 0x555571b847d0 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01a8fc0 con 0x555571b83eb0 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.245+0000 7f7bf99da740 10 client.0 _ll_register_callbacks cb 0x7f7be0009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b96560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf5378640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b96560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:60888/0 (socket says 172.21.15.103:60888) 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf4b77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b96aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf5378640 1 -- 172.21.15.103:0/3415048505 learned_addr learned my addr 172.21.15.103:0/3415048505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571c0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf4b77640 1 -- 172.21.15.103:0/3415048505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 msgr2=0x555571b96560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b96560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf4b77640 1 -- 172.21.15.103:0/3415048505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 msgr2=0x555571c0b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571c0b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf4b77640 1 -- 172.21.15.103:0/3415048505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555571c0bd90 con 0x555571b847d0 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf5378640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b96560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf5b79640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571c0b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf4b77640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b96aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x555571ba5830 tx=0x7f7be001f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990aab0132999b76 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/3415048505 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be002b060 con 0x555571b847d0 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/3415048505 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f7be0012610 con 0x555571b847d0 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.246+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8017070 con 0x555571b847d0 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/3415048505 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7be0012810 con 0x555571b847d0 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 10 client.26258 fetch_fsmap learned FSMap version 34 2024-08-19T01:41:29.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555571c0cff0 con 0x555571b847d0 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/3415048505 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be8017070 con 0x555571b847d0 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/3415048505 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7be0025030 con 0x555571b847d0 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 10 client.26258 fetch_fsmap finished waiting for FSMap version 34 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 20 client.26258 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571c0d770 0x555571c0fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 4 client.26258 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555571a6e170 con 0x555571c0d770 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.247+0000 7f7bf5378640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571c0d770 0x555571c0fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bd8ff9640 10 client.26258 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf5378640 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571c0d770 0x555571c0fb90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a17dd52e1282178 server_cookie=85e42ccff6e5b902 in_seq=0 out_seq=0 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bd8ff9640 1 -- 172.21.15.103:0/3415048505 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555571a6e170 con 0x555571c0d770 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bd8ff9640 10 client.26258 handle_command_reply: tid=0 2024-08-19T01:41:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 client.26258 shutdown 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 msgr2=0x555571b96aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b96aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x555571ba5830 tx=0x7f7be001f920 comp rx=0 tx=0).stop 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 shutdown_connections 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x555571c0d770 0x555571c0fb90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555571b935e0 0x555571c0b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bd8ff9640 0 client.26258 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555571b847d0 0x555571b96aa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 --2- 172.21.15.103:0/3415048505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555571b83eb0 0x555571b96560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 >> 172.21.15.103:0/3415048505 conn(0x555571b80580 msgr2=0x555571b91b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 shutdown_connections 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 1 -- 172.21.15.103:0/3415048505 wait complete. 2024-08-19T01:41:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.248+0000 7f7bf99da740 20 client.26258 trim_cache size 0 max 16384 2024-08-19T01:41:29.255 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:41:29.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7bdc0a2b80 msgr2=0x7f7bdc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7bdc0a2b80 0x7f7bdc0a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7be80092f0 tx=0x7f7be8004a40 comp rx=0 tx=0).stop 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01018a0 msgr2=0x7f7bf01980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01018a0 0x7f7bf01980d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005940 tx=0x7f7bec0042f0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 shutdown_connections 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f7bdc0a2b80 0x7f7bdc0a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7bf01078a0 0x7f7bf019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7bf0102240 0x7f7bf0198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 --2- 172.21.15.103:0/1490335600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bf01018a0 0x7f7bf01980d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 >> 172.21.15.103:0/1490335600 conn(0x7f7bf00fd5e0 msgr2=0x7f7bf00ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 shutdown_connections 2024-08-19T01:41:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:29.249+0000 7f7bf75dc640 1 -- 172.21.15.103:0/1490335600 wait complete. 2024-08-19T01:41:30.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:29 smithi103 ceph-mon[18436]: pgmap v510: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:41:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:29 smithi190 ceph-mon[22032]: pgmap v510: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:41:30.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:29 smithi146 ceph-mon[22098]: pgmap v510: 97 pgs: 97 active+clean; 8.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:41:31.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:31 smithi103 ceph-mon[18436]: pgmap v511: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:41:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:31 smithi190 ceph-mon[22032]: pgmap v511: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:41:31.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:31 smithi146 ceph-mon[22098]: pgmap v511: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-08-19T01:41:32.215 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:41:32.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:41:32.516 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:33.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:33 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:33.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:33 smithi103 ceph-mon[18436]: pgmap v512: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:41:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:33 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:33 smithi190 ceph-mon[22032]: pgmap v512: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:41:33.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:33 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:41:33.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:33 smithi146 ceph-mon[22098]: pgmap v512: 97 pgs: 97 active+clean; 8.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:41:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.069+0000 7f637b105640 1 -- 172.21.15.103:0/3315032136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374102320 msgr2=0x7f63740695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.069+0000 7f637b105640 1 --2- 172.21.15.103:0/3315032136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374102320 0x7f63740695d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f636c009860 tx=0x7f636c02a340 comp rx=0 tx=0).stop 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.070+0000 7f637b105640 1 -- 172.21.15.103:0/3315032136 shutdown_connections 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.070+0000 7f637b105640 1 --2- 172.21.15.103:0/3315032136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374102320 0x7f63740695d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.070+0000 7f637b105640 1 --2- 172.21.15.103:0/3315032136 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374101960 0x7f6374101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.070+0000 7f637b105640 1 --2- 172.21.15.103:0/3315032136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6374100770 0x7f6374100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.070+0000 7f637b105640 1 -- 172.21.15.103:0/3315032136 >> 172.21.15.103:0/3315032136 conn(0x7f63740fbf10 msgr2=0x7f63740fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.070+0000 7f637b105640 1 -- 172.21.15.103:0/3315032136 shutdown_connections 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.070+0000 7f637b105640 1 -- 172.21.15.103:0/3315032136 wait complete. 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 Processor -- start 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 -- start start 2024-08-19T01:41:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374100770 0x7f637419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6374101960 0x7f637419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374102320 0x7f63741a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f637410b1a0 con 0x7f6374100770 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f637410b020 con 0x7f6374102320 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f637b105640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f637410b320 con 0x7f6374101960 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374100770 0x7f637419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374100770 0x7f637419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54304/0 (socket says 172.21.15.103:54304) 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 -- 172.21.15.103:0/531493331 learned_addr learned my addr 172.21.15.103:0/531493331 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f63796a2640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374102320 0x7f63741a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6373fff640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6374101960 0x7f637419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 -- 172.21.15.103:0/531493331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6374101960 msgr2=0x7f637419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6374101960 0x7f637419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 -- 172.21.15.103:0/531493331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374102320 msgr2=0x7f63741a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374102320 0x7f63741a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6378ea1640 1 -- 172.21.15.103:0/531493331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63741a5c10 con 0x7f6374100770 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.071+0000 7f6373fff640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6374101960 0x7f637419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.072+0000 7f6378ea1640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374100770 0x7f637419dff0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f636400ea80 tx=0x7f636400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.072+0000 7f6371ffb640 1 -- 172.21.15.103:0/531493331 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400ce40 con 0x7f6374100770 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.072+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741a5f00 con 0x7f6374100770 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.072+0000 7f6371ffb640 1 -- 172.21.15.103:0/531493331 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6364011920 con 0x7f6374100770 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.072+0000 7f6371ffb640 1 -- 172.21.15.103:0/531493331 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636401aaa0 con 0x7f6374100770 2024-08-19T01:41:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.072+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a63c0 con 0x7f6374100770 2024-08-19T01:41:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.075+0000 7f637dec2740 1 Processor -- start 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.075+0000 7f6371ffb640 1 -- 172.21.15.103:0/531493331 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6364011ad0 con 0x7f6374100770 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.075+0000 7f637dec2740 1 -- start start 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.075+0000 7f637dec2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705ed680 0x55ed705eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.075+0000 7f637dec2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.075+0000 7f637dec2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705f5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f637dec2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed705509d0 con 0x55ed705edfc0 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f637dec2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ed704f8820 con 0x55ed705ed680 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f637dec2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ed7054c990 con 0x55ed705f3050 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f63796a2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705ed680 0x55ed705eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f63796a2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705ed680 0x55ed705eda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:36902/0 (socket says 172.21.15.103:36902) 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f63796a2640 1 -- 172.21.15.103:0/1372830720 learned_addr learned my addr 172.21.15.103:0/1372830720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6373fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6378ea1640 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6373fff640 1 -- 172.21.15.103:0/1372830720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705ed680 msgr2=0x55ed705eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6373fff640 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705ed680 0x55ed705eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6373fff640 1 -- 172.21.15.103:0/1372830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 msgr2=0x55ed705f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6373fff640 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6373fff640 1 -- 172.21.15.103:0/1372830720 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed70264210 con 0x55ed705f3050 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.076+0000 7f6378ea1640 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.077+0000 7f6371ffb640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63600a2cb0 0x7f63600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.077+0000 7f63796a2640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63600a2cb0 0x7f63600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.077+0000 7f6373fff640 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705f5510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f635c005fd0 tx=0x7f635c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d6008c342797b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.077+0000 7f6371ffb640 1 -- 172.21.15.103:0/531493331 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f63640cf590 con 0x7f6374100770 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.077+0000 7f636affd640 1 -- 172.21.15.103:0/1372830720 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c012450 con 0x55ed705f3050 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.077+0000 7f636affd640 1 -- 172.21.15.103:0/1372830720 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f635c012aa0 con 0x55ed705f3050 2024-08-19T01:41:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.077+0000 7f636affd640 1 -- 172.21.15.103:0/1372830720 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c00fad0 con 0x55ed705f3050 2024-08-19T01:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f637dec2740 1 -- 172.21.15.103:0/1372830720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 msgr2=0x55ed705f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f63796a2640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63600a2cb0 0x7f63600a5170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f636c0095f0 tx=0x7f636c0056d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f637dec2740 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705f5510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f635c005fd0 tx=0x7f635c0117b0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f637dec2740 1 -- 172.21.15.103:0/1372830720 shutdown_connections 2024-08-19T01:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f637dec2740 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705f5510 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f637dec2740 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f637dec2740 1 --2- 172.21.15.103:0/1372830720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705ed680 0x55ed705eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.078+0000 7f637dec2740 1 -- 172.21.15.103:0/1372830720 >> 172.21.15.103:0/1372830720 conn(0x55ed705e8cf0 msgr2=0x55ed705e90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.079+0000 7f637dec2740 1 -- 172.21.15.103:0/1372830720 shutdown_connections 2024-08-19T01:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.079+0000 7f637dec2740 1 -- 172.21.15.103:0/1372830720 wait complete. 2024-08-19T01:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.079+0000 7f637dec2740 1 -- start start 2024-08-19T01:41:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.079+0000 7f637dec2740 1 Processor -- start 2024-08-19T01:41:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f637dec2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705ed680 0x55ed705f7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f637dec2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705edfc0 0x55ed705f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f637dec2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f637dec2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed705f6020 con 0x55ed705ed680 2024-08-19T01:41:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f637dec2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ed7050c520 con 0x55ed705f3050 2024-08-19T01:41:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f637dec2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ed705f61a0 con 0x55ed705edfc0 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f6378ea1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705ed680 0x55ed705f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f6373fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705edfc0 0x55ed705f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f6378ea1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705ed680 0x55ed705f7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54318/0 (socket says 172.21.15.103:54318) 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f6378ea1640 1 -- 172.21.15.103:0/2583390890 learned_addr learned my addr 172.21.15.103:0/2583390890 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.080+0000 7f637dec2740 10 client.0 _ll_register_callbacks cb 0x55ed70263a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f63796a2640 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6378ea1640 1 -- 172.21.15.103:0/2583390890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705edfc0 msgr2=0x55ed705f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6378ea1640 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705edfc0 0x55ed705f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6378ea1640 1 -- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 msgr2=0x55ed705fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6378ea1640 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6378ea1640 1 -- 172.21.15.103:0/2583390890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed70330620 con 0x55ed705ed680 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f63796a2640 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6378ea1640 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705ed680 0x55ed705f7830 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f636400ea50 tx=0x7f636409bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0936e76fc41ff8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6353fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6353fff640 1 -- 172.21.15.103:0/2583390890 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364011e00 con 0x55ed705ed680 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f6353fff640 1 -- 172.21.15.103:0/2583390890 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f63640cf110 con 0x55ed705ed680 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.081+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f635c012aa0 con 0x55ed705ed680 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f6353fff640 1 -- 172.21.15.103:0/2583390890 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f636401ae40 con 0x55ed705ed680 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 10 client.16941 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ed70601c50 con 0x55ed705ed680 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f6353fff640 1 -- 172.21.15.103:0/2583390890 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f635c012aa0 con 0x55ed705ed680 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f6353fff640 1 -- 172.21.15.103:0/2583390890 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f6364022030 con 0x55ed705ed680 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 10 client.16941 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 20 client.16941 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed706032d0 0x55ed706056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 4 client.16941 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.082+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7055e100 con 0x55ed706032d0 2024-08-19T01:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.083+0000 7f6373fff640 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed706032d0 0x55ed706056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.084+0000 7f6373fff640 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed706032d0 0x55ed706056e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564129e14745dec2 server_cookie=a3a8be91284dfdfa in_seq=0 out_seq=0 2024-08-19T01:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.084+0000 7f6353fff640 10 client.16941 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f6353fff640 1 -- 172.21.15.103:0/2583390890 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed7055e100 con 0x55ed706032d0 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f6353fff640 10 client.16941 handle_command_reply: tid=0 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 client.16941 shutdown 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705ed680 msgr2=0x55ed705f7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705ed680 0x55ed705f7830 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f636400ea50 tx=0x7f636409bbf0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 shutdown_connections 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed706032d0 0x55ed706056e0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f6353fff640 0 client.16941 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705edfc0 0x55ed705f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 --2- 172.21.15.103:0/2583390890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705ed680 0x55ed705f7830 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 >> 172.21.15.103:0/2583390890 conn(0x55ed705e8cf0 msgr2=0x55ed705ea500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 shutdown_connections 2024-08-19T01:41:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 1 -- 172.21.15.103:0/2583390890 wait complete. 2024-08-19T01:41:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.085+0000 7f637dec2740 20 client.16941 trim_cache size 0 max 16384 2024-08-19T01:41:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 Processor -- start 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 -- start start 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705e3d80 0x55ed705e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63741af0d0 con 0x55ed705edfc0 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63741b0c40 con 0x55ed705f3050 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f637dec2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63741af7a0 con 0x55ed705e3d80 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54334/0 (socket says 172.21.15.103:54334) 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 -- 172.21.15.103:0/2610672532 learned_addr learned my addr 172.21.15.103:0/2610672532 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6373fff640 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f63796a2640 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705e3d80 0x55ed705e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 -- 172.21.15.103:0/2610672532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705e3d80 msgr2=0x55ed705e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705e3d80 0x55ed705e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 -- 172.21.15.103:0/2610672532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 msgr2=0x55ed705fed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.105+0000 7f6378ea1640 1 -- 172.21.15.103:0/2610672532 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed705ee4a0 con 0x55ed705edfc0 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.106+0000 7f6378ea1640 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f7e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f63640987f0 tx=0x7f63640bc050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1542fc636c8436b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.106+0000 7f636a7fc640 1 -- 172.21.15.103:0/2610672532 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364011e00 con 0x55ed705edfc0 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.106+0000 7f636a7fc640 1 -- 172.21.15.103:0/2610672532 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f636409c520 con 0x55ed705edfc0 2024-08-19T01:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.106+0000 7f636a7fc640 1 -- 172.21.15.103:0/2610672532 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636409b430 con 0x55ed705edfc0 2024-08-19T01:41:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 -- 172.21.15.103:0/2610672532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 msgr2=0x55ed705f7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f7e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f63640987f0 tx=0x7f63640bc050 comp rx=0 tx=0).stop 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 -- 172.21.15.103:0/2610672532 shutdown_connections 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705e3d80 0x55ed705e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705f3050 0x55ed705fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 --2- 172.21.15.103:0/2610672532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705f7e00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 -- 172.21.15.103:0/2610672532 >> 172.21.15.103:0/2610672532 conn(0x55ed705e8cf0 msgr2=0x55ed705e7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 -- 172.21.15.103:0/2610672532 shutdown_connections 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 -- 172.21.15.103:0/2610672532 wait complete. 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.107+0000 7f637dec2740 1 -- start start 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 1 Processor -- start 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705fdf60 0x55ed705fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63741b0570 con 0x55ed705edfc0 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63741afe70 con 0x55ed705fdf60 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ad0b0 con 0x55ed705f3050 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f637dec2740 10 client.0 _ll_register_callbacks cb 0x7f636401a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f6373fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f6373fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705fcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:56686/0 (socket says 172.21.15.103:56686) 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f6373fff640 1 -- 172.21.15.103:0/378757173 learned_addr learned my addr 172.21.15.103:0/378757173 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f6378ea1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.108+0000 7f63796a2640 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705fdf60 0x55ed705fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6378ea1640 1 -- 172.21.15.103:0/378757173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 msgr2=0x55ed705fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6378ea1640 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6378ea1640 1 -- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705fdf60 msgr2=0x55ed705fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6378ea1640 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705fdf60 0x55ed705fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6378ea1640 1 -- 172.21.15.103:0/378757173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed705fe950 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6378ea1640 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705fca10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x55ed70601b60 tx=0x7f63640980c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464220cea5e875db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6353fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6353fff640 1 -- 172.21.15.103:0/378757173 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636407f070 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6353fff640 1 -- 172.21.15.103:0/378757173 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6364011470 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636409c520 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6353fff640 1 -- 172.21.15.103:0/378757173 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6364011e00 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f637dec2740 10 client.16947 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ed7066e8c0 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.109+0000 7f6353fff640 1 -- 172.21.15.103:0/378757173 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f636409c520 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f6353fff640 1 -- 172.21.15.103:0/378757173 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f63640a4ce0 con 0x55ed705edfc0 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f637dec2740 10 client.16947 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f637dec2740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f637dec2740 20 client.16947 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f637dec2740 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed7066f190 0x55ed706715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f637dec2740 4 client.16947 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:41:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed70558450 con 0x55ed7066f190 2024-08-19T01:41:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f6373fff640 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed7066f190 0x55ed706715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f6353fff640 10 client.16947 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.110+0000 7f6373fff640 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed7066f190 0x55ed706715b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb11f1ed8229b51 server_cookie=32420f7cd50627fc in_seq=0 out_seq=0 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.116+0000 7f6353fff640 1 -- 172.21.15.103:0/378757173 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed70558450 con 0x55ed7066f190 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.116+0000 7f6353fff640 10 client.16947 handle_command_reply: tid=0 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.116+0000 7f637dec2740 1 client.16947 shutdown 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.116+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 msgr2=0x55ed705fca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.116+0000 7f637dec2740 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705fca10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x55ed70601b60 tx=0x7f63640980c0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 shutdown_connections 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed7066f190 0x55ed706715b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed705fdf60 0x55ed705fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed705f3050 0x55ed705fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 --2- 172.21.15.103:0/378757173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed705edfc0 0x55ed705fca10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f6353fff640 0 client.16947 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 >> 172.21.15.103:0/378757173 conn(0x55ed705e8cf0 msgr2=0x55ed705e7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 shutdown_connections 2024-08-19T01:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 1 -- 172.21.15.103:0/378757173 wait complete. 2024-08-19T01:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637dec2740 20 client.16947 trim_cache size 0 max 16384 2024-08-19T01:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:41:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63600a2cb0 msgr2=0x7f63600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637b105640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63600a2cb0 0x7f63600a5170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f636c0095f0 tx=0x7f636c0056d0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.117+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374100770 msgr2=0x7f637419dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374100770 0x7f637419dff0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f636400ea80 tx=0x7f636400ef40 comp rx=0 tx=0).stop 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 shutdown_connections 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63600a2cb0 0x7f63600a5170 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6374102320 0x7f63741a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6374101960 0x7f637419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 --2- 172.21.15.103:0/531493331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6374100770 0x7f637419dff0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 >> 172.21.15.103:0/531493331 conn(0x7f63740fbf10 msgr2=0x7f637410bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 shutdown_connections 2024-08-19T01:41:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:36.118+0000 7f637b105640 1 -- 172.21.15.103:0/531493331 wait complete. 2024-08-19T01:41:36.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:35 smithi103 ceph-mon[18436]: pgmap v513: 97 pgs: 97 active+clean; 8.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:41:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:35 smithi190 ceph-mon[22032]: pgmap v513: 97 pgs: 97 active+clean; 8.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:41:36.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:35 smithi146 ceph-mon[22098]: pgmap v513: 97 pgs: 97 active+clean; 8.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-08-19T01:41:36.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 -- 172.21.15.103:0/1250473224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c102a60 msgr2=0x7fc23c102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 --2- 172.21.15.103:0/1250473224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c102a60 0x7fc23c102e40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009930 tx=0x7fc22c02a420 comp rx=0 tx=0).stop 2024-08-19T01:41:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 -- 172.21.15.103:0/1250473224 shutdown_connections 2024-08-19T01:41:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 --2- 172.21.15.103:0/1250473224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc23c107fd0 0x7fc23c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 --2- 172.21.15.103:0/1250473224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c103410 0x7fc23c1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 --2- 172.21.15.103:0/1250473224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c102a60 0x7fc23c102e40 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 -- 172.21.15.103:0/1250473224 >> 172.21.15.103:0/1250473224 conn(0x7fc23c0fc500 msgr2=0x7fc23c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 -- 172.21.15.103:0/1250473224 shutdown_connections 2024-08-19T01:41:36.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.031+0000 7fc2409af640 1 -- 172.21.15.103:0/1250473224 wait complete. 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.032+0000 7fc2409af640 1 Processor -- start 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.032+0000 7fc2409af640 1 -- start start 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.032+0000 7fc2409af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c102a60 0x7fc23c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.032+0000 7fc2409af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c103410 0x7fc23c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc2409af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc23c107fd0 0x7fc23c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c103410 0x7fc23c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc2409af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c111c80 con 0x7fc23c103410 2024-08-19T01:41:36.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc2409af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c111b00 con 0x7fc23c102a60 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc2409af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c111e00 con 0x7fc23c107fd0 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c103410 0x7fc23c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54258/0 (socket says 172.21.15.103:54258) 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 -- 172.21.15.103:0/3256302592 learned_addr learned my addr 172.21.15.103:0/3256302592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc23a59c640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c102a60 0x7fc23c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc23c107fd0 0x7fc23c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 -- 172.21.15.103:0/3256302592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc23c107fd0 msgr2=0x7fc23c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc23c107fd0 0x7fc23c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 -- 172.21.15.103:0/3256302592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c102a60 msgr2=0x7fc23c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c102a60 0x7fc23c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 -- 172.21.15.103:0/3256302592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c19d1b0 con 0x7fc23c103410 2024-08-19T01:41:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc23c107fd0 0x7fc23c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.033+0000 7fc239d9b640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c103410 0x7fc23c198740 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc23000d950 tx=0x7fc23000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.034+0000 7fc22b7fe640 1 -- 172.21.15.103:0/3256302592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230017070 con 0x7fc23c103410 2024-08-19T01:41:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.034+0000 7fc22b7fe640 1 -- 172.21.15.103:0/3256302592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc230009c30 con 0x7fc23c103410 2024-08-19T01:41:36.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.034+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c19d4a0 con 0x7fc23c103410 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.034+0000 7fc22b7fe640 1 -- 172.21.15.103:0/3256302592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2300047a0 con 0x7fc23c103410 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.034+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c1005f0 con 0x7fc23c103410 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.036+0000 7fc22b7fe640 1 -- 172.21.15.103:0/3256302592 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc230004020 con 0x7fc23c103410 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.037+0000 7fc22b7fe640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2240a2c00 0x7fc2240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.037+0000 7fc22b7fe640 1 -- 172.21.15.103:0/3256302592 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc2300cecd0 con 0x7fc23c103410 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.037+0000 7fc23a59c640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2240a2c00 0x7fc2240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 Processor -- start 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 -- start start 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6120 0x5633f11a6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc23a59c640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2240a2c00 0x7fc2240a50c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009900 tx=0x7fc22c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11b9bf0 0x5633f11bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5633f1169e40 con 0x5633f11a6a40 2024-08-19T01:41:36.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633f110b7f0 con 0x5633f11b9bf0 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc242dad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5633f10b2fb0 con 0x5633f11a6120 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc239d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc239d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54260/0 (socket says 172.21.15.103:54260) 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc239d9b640 1 -- 172.21.15.103:0/2651523553 learned_addr learned my addr 172.21.15.103:0/2651523553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.038+0000 7fc23a59c640 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6120 0x5633f11a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11b9bf0 0x5633f11bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc239d9b640 1 -- 172.21.15.103:0/2651523553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6120 msgr2=0x5633f11a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc239d9b640 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6120 0x5633f11a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc239d9b640 1 -- 172.21.15.103:0/2651523553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11b9bf0 msgr2=0x5633f11bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc239d9b640 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11b9bf0 0x5633f11bbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc239d9b640 1 -- 172.21.15.103:0/2651523553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f0e7d6a0 con 0x5633f11a6a40 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc239d9b640 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b96b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc23009b490 tx=0x7fc2300056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f0eacc21cc94d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:37.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc228ff9640 1 -- 172.21.15.103:0/2651523553 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230009de0 con 0x5633f11a6a40 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc228ff9640 1 -- 172.21.15.103:0/2651523553 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc23009b500 con 0x5633f11a6a40 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.039+0000 7fc228ff9640 1 -- 172.21.15.103:0/2651523553 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230004460 con 0x5633f11a6a40 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 -- 172.21.15.103:0/2651523553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 msgr2=0x5633f11b96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b96b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc23009b490 tx=0x7fc2300056a0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 -- 172.21.15.103:0/2651523553 shutdown_connections 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11b9bf0 0x5633f11bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b96b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 --2- 172.21.15.103:0/2651523553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6120 0x5633f11a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 -- 172.21.15.103:0/2651523553 >> 172.21.15.103:0/2651523553 conn(0x5633f11a2790 msgr2=0x5633f11a2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 -- 172.21.15.103:0/2651523553 shutdown_connections 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.040+0000 7fc242dad740 1 -- 172.21.15.103:0/2651523553 wait complete. 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 -- start start 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 Processor -- start 2024-08-19T01:41:37.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6a40 0x5633f11b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11b9bf0 0x5633f11b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5633f10c6cb0 con 0x5633f11b9bf0 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633f11077b0 con 0x5633f11a6120 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5633f11bc700 con 0x5633f11a6a40 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc242dad740 10 client.0 _ll_register_callbacks cb 0x5633f0e1aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc23ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11b9bf0 0x5633f11b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc23a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc23ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11b9bf0 0x5633f11b75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54264/0 (socket says 172.21.15.103:54264) 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc23ad9d640 1 -- 172.21.15.103:0/2278638256 learned_addr learned my addr 172.21.15.103:0/2278638256 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.041+0000 7fc239d9b640 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6a40 0x5633f11b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc23ad9d640 1 -- 172.21.15.103:0/2278638256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6a40 msgr2=0x5633f11b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6a40 0x5633f11b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc23ad9d640 1 -- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 msgr2=0x5633f11b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc23ad9d640 1 -- 172.21.15.103:0/2278638256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f10c5bc0 con 0x5633f11b9bf0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc23a59c640 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11b9bf0 0x5633f11b75e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc234012a10 tx=0x7fc234012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd10f8e0f64676e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc221ffb640 1 -- 172.21.15.103:0/2278638256 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234010c50 con 0x5633f11b9bf0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc221ffb640 1 -- 172.21.15.103:0/2278638256 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc234010df0 con 0x5633f11b9bf0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.042+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23009b500 con 0x5633f11b9bf0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc221ffb640 1 -- 172.21.15.103:0/2278638256 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc23401c410 con 0x5633f11b9bf0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 10 client.16923 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5633f11c8730 con 0x5633f11b9bf0 2024-08-19T01:41:37.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc221ffb640 1 -- 172.21.15.103:0/2278638256 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc23009b500 con 0x5633f11b9bf0 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc221ffb640 1 -- 172.21.15.103:0/2278638256 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fc234028820 con 0x5633f11b9bf0 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 10 client.16923 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 20 client.16923 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f11c9b90 0x5633f11cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 4 client.16923 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.043+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633f1118d60 con 0x5633f11c9b90 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.044+0000 7fc23a59c640 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f11c9b90 0x5633f11cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.044+0000 7fc221ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.044+0000 7fc23a59c640 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f11c9b90 0x5633f11cbfb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b0c38f772d4dbc server_cookie=ce3dbebd367a518 in_seq=0 out_seq=0 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc221ffb640 1 -- 172.21.15.103:0/2278638256 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633f1118d60 con 0x5633f11c9b90 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc221ffb640 10 client.16923 handle_command_reply: tid=0 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 client.16923 shutdown 2024-08-19T01:41:37.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11b9bf0 msgr2=0x5633f11b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11b9bf0 0x5633f11b75e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc234012a10 tx=0x7fc234012ed0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 shutdown_connections 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f11c9b90 0x5633f11cbfb0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11b9bf0 0x5633f11b75e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11a6a40 0x5633f11b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 --2- 172.21.15.103:0/2278638256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc221ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 >> 172.21.15.103:0/2278638256 conn(0x5633f11a2790 msgr2=0x5633f11a8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 shutdown_connections 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 1 -- 172.21.15.103:0/2278638256 wait complete. 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.045+0000 7fc242dad740 20 client.16923 trim_cache size 0 max 16384 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.063+0000 7fc242dad740 1 Processor -- start 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.063+0000 7fc242dad740 1 -- start start 2024-08-19T01:41:37.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.063+0000 7fc242dad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc242dad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc242dad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc242dad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c1aaf90 con 0x5633f11a6a40 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc242dad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c1acb00 con 0x5633f11a6120 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc242dad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c1ab660 con 0x5633f11b9bf0 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54274/0 (socket says 172.21.15.103:54274) 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 -- 172.21.15.103:0/551320853 learned_addr learned my addr 172.21.15.103:0/551320853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc23a59c640 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 -- 172.21.15.103:0/551320853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 msgr2=0x5633f11d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 -- 172.21.15.103:0/551320853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 msgr2=0x5633f11b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 -- 172.21.15.103:0/551320853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f11a7450 con 0x5633f11a6a40 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.064+0000 7fc239d9b640 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b5b40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc230005fd0 tx=0x7fc23000b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2202f698d90bf7c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.065+0000 7fc223fff640 1 -- 172.21.15.103:0/551320853 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230009de0 con 0x5633f11a6a40 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.065+0000 7fc223fff640 1 -- 172.21.15.103:0/551320853 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc230065070 con 0x5633f11a6a40 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.065+0000 7fc223fff640 1 -- 172.21.15.103:0/551320853 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230097b70 con 0x5633f11a6a40 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.065+0000 7fc242dad740 1 -- 172.21.15.103:0/551320853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 msgr2=0x5633f11b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b5b40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc230005fd0 tx=0x7fc23000b710 comp rx=0 tx=0).stop 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 -- 172.21.15.103:0/551320853 shutdown_connections 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11b5b40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 --2- 172.21.15.103:0/551320853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 -- 172.21.15.103:0/551320853 >> 172.21.15.103:0/551320853 conn(0x5633f11a2790 msgr2=0x5633f11b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:37.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 -- 172.21.15.103:0/551320853 shutdown_connections 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 -- 172.21.15.103:0/551320853 wait complete. 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 -- start start 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 Processor -- start 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.066+0000 7fc242dad740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc242dad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc242dad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11bd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc242dad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c1ac430 con 0x5633f11a6a40 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc242dad740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c1abd30 con 0x5633f11a6120 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc242dad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c1a8f70 con 0x5633f11b9bf0 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc242dad740 10 client.0 _ll_register_callbacks cb 0x7fc234009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:36880/0 (socket says 172.21.15.103:36880) 2024-08-19T01:41:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 -- 172.21.15.103:0/398210132 learned_addr learned my addr 172.21.15.103:0/398210132 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc239d9b640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23ad9d640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11bd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 -- 172.21.15.103:0/398210132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 msgr2=0x5633f11bd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11bd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 -- 172.21.15.103:0/398210132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 msgr2=0x5633f11bd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:37.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11bd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 -- 172.21.15.103:0/398210132 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f11be080 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc239d9b640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11bd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc23a59c640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11c1fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0096c0 tx=0x7fc22c03d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62ac6ec8c2b0c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.067+0000 7fc221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc221ffb640 1 -- 172.21.15.103:0/398210132 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c035040 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc221ffb640 1 -- 172.21.15.103:0/398210132 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc22c003940 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc230065070 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc221ffb640 1 -- 172.21.15.103:0/398210132 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc22c003b40 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 10 client.26276 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5633f1234790 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc221ffb640 1 -- 172.21.15.103:0/398210132 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc230065070 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc221ffb640 1 -- 172.21.15.103:0/398210132 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7fc22c049420 con 0x5633f11a6120 2024-08-19T01:41:37.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 10 client.26276 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 20 client.26276 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f1234f30 0x5633f1237350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 4 client.26276 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.068+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5633f1113270 con 0x5633f1234f30 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.069+0000 7fc239d9b640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f1234f30 0x5633f1237350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.069+0000 7fc221ffb640 10 client.26276 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:41:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "4cfc91f9-7bf5-4e23-861c-2ffbdfed9b00", 2024-08-19T01:41:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:41:37.012 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:41:37.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.069+0000 7fc239d9b640 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f1234f30 0x5633f1237350 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f15ad71bc0594bcd server_cookie=b4b53590f6cd7577 in_seq=0 out_seq=0 2024-08-19T01:41:37.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc221ffb640 1 -- 172.21.15.103:0/398210132 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5633f1113270 con 0x5633f1234f30 2024-08-19T01:41:37.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc221ffb640 10 client.26276 handle_command_reply: tid=0 2024-08-19T01:41:37.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc242dad740 1 client.26276 shutdown 2024-08-19T01:41:37.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 msgr2=0x5633f11c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:37.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc242dad740 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11c1fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0096c0 tx=0x7fc22c03d000 comp rx=0 tx=0).stop 2024-08-19T01:41:37.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 shutdown_connections 2024-08-19T01:41:37.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc242dad740 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5633f1234f30 0x5633f1237350 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc242dad740 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633f11b9bf0 0x5633f11bd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.070+0000 7fc221ffb640 0 client.26276 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:37.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.071+0000 7fc242dad740 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633f11a6a40 0x5633f11bd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.071+0000 7fc242dad740 1 --2- 172.21.15.103:0/398210132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633f11a6120 0x5633f11c1fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.071+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 >> 172.21.15.103:0/398210132 conn(0x5633f11a2790 msgr2=0x5633f11b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:37.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.071+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 shutdown_connections 2024-08-19T01:41:37.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.071+0000 7fc242dad740 1 -- 172.21.15.103:0/398210132 wait complete. 2024-08-19T01:41:37.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.071+0000 7fc242dad740 20 client.26276 trim_cache size 0 max 16384 2024-08-19T01:41:37.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.072+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2240a2c00 msgr2=0x7fc2240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:37.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.072+0000 7fc2409af640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2240a2c00 0x7fc2240a50c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009900 tx=0x7fc22c000bc0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.072+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c103410 msgr2=0x7fc23c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.072+0000 7fc2409af640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c103410 0x7fc23c198740 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc23000d950 tx=0x7fc23000de10 comp rx=0 tx=0).stop 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 shutdown_connections 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2240a2c00 0x7fc2240a50c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc23c107fd0 0x7fc23c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc23c103410 0x7fc23c198740 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 --2- 172.21.15.103:0/3256302592 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc23c102a60 0x7fc23c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 >> 172.21.15.103:0/3256302592 conn(0x7fc23c0fc500 msgr2=0x7fc23c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:37.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 shutdown_connections 2024-08-19T01:41:37.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:36.073+0000 7fc2409af640 1 -- 172.21.15.103:0/3256302592 wait complete. 2024-08-19T01:41:37.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:41:37.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:36 smithi103 ceph-mon[18436]: pgmap v514: 97 pgs: 97 active+clean; 8.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:41:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:41:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:41:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:41:37.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:36 smithi190 ceph-mon[22032]: pgmap v514: 97 pgs: 97 active+clean; 8.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:41:37.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:41:37.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:41:37.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:36 smithi146 ceph-mon[22098]: pgmap v514: 97 pgs: 97 active+clean; 8.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-08-19T01:41:38.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:41:38.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:41:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:41:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:41:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:41:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:41:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:41:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:41:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:41:39.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:38 smithi103 ceph-mon[18436]: pgmap v515: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 35 op/s 2024-08-19T01:41:39.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:38 smithi190 ceph-mon[22032]: pgmap v515: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 35 op/s 2024-08-19T01:41:39.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:38 smithi146 ceph-mon[22098]: pgmap v515: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 MiB/s wr, 35 op/s 2024-08-19T01:41:40.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:41:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:41:41.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:41 smithi103 ceph-mon[18436]: pgmap v516: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-08-19T01:41:41.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:41 smithi190 ceph-mon[22032]: pgmap v516: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-08-19T01:41:41.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:41 smithi146 ceph-mon[22098]: pgmap v516: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 42 op/s 2024-08-19T01:41:41.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 -- 172.21.15.103:0/308715922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0100790 msgr2=0x7f8fe0100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 --2- 172.21.15.103:0/308715922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0100790 0x7f8fe0100bf0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009930 tx=0x7f8fd402a420 comp rx=0 tx=0).stop 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 -- 172.21.15.103:0/308715922 shutdown_connections 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 --2- 172.21.15.103:0/308715922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe0101130 0x7f8fe010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 --2- 172.21.15.103:0/308715922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0100790 0x7f8fe0100bf0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 --2- 172.21.15.103:0/308715922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe0106790 0x7f8fe0106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 -- 172.21.15.103:0/308715922 >> 172.21.15.103:0/308715922 conn(0x7f8fe00fc4b0 msgr2=0x7f8fe00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 -- 172.21.15.103:0/308715922 shutdown_connections 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.793+0000 7f8fe6780640 1 -- 172.21.15.103:0/308715922 wait complete. 2024-08-19T01:41:41.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.794+0000 7f8fe6780640 1 Processor -- start 2024-08-19T01:41:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.794+0000 7f8fe6780640 1 -- start start 2024-08-19T01:41:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.794+0000 7f8fe6780640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe0100790 0x7f8fe019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.794+0000 7f8fe6780640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0101130 0x7f8fe019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.794+0000 7f8fe6780640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe0106790 0x7f8fe019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fe6780640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0110a40 con 0x7f8fe0101130 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fe6780640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01108c0 con 0x7f8fe0106790 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fe6780640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0110bc0 con 0x7f8fe0100790 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0101130 0x7f8fe019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0101130 0x7f8fe019a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54366/0 (socket says 172.21.15.103:54366) 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 -- 172.21.15.103:0/3001942058 learned_addr learned my addr 172.21.15.103:0/3001942058 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdffff640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe0100790 0x7f8fe019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fe4d1d640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe0106790 0x7f8fe019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 -- 172.21.15.103:0/3001942058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe0100790 msgr2=0x7f8fe019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe0100790 0x7f8fe019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 -- 172.21.15.103:0/3001942058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe0106790 msgr2=0x7f8fe019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe0106790 0x7f8fe019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 -- 172.21.15.103:0/3001942058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe019f200 con 0x7f8fe0101130 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0101130 0x7f8fe019a830 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4004770 tx=0x7f8fd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdd7fa640 1 -- 172.21.15.103:0/3001942058 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd403f070 con 0x7f8fe0101130 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe019f490 con 0x7f8fe0101130 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdd7fa640 1 -- 172.21.15.103:0/3001942058 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd403b460 con 0x7f8fe0101130 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.795+0000 7f8fdd7fa640 1 -- 172.21.15.103:0/3001942058 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4035070 con 0x7f8fe0101130 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.796+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe01a6dd0 con 0x7f8fe0101130 2024-08-19T01:41:41.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.798+0000 7f8fdd7fa640 1 -- 172.21.15.103:0/3001942058 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8fd403b610 con 0x7f8fe0101130 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.798+0000 7f8fe8b7e740 1 Processor -- start 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.798+0000 7f8fe8b7e740 1 -- start start 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.798+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df29205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.798+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920b30 0x556df292aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df292d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556df2885980 con 0x556df2920b30 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556df281da70 con 0x556df2920210 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556df2881940 con 0x556df292b3f0 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920b30 0x556df292aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920b30 0x556df292aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54374/0 (socket says 172.21.15.103:54374) 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 -- 172.21.15.103:0/4016839584 learned_addr learned my addr 172.21.15.103:0/4016839584 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdd7fa640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8fc80a2b80 0x7f8fc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 -- 172.21.15.103:0/4016839584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 msgr2=0x556df292d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df292d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fe4d1d640 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df29205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df292d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 -- 172.21.15.103:0/4016839584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 msgr2=0x556df29205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df29205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 -- 172.21.15.103:0/4016839584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df28806e0 con 0x556df2920b30 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdd7fa640 1 -- 172.21.15.103:0/3001942058 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8fd40ef3d0 con 0x7f8fe0101130 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fe4d1d640 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df29205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fe4d1d640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8fc80a2b80 0x7f8fc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df292d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.799+0000 7f8fdffff640 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920b30 0x556df292aeb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000bc90 tx=0x7f8fd00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5907842a967090 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fc67fc640 1 -- 172.21.15.103:0/4016839584 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd001c480 con 0x556df2920b30 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fc67fc640 1 -- 172.21.15.103:0/4016839584 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd001cad0 con 0x556df2920b30 2024-08-19T01:41:41.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fc67fc640 1 -- 172.21.15.103:0/4016839584 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0011bb0 con 0x556df2920b30 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/4016839584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920b30 msgr2=0x556df292aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920b30 0x556df292aeb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000bc90 tx=0x7f8fd00107a0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe4d1d640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8fc80a2b80 0x7f8fc80a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8007e20 tx=0x7f8fd80074a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/4016839584 shutdown_connections 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df292d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920b30 0x556df292aeb0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/4016839584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df29205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.800+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/4016839584 >> 172.21.15.103:0/4016839584 conn(0x556df291c7a0 msgr2=0x556df291cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.801+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/4016839584 shutdown_connections 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.801+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/4016839584 wait complete. 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.801+0000 7f8fe8b7e740 1 -- start start 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.801+0000 7f8fe8b7e740 1 Processor -- start 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df2928290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df29287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df292b3f0 0x556df2928d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556df292dfc0 con 0x556df2920210 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556df2840aa0 con 0x556df292b3f0 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556df292e140 con 0x556df2920b30 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe8b7e740 10 client.0 _ll_register_callbacks cb 0x556df282fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df2928290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df2928290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54376/0 (socket says 172.21.15.103:54376) 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 -- 172.21.15.103:0/652874739 learned_addr learned my addr 172.21.15.103:0/652874739 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df29287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fe4d1d640 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df292b3f0 0x556df2928d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 -- 172.21.15.103:0/652874739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 msgr2=0x556df29287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df29287d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 -- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df292b3f0 msgr2=0x556df2928d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df292b3f0 0x556df2928d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 -- 172.21.15.103:0/652874739 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df2824650 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fdffff640 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df2928290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0000c00 tx=0x7f8fd000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790ebec909e5c585 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/652874739 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd001c890 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.802+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/652874739 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8fd0011930 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd001cad0 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/652874739 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8fd002e050 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 10 client.16959 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556df25f73c0 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/652874739 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd0005030 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/652874739 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f8fd0004750 con 0x556df2920210 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:41.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 20 client.16959 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df293b0b0 0x556df293d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 4 client.16959 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.803+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8fd001cad0 con 0x556df293b0b0 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.804+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df293b0b0 0x556df293d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.804+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df293b0b0 0x556df293d4a0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2a017bc0e7a5ef server_cookie=b57373b7879920db in_seq=0 out_seq=0 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.804+0000 7f8fbf7fe640 10 client.16959 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/652874739 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8fd001cad0 con 0x556df293b0b0 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fbf7fe640 10 client.16959 handle_command_reply: tid=0 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 client.16959 shutdown 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 msgr2=0x556df2928290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df2928290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0000c00 tx=0x7f8fd000b9e0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 shutdown_connections 2024-08-19T01:41:41.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df293b0b0 0x556df293d4a0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df292b3f0 0x556df2928d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df29287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fbf7fe640 0 client.16959 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/652874739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df2928290 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 >> 172.21.15.103:0/652874739 conn(0x556df291c7a0 msgr2=0x556df29224d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 shutdown_connections 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/652874739 wait complete. 2024-08-19T01:41:41.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.805+0000 7f8fe8b7e740 20 client.16959 trim_cache size 0 max 16384 2024-08-19T01:41:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 Processor -- start 2024-08-19T01:41:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 -- start start 2024-08-19T01:41:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df292a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920b30 0x556df29404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df2940a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01af2b0 con 0x556df2920210 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b0e20 con 0x556df2920b30 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01af980 con 0x556df292b3f0 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df292a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df292a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54382/0 (socket says 172.21.15.103:54382) 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdffff640 1 -- 172.21.15.103:0/1534689156 learned_addr learned my addr 172.21.15.103:0/1534689156 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920b30 0x556df29404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fe4d1d640 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df2940a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdffff640 1 -- 172.21.15.103:0/1534689156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 msgr2=0x556df2940a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdffff640 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df2940a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdffff640 1 -- 172.21.15.103:0/1534689156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920b30 msgr2=0x556df29404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.827+0000 7f8fdffff640 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920b30 0x556df29404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.828+0000 7f8fdffff640 1 -- 172.21.15.103:0/1534689156 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df291c5d0 con 0x556df2920210 2024-08-19T01:41:41.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.828+0000 7f8fdffff640 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df292a9f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0004b20 tx=0x7f8fd0010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77450f5050ce615b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:41.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.828+0000 7f8fc5ffb640 1 -- 172.21.15.103:0/1534689156 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000bd80 con 0x556df2920210 2024-08-19T01:41:41.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.828+0000 7f8fc5ffb640 1 -- 172.21.15.103:0/1534689156 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd0037070 con 0x556df2920210 2024-08-19T01:41:41.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.828+0000 7f8fc5ffb640 1 -- 172.21.15.103:0/1534689156 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0011e30 con 0x556df2920210 2024-08-19T01:41:41.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/1534689156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 msgr2=0x556df292a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df292a9f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0004b20 tx=0x7f8fd0010e30 comp rx=0 tx=0).stop 2024-08-19T01:41:41.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/1534689156 shutdown_connections 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df292b3f0 0x556df2940a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920b30 0x556df29404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/1534689156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df2920210 0x556df292a9f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/1534689156 >> 172.21.15.103:0/1534689156 conn(0x556df291c7a0 msgr2=0x556df29224d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/1534689156 shutdown_connections 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/1534689156 wait complete. 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 -- start start 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.829+0000 7f8fe8b7e740 1 Processor -- start 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df299cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df299d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe8b7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df292b3f0 0x556df293da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b0750 con 0x556df292b3f0 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01b0050 con 0x556df2920210 2024-08-19T01:41:41.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe8b7e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01ad290 con 0x556df2920b30 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe8b7e740 10 client.0 _ll_register_callbacks cb 0x7f8fd000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe4d1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df292b3f0 0x556df293da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe4d1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df292b3f0 0x556df293da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54388/0 (socket says 172.21.15.103:54388) 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fe4d1d640 1 -- 172.21.15.103:0/3554024625 learned_addr learned my addr 172.21.15.103:0/3554024625 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df299d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdffff640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df299cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdf7fe640 1 -- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 msgr2=0x556df299cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df299cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdf7fe640 1 -- 172.21.15.103:0/3554024625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df292b3f0 msgr2=0x556df293da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df292b3f0 0x556df293da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdf7fe640 1 -- 172.21.15.103:0/3554024625 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df293e2c0 con 0x556df2920b30 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.830+0000 7f8fdffff640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df299cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:41.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fdf7fe640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df299d2a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x556df291db20 tx=0x7f8fd408abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ca55486a242e4b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/3554024625 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4048040 con 0x556df2920b30 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd0037070 con 0x556df2920b30 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/3554024625 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8fd40ecf50 con 0x556df2920b30 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/3554024625 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8fd40c48d0 con 0x556df2920b30 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fe8b7e740 10 client.26260 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556df29a5590 con 0x556df2920b30 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/3554024625 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd40d1080 con 0x556df2920b30 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/3554024625 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f8fd404b060 con 0x556df2920b30 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fe8b7e740 10 client.26260 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fe8b7e740 10 client.26260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.831+0000 7f8fe8b7e740 20 client.26260 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.832+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df29a5d10 0x556df29a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.832+0000 7f8fe8b7e740 4 client.26260 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:41:41.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.832+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556df2892ef0 con 0x556df29a5d10 2024-08-19T01:41:41.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.832+0000 7f8fdffff640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df29a5d10 0x556df29a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:41.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.832+0000 7f8fbf7fe640 10 client.26260 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:41.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.832+0000 7f8fdffff640 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df29a5d10 0x556df29a8130 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97420b0e84fc7c4 server_cookie=34c28ad1cd46501f in_seq=0 out_seq=0 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fbf7fe640 1 -- 172.21.15.103:0/3554024625 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556df2892ef0 con 0x556df29a5d10 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fbf7fe640 10 client.26260 handle_command_reply: tid=0 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 client.26260 shutdown 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 msgr2=0x556df299d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df299d2a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x556df291db20 tx=0x7f8fd408abe0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 shutdown_connections 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x556df29a5d10 0x556df29a8130 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556df292b3f0 0x556df293da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556df2920b30 0x556df299d2a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fbf7fe640 0 client.26260 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 --2- 172.21.15.103:0/3554024625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556df2920210 0x556df299cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 >> 172.21.15.103:0/3554024625 conn(0x556df291c7a0 msgr2=0x556df2936390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 shutdown_connections 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 1 -- 172.21.15.103:0/3554024625 wait complete. 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.833+0000 7f8fe8b7e740 20 client.26260 trim_cache size 0 max 16384 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:41:41.840 INFO:teuthology.orchestra.run.smithi103.stdout: "4cfc91f9-7bf5-4e23-861c-2ffbdfed9b00": { 2024-08-19T01:41:41.841 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:41:41.841 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "4cfc91f9-7bf5-4e23-861c-2ffbdfed9b00", 2024-08-19T01:41:41.841 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:41:41.841 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:41:41.841 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:41:41.841 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8fc80a2b80 msgr2=0x7f8fc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8fc80a2b80 0x7f8fc80a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8007e20 tx=0x7f8fd80074a0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0101130 msgr2=0x7f8fe019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0101130 0x7f8fe019a830 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4004770 tx=0x7f8fd40042f0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 shutdown_connections 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8fc80a2b80 0x7f8fc80a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8fe0106790 0x7f8fe019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fe0101130 0x7f8fe019a830 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 --2- 172.21.15.103:0/3001942058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe0100790 0x7f8fe019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:41.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 >> 172.21.15.103:0/3001942058 conn(0x7f8fe00fc4b0 msgr2=0x7f8fe006a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:41.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 shutdown_connections 2024-08-19T01:41:41.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:41:41.834+0000 7f8fe6780640 1 -- 172.21.15.103:0/3001942058 wait complete. 2024-08-19T01:41:42.574 INFO:tasks.cephfs.filesystem:scrub status for tag:4cfc91f9-7bf5-4e23-861c-2ffbdfed9b00 - {'path': '/', 'tag': '4cfc91f9-7bf5-4e23-861c-2ffbdfed9b00', 'options': 'recursive,force'} 2024-08-19T01:41:43.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:43 smithi103 ceph-mon[18436]: pgmap v517: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 47 op/s 2024-08-19T01:41:43.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:43 smithi190 ceph-mon[22032]: pgmap v517: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 47 op/s 2024-08-19T01:41:43.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:43 smithi146 ceph-mon[22098]: pgmap v517: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 27 MiB/s wr, 47 op/s 2024-08-19T01:41:45.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:45 smithi103 ceph-mon[18436]: pgmap v518: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-08-19T01:41:45.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:45 smithi190 ceph-mon[22032]: pgmap v518: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-08-19T01:41:45.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:45 smithi146 ceph-mon[22098]: pgmap v518: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-08-19T01:41:46.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:46 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:41:46.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:46 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:41:46.950 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:46 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:41:47.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:47 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:41:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:47 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:41:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:47 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:41:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:47 smithi103 ceph-mon[18436]: pgmap v519: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:41:47.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:47 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:41:47.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:47 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:41:47.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:47 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:41:47.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:47 smithi190 ceph-mon[22032]: pgmap v519: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:41:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:47 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:41:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:47 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:41:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:47 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:41:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:47 smithi146 ceph-mon[22098]: pgmap v519: 97 pgs: 97 active+clean; 8.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.231+0000 7f6348e75640 1 -- 172.21.15.103:0/377255344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 msgr2=0x7f63441046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.231+0000 7f6348e75640 1 --2- 172.21.15.103:0/377255344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f63441046e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6334009900 tx=0x7f633402a410 comp rx=0 tx=0).stop 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 -- 172.21.15.103:0/377255344 shutdown_connections 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 --2- 172.21.15.103:0/377255344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6344107550 0x7f63441099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 --2- 172.21.15.103:0/377255344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6344104c20 0x7f6344107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 --2- 172.21.15.103:0/377255344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f63441046e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 -- 172.21.15.103:0/377255344 >> 172.21.15.103:0/377255344 conn(0x7f63440fbf10 msgr2=0x7f63440fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 -- 172.21.15.103:0/377255344 shutdown_connections 2024-08-19T01:41:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 -- 172.21.15.103:0/377255344 wait complete. 2024-08-19T01:41:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 Processor -- start 2024-08-19T01:41:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.232+0000 7f6348e75640 1 -- start start 2024-08-19T01:41:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6348e75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f6344197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6348e75640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6344104c20 0x7f6344197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6348e75640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6344107550 0x7f634419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6348e75640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f634410d460 con 0x7f63441022f0 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6348e75640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f634410d2e0 con 0x7f6344107550 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6348e75640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f634410d5e0 con 0x7f6344104c20 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f6344197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f6344197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49138/0 (socket says 172.21.15.103:49138) 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 -- 172.21.15.103:0/865216393 learned_addr learned my addr 172.21.15.103:0/865216393 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f633bfff640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6344104c20 0x7f6344197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6342d9d640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6344107550 0x7f634419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 -- 172.21.15.103:0/865216393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6344104c20 msgr2=0x7f6344197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6344104c20 0x7f6344197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 -- 172.21.15.103:0/865216393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6344107550 msgr2=0x7f634419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6344107550 0x7f634419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 -- 172.21.15.103:0/865216393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634419f650 con 0x7f63441022f0 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6342d9d640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6344107550 0x7f634419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f633bfff640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6344104c20 0x7f6344197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f634259c640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f6344197a30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f63340314e0 tx=0x7f6334003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f633b7fe640 1 -- 172.21.15.103:0/865216393 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633403f070 con 0x7f63441022f0 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f633b7fe640 1 -- 172.21.15.103:0/865216393 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6334031e40 con 0x7f63441022f0 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634419f8e0 con 0x7f63441022f0 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.233+0000 7f633b7fe640 1 -- 172.21.15.103:0/865216393 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334035030 con 0x7f63441022f0 2024-08-19T01:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.234+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634419fe20 con 0x7f63441022f0 2024-08-19T01:41:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.236+0000 7f634bc30740 1 Processor -- start 2024-08-19T01:41:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.236+0000 7f634bc30740 1 -- start start 2024-08-19T01:41:49.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f634bc30740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f308ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f634bc30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 0x5654f3090a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f634bc30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f3091050 0x5654f3093510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f634bc30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5654f2fee9d0 con 0x5654f3091050 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f634bc30740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5654f2f96820 con 0x5654f308b680 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f634bc30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5654f2fea990 con 0x5654f308bfc0 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633b7fe640 1 -- 172.21.15.103:0/865216393 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6334042070 con 0x7f63441022f0 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f3091050 0x5654f3093510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f3091050 0x5654f3093510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49152/0 (socket says 172.21.15.103:49152) 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 -- 172.21.15.103:0/3992093485 learned_addr learned my addr 172.21.15.103:0/3992093485 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f6342d9d640 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f308ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.249 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f634259c640 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 0x5654f3090a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 -- 172.21.15.103:0/3992093485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 msgr2=0x5654f3090a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 0x5654f3090a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 -- 172.21.15.103:0/3992093485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 msgr2=0x5654f308ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f308ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 -- 172.21.15.103:0/3992093485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f2d02210 con 0x5654f3091050 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f6342d9d640 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f308ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633bfff640 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f3091050 0x5654f3093510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f632c008c00 tx=0x7f632c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d152e672ede6dd72 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f6338ff9640 1 -- 172.21.15.103:0/3992093485 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c014070 con 0x5654f3091050 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f633b7fe640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63300a2cb0 0x7f63300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f6338ff9640 1 -- 172.21.15.103:0/3992093485 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f632c0047a0 con 0x5654f3091050 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.237+0000 7f6338ff9640 1 -- 172.21.15.103:0/3992093485 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c010670 con 0x5654f3091050 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.238+0000 7f6342d9d640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63300a2cb0 0x7f63300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.238+0000 7f633b7fe640 1 -- 172.21.15.103:0/865216393 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f63340ef940 con 0x7f63441022f0 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.238+0000 7f634bc30740 1 -- 172.21.15.103:0/3992093485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f3091050 msgr2=0x5654f3093510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.238+0000 7f6342d9d640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63300a2cb0 0x7f63300a5170 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f634406a6b0 tx=0x7f633c009590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.238+0000 7f634bc30740 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f3091050 0x5654f3093510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f632c008c00 tx=0x7f632c00e720 comp rx=0 tx=0).stop 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- 172.21.15.103:0/3992093485 shutdown_connections 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f3091050 0x5654f3093510 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 0x5654f3090a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 --2- 172.21.15.103:0/3992093485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f308ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- 172.21.15.103:0/3992093485 >> 172.21.15.103:0/3992093485 conn(0x5654f3086cf0 msgr2=0x5654f30870f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- 172.21.15.103:0/3992093485 shutdown_connections 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- 172.21.15.103:0/3992093485 wait complete. 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- start start 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 Processor -- start 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3095810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308bfc0 0x5654f3095d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 0x5654f309cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5654f3094020 con 0x5654f308b680 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5654f2faa520 con 0x5654f308bfc0 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5654f30941a0 con 0x5654f3091050 2024-08-19T01:41:49.252 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634bc30740 10 client.0 _ll_register_callbacks cb 0x5654f2d01a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3095810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3095810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49156/0 (socket says 172.21.15.103:49156) 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.239+0000 7f634259c640 1 -- 172.21.15.103:0/3649223437 learned_addr learned my addr 172.21.15.103:0/3649223437 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f633bfff640 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308bfc0 0x5654f3095d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f6342d9d640 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 0x5654f309cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634259c640 1 -- 172.21.15.103:0/3649223437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 msgr2=0x5654f309cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634259c640 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 0x5654f309cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634259c640 1 -- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308bfc0 msgr2=0x5654f3095d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634259c640 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308bfc0 0x5654f3095d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634259c640 1 -- 172.21.15.103:0/3649223437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f2dce620 con 0x5654f308b680 2024-08-19T01:41:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634259c640 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3095810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f63340bc060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9730d689b1e90885 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f6329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f6329ffb640 1 -- 172.21.15.103:0/3649223437 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334049050 con 0x5654f308b680 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f6329ffb640 1 -- 172.21.15.103:0/3649223437 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f63340b8840 con 0x5654f308b680 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f632c0047a0 con 0x5654f308b680 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f6329ffb640 1 -- 172.21.15.103:0/3649223437 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f63340bc8d0 con 0x5654f308b680 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634bc30740 10 client.16977 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654f3094020 con 0x5654f308b680 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.240+0000 7f6329ffb640 1 -- 172.21.15.103:0/3649223437 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f632c0047a0 con 0x5654f308b680 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f6329ffb640 1 -- 172.21.15.103:0/3649223437 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f63340dc0b0 con 0x5654f308b680 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f634bc30740 10 client.16977 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:49.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f634bc30740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f634bc30740 20 client.16977 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f634bc30740 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f30a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f634bc30740 4 client.16977 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f2ffc100 con 0x5654f30a11d0 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f633bfff640 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f30a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f633bfff640 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f30a3600 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2107d97690b9822 server_cookie=32e00e595c1ca9db in_seq=0 out_seq=0 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.241+0000 7f6329ffb640 10 client.16977 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f6329ffb640 1 -- 172.21.15.103:0/3649223437 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654f2ffc100 con 0x5654f30a11d0 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f6329ffb640 10 client.16977 handle_command_reply: tid=0 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 client.16977 shutdown 2024-08-19T01:41:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 msgr2=0x5654f3095810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3095810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f63340bc060 comp rx=0 tx=0).stop 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 shutdown_connections 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f30a3600 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 0x5654f309cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f6329ffb640 0 client.16977 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308bfc0 0x5654f3095d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 --2- 172.21.15.103:0/3649223437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3095810 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 >> 172.21.15.103:0/3649223437 conn(0x5654f3086cf0 msgr2=0x5654f3088500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 shutdown_connections 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 1 -- 172.21.15.103:0/3649223437 wait complete. 2024-08-19T01:41:49.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.242+0000 7f634bc30740 20 client.16977 trim_cache size 0 max 16384 2024-08-19T01:41:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.261+0000 7f634bc30740 1 Processor -- start 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.261+0000 7f634bc30740 1 -- start start 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f634bc30740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f30907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f634bc30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308bfc0 0x5654f309cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f634bc30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 0x5654f3095b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f634bc30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a8b00 con 0x5654f308bfc0 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f634bc30740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63441aa670 con 0x5654f308b680 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f634bc30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a91d0 con 0x5654f3091050 2024-08-19T01:41:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308bfc0 0x5654f309cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308bfc0 0x5654f309cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49160/0 (socket says 172.21.15.103:49160) 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 -- 172.21.15.103:0/2941604932 learned_addr learned my addr 172.21.15.103:0/2941604932 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 -- 172.21.15.103:0/2941604932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 msgr2=0x5654f3095b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 --2- 172.21.15.103:0/2941604932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 0x5654f3095b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 -- 172.21.15.103:0/2941604932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 msgr2=0x5654f30907f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 --2- 172.21.15.103:0/2941604932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f30907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 -- 172.21.15.103:0/2941604932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f308c4a0 con 0x5654f308bfc0 2024-08-19T01:41:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.262+0000 7f633bfff640 1 --2- 172.21.15.103:0/2941604932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308bfc0 0x5654f309cd20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f632c00ec20 tx=0x7f632c0084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27ab1100015b69a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.263+0000 7f632bfff640 1 -- 172.21.15.103:0/2941604932 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c014050 con 0x5654f308bfc0 2024-08-19T01:41:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.263+0000 7f632bfff640 1 -- 172.21.15.103:0/2941604932 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f632c005360 con 0x5654f308bfc0 2024-08-19T01:41:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.263+0000 7f632bfff640 1 -- 172.21.15.103:0/2941604932 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c010690 con 0x5654f308bfc0 2024-08-19T01:41:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 -- 172.21.15.103:0/2941604932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308bfc0 msgr2=0x5654f309cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 --2- 172.21.15.103:0/2941604932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308bfc0 0x5654f309cd20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f632c00ec20 tx=0x7f632c0084f0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 -- 172.21.15.103:0/2941604932 shutdown_connections 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 --2- 172.21.15.103:0/2941604932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f3091050 0x5654f3095b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 --2- 172.21.15.103:0/2941604932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308bfc0 0x5654f309cd20 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 --2- 172.21.15.103:0/2941604932 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f308b680 0x5654f30907f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 -- 172.21.15.103:0/2941604932 >> 172.21.15.103:0/2941604932 conn(0x5654f3086cf0 msgr2=0x5654f30953f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 -- 172.21.15.103:0/2941604932 shutdown_connections 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 -- 172.21.15.103:0/2941604932 wait complete. 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 -- start start 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.264+0000 7f634bc30740 1 Processor -- start 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634bc30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3097ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634bc30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 0x5654f3098420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634bc30740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f3091050 0x5654f309b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634bc30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a9fa0 con 0x5654f308b680 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634bc30740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a98a0 con 0x5654f3091050 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634bc30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63441a6ae0 con 0x5654f308bfc0 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634bc30740 10 client.0 _ll_register_callbacks cb 0x7f6334005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3097ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3097ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49168/0 (socket says 172.21.15.103:49168) 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 -- 172.21.15.103:0/4038830222 learned_addr learned my addr 172.21.15.103:0/4038830222 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f6342d9d640 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f3091050 0x5654f309b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 -- 172.21.15.103:0/4038830222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 msgr2=0x5654f3098420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 0x5654f3098420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 -- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f3091050 msgr2=0x5654f309b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f3091050 0x5654f309b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.265+0000 7f634259c640 1 -- 172.21.15.103:0/4038830222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f309ba00 con 0x5654f308b680 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f634259c640 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3097ee0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f63340b8ba0 tx=0x7f63340bc520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d747e52a05102974 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f6329ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f6329ffb640 1 -- 172.21.15.103:0/4038830222 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334049070 con 0x5654f308b680 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f6329ffb640 1 -- 172.21.15.103:0/4038830222 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f63340c0100 con 0x5654f308b680 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f632c005360 con 0x5654f308b680 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f6329ffb640 1 -- 172.21.15.103:0/4038830222 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f63340dc2b0 con 0x5654f308b680 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f634bc30740 10 client.16983 fetch_fsmap learned FSMap version 35 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654f309cd80 con 0x5654f308b680 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.266+0000 7f6329ffb640 1 -- 172.21.15.103:0/4038830222 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f632c005360 con 0x5654f308b680 2024-08-19T01:41:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f6329ffb640 1 -- 172.21.15.103:0/4038830222 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f63340c49b0 con 0x5654f308b680 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f634bc30740 10 client.16983 fetch_fsmap finished waiting for FSMap version 35 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f634bc30740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f634bc30740 20 client.16983 populate_metadata read hostname 'smithi103' 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f634bc30740 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f3112640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f634bc30740 4 client.16983 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654f2ff6450 con 0x5654f30a11d0 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.267+0000 7f633bfff640 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f3112640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.268+0000 7f633bfff640 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f3112640 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f04da3273234c3 server_cookie=42b3848024b31804 in_seq=0 out_seq=0 2024-08-19T01:41:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.268+0000 7f6329ffb640 10 client.16983 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f6329ffb640 1 -- 172.21.15.103:0/4038830222 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654f2ff6450 con 0x5654f30a11d0 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f6329ffb640 10 client.16983 handle_command_reply: tid=0 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 client.16983 shutdown 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 msgr2=0x5654f3097ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3097ee0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f63340b8ba0 tx=0x7f63340bc520 comp rx=0 tx=0).stop 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 shutdown_connections 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5654f30a11d0 0x5654f3112640 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5654f3091050 0x5654f309b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f6329ffb640 0 client.16983 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5654f308bfc0 0x5654f3098420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 --2- 172.21.15.103:0/4038830222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5654f308b680 0x5654f3097ee0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 >> 172.21.15.103:0/4038830222 conn(0x5654f3086cf0 msgr2=0x5654f3095370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 shutdown_connections 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.275+0000 7f634bc30740 1 -- 172.21.15.103:0/4038830222 wait complete. 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f634bc30740 20 client.16983 trim_cache size 0 max 16384 2024-08-19T01:41:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:41:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:41:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:41:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:41:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63300a2cb0 msgr2=0x7f63300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63300a2cb0 0x7f63300a5170 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f634406a6b0 tx=0x7f633c009590 comp rx=0 tx=0).stop 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 msgr2=0x7f6344197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f6344197a30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f63340314e0 tx=0x7f6334003c80 comp rx=0 tx=0).stop 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 shutdown_connections 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f63300a2cb0 0x7f63300a5170 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6344107550 0x7f634419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6344104c20 0x7f6344197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 --2- 172.21.15.103:0/865216393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63441022f0 0x7f6344197a30 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 >> 172.21.15.103:0/865216393 conn(0x7f63440fbf10 msgr2=0x7f6344102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 shutdown_connections 2024-08-19T01:41:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:41:49.276+0000 7f6348e75640 1 -- 172.21.15.103:0/865216393 wait complete. 2024-08-19T01:41:49.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:49 smithi103 ceph-mon[18436]: pgmap v520: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 45 op/s 2024-08-19T01:41:49.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:49 smithi190 ceph-mon[22032]: pgmap v520: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 45 op/s 2024-08-19T01:41:49.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:49 smithi146 ceph-mon[22098]: pgmap v520: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 45 op/s 2024-08-19T01:41:51.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:51 smithi103 ceph-mon[18436]: pgmap v521: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 35 op/s 2024-08-19T01:41:51.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:51 smithi190 ceph-mon[22032]: pgmap v521: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 35 op/s 2024-08-19T01:41:51.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:51 smithi146 ceph-mon[22098]: pgmap v521: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 35 op/s 2024-08-19T01:41:53.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:53 smithi103 ceph-mon[18436]: pgmap v522: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-08-19T01:41:53.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:53 smithi190 ceph-mon[22032]: pgmap v522: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-08-19T01:41:53.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:53 smithi146 ceph-mon[22098]: pgmap v522: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-08-19T01:41:55.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:55 smithi103 ceph-mon[18436]: pgmap v523: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:41:55.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:55 smithi190 ceph-mon[22032]: pgmap v523: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:41:55.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:55 smithi146 ceph-mon[22098]: pgmap v523: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:41:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:57 smithi103 ceph-mon[18436]: pgmap v524: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:41:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:57 smithi190 ceph-mon[22032]: pgmap v524: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:41:57.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:57 smithi146 ceph-mon[22098]: pgmap v524: 97 pgs: 97 active+clean; 8.8 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:41:59.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:41:59 smithi103 ceph-mon[18436]: pgmap v525: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:41:59.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:41:59 smithi190 ceph-mon[22032]: pgmap v525: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:41:59.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:41:59 smithi146 ceph-mon[22098]: pgmap v525: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-08-19T01:42:01.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:01 smithi103 ceph-mon[18436]: pgmap v526: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 24 op/s 2024-08-19T01:42:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:01 smithi190 ceph-mon[22032]: pgmap v526: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 24 op/s 2024-08-19T01:42:01.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:01 smithi146 ceph-mon[22098]: pgmap v526: 97 pgs: 97 active+clean; 8.9 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 24 op/s 2024-08-19T01:42:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 -- 172.21.15.103:0/723081326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8101870 msgr2=0x7f4ee8101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 --2- 172.21.15.103:0/723081326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8101870 0x7f4ee8101c70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009900 tx=0x7f4ed802a410 comp rx=0 tx=0).stop 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 -- 172.21.15.103:0/723081326 shutdown_connections 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 --2- 172.21.15.103:0/723081326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee8103420 0x7f4ee8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 --2- 172.21.15.103:0/723081326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ee8102a60 0x7f4ee8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 --2- 172.21.15.103:0/723081326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8101870 0x7f4ee8101c70 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 -- 172.21.15.103:0/723081326 >> 172.21.15.103:0/723081326 conn(0x7f4ee80fd010 msgr2=0x7f4ee80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 -- 172.21.15.103:0/723081326 shutdown_connections 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.369+0000 7f4eee54d640 1 -- 172.21.15.103:0/723081326 wait complete. 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 Processor -- start 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 -- start start 2024-08-19T01:42:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ee8101870 0x7f4ee8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8102a60 0x7f4ee8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee8103420 0x7f4ee819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee810d3a0 con 0x7f4ee8102a60 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee810d220 con 0x7f4ee8101870 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4eee54d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee810d520 con 0x7f4ee8103420 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4ee77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8102a60 0x7f4ee8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4ee77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8102a60 0x7f4ee8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38068/0 (socket says 172.21.15.103:38068) 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.370+0000 7f4ee77fe640 1 -- 172.21.15.103:0/4281617071 learned_addr learned my addr 172.21.15.103:0/4281617071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee77fe640 1 -- 172.21.15.103:0/4281617071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee8103420 msgr2=0x7f4ee819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee77fe640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee8103420 0x7f4ee819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee77fe640 1 -- 172.21.15.103:0/4281617071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ee8101870 msgr2=0x7f4ee8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee77fe640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ee8101870 0x7f4ee8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee77fe640 1 -- 172.21.15.103:0/4281617071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee819f670 con 0x7f4ee8102a60 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee77fe640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8102a60 0x7f4ee8197f90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00e9f0 tx=0x7f4edc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee57fa640 1 -- 172.21.15.103:0/4281617071 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00cd80 con 0x7f4ee8102a60 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee57fa640 1 -- 172.21.15.103:0/4281617071 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4edc0108e0 con 0x7f4ee8102a60 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4ee57fa640 1 -- 172.21.15.103:0/4281617071 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc018920 con 0x7f4ee8102a60 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee819f960 con 0x7f4ee8102a60 2024-08-19T01:42:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.371+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee819ff00 con 0x7f4ee8102a60 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.373+0000 7f4ee57fa640 1 -- 172.21.15.103:0/4281617071 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4edc010a90 con 0x7f4ee8102a60 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 Processor -- start 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 -- start start 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c507a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ee57fa640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4ed00a2cb0 0x7f4ed00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c50ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c50f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b33c46a9d0 con 0x55b33c50d050 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b33c412820 con 0x55b33c507680 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b33c466990 con 0x55b33c507fc0 2024-08-19T01:42:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4eecaea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c50f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4ed00a2cb0 0x7f4ed00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4eecaea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c50f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38070/0 (socket says 172.21.15.103:38070) 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4eecaea640 1 -- 172.21.15.103:0/1496655368 learned_addr learned my addr 172.21.15.103:0/1496655368 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4eecaea640 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c507a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ee57fa640 1 -- 172.21.15.103:0/4281617071 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4edc0cf3f0 con 0x7f4ee8102a60 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.374+0000 7f4ee77fe640 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c50ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4eecaea640 1 -- 172.21.15.103:0/1496655368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 msgr2=0x55b33c50ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4eecaea640 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c50ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4eecaea640 1 -- 172.21.15.103:0/1496655368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 msgr2=0x55b33c50f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4eecaea640 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c50f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4eecaea640 1 -- 172.21.15.103:0/1496655368 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33c17e210 con 0x55b33c507680 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4ed00a2cb0 0x7f4ed00a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802a930 tx=0x7f4ed80314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4eecaea640 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c507a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0012a10 tx=0x7f4ee0005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00d98c3f91d887f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4ece7fc640 1 -- 172.21.15.103:0/1496655368 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0004d20 con 0x55b33c507680 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4ece7fc640 1 -- 172.21.15.103:0/1496655368 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee000fcd0 con 0x55b33c507680 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.375+0000 7f4ece7fc640 1 -- 172.21.15.103:0/1496655368 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00055b0 con 0x55b33c507680 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1496655368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 msgr2=0x55b33c507a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c507a80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0012a10 tx=0x7f4ee0005bd0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1496655368 shutdown_connections 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c50f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c50ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1496655368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c507a80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1496655368 >> 172.21.15.103:0/1496655368 conn(0x55b33c502cf0 msgr2=0x55b33c5030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1496655368 shutdown_connections 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1496655368 wait complete. 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 -- start start 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.376+0000 7f4ef13e9740 1 Processor -- start 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c511d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c518d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b33c510020 con 0x55b33c50d050 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b33c426520 con 0x55b33c507680 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b33c5101a0 con 0x55b33c507fc0 2024-08-19T01:42:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ef13e9740 10 client.0 _ll_register_callbacks cb 0x55b33c17da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c518d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c518d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38078/0 (socket says 172.21.15.103:38078) 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 -- 172.21.15.103:0/2088531696 learned_addr learned my addr 172.21.15.103:0/2088531696 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ee77fe640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c511d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 -- 172.21.15.103:0/2088531696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 msgr2=0x55b33c511d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c511d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 -- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 msgr2=0x55b33c5117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 -- 172.21.15.103:0/2088531696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33c24a620 con 0x55b33c50d050 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4eecaea640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c518d50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009990 tx=0x7f4ee0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5097ed658a66e91b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.377+0000 7f4ec77fe640 1 -- 172.21.15.103:0/2088531696 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee002a060 con 0x55b33c50d050 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ec77fe640 1 -- 172.21.15.103:0/2088531696 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4ee0004d30 con 0x55b33c50d050 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee000fcd0 con 0x55b33c50d050 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ec77fe640 1 -- 172.21.15.103:0/2088531696 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4ee000f820 con 0x55b33c50d050 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 10 client.16995 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b33c51bb90 con 0x55b33c50d050 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ec77fe640 1 -- 172.21.15.103:0/2088531696 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee000fcd0 con 0x55b33c50d050 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ec77fe640 1 -- 172.21.15.103:0/2088531696 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4ee0005630 con 0x55b33c50d050 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 10 client.16995 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 20 client.16995 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c51f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 4 client.16995 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33c478100 con 0x55b33c51d0e0 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.378+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c51f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.379+0000 7f4ec77fe640 10 client.16995 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.379+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c51f4d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ec0b2f3ff4f2c8 server_cookie=92870448f23af9f2 in_seq=0 out_seq=0 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.379+0000 7f4ec77fe640 1 -- 172.21.15.103:0/2088531696 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b33c478100 con 0x55b33c51d0e0 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.379+0000 7f4ec77fe640 10 client.16995 handle_command_reply: tid=0 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.379+0000 7f4ef13e9740 1 client.16995 shutdown 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 msgr2=0x55b33c518d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c518d50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009990 tx=0x7f4ee0005020 comp rx=0 tx=0).stop 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 shutdown_connections 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c51f4d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c518d50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ec77fe640 0 client.16995 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507fc0 0x55b33c511d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/2088531696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 >> 172.21.15.103:0/2088531696 conn(0x55b33c502cf0 msgr2=0x55b33c504500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 shutdown_connections 2024-08-19T01:42:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 1 -- 172.21.15.103:0/2088531696 wait complete. 2024-08-19T01:42:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.380+0000 7f4ef13e9740 20 client.16995 trim_cache size 0 max 16384 2024-08-19T01:42:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 Processor -- start 2024-08-19T01:42:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 -- start start 2024-08-19T01:42:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 0x55b33c518d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c50d050 0x55b33c511d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81a8c20 con 0x55b33c507680 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81aa760 con 0x55b33c507fc0 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81a92f0 con 0x55b33c50d050 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ee7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ee7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38082/0 (socket says 172.21.15.103:38082) 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.397+0000 7f4ee7fff640 1 -- 172.21.15.103:0/1153171361 learned_addr learned my addr 172.21.15.103:0/1153171361 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4eecaea640 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c50d050 0x55b33c511d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ee7fff640 1 -- 172.21.15.103:0/1153171361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c50d050 msgr2=0x55b33c511d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c50d050 0x55b33c511d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ee77fe640 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 0x55b33c518d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ee7fff640 1 -- 172.21.15.103:0/1153171361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 msgr2=0x55b33c518d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 0x55b33c518d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ee7fff640 1 -- 172.21.15.103:0/1153171361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b33c502b10 con 0x55b33c507680 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c507680 0x55b33c5117d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4ee81028c0 tx=0x7f4ed803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28463c651535d0ce server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ecdffb640 1 -- 172.21.15.103:0/1153171361 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8035040 con 0x55b33c507680 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ecdffb640 1 -- 172.21.15.103:0/1153171361 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed80318c0 con 0x55b33c507680 2024-08-19T01:42:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.398+0000 7f4ecdffb640 1 -- 172.21.15.103:0/1153171361 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80536e0 con 0x55b33c507680 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1153171361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c507680 msgr2=0x55b33c5117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c507680 0x55b33c5117d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4ee81028c0 tx=0x7f4ed803e000 comp rx=0 tx=0).stop 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1153171361 shutdown_connections 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c50d050 0x55b33c511d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 0x55b33c518d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/1153171361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c507680 0x55b33c5117d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1153171361 >> 172.21.15.103:0/1153171361 conn(0x55b33c502cf0 msgr2=0x55b33c509170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1153171361 shutdown_connections 2024-08-19T01:42:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 -- 172.21.15.103:0/1153171361 wait complete. 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.399+0000 7f4ef13e9740 1 -- start start 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 1 Processor -- start 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507680 0x55b33c515d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 0x55b33c5162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c5172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81aa090 con 0x55b33c50d050 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81a99c0 con 0x55b33c507fc0 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81a6c00 con 0x55b33c507680 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ef13e9740 10 client.0 _ll_register_callbacks cb 0x7f4ee0009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c5172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c5172d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38098/0 (socket says 172.21.15.103:38098) 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 -- 172.21.15.103:0/3141406585 learned_addr learned my addr 172.21.15.103:0/3141406585 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507680 0x55b33c515d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 -- 172.21.15.103:0/3141406585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507680 msgr2=0x55b33c515d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507680 0x55b33c515d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 -- 172.21.15.103:0/3141406585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 msgr2=0x55b33c5162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 0x55b33c5162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 -- 172.21.15.103:0/3141406585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33c5178e0 con 0x55b33c50d050 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507680 0x55b33c515d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4eecaea640 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c5172d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005a50 tx=0x7f4ee00129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d156d44071b9c35d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.400+0000 7f4ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ec77fe640 1 -- 172.21.15.103:0/3141406585 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0035030 con 0x55b33c50d050 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ec77fe640 1 -- 172.21.15.103:0/3141406585 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f4ee00093c0 con 0x55b33c50d050 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed80318c0 con 0x55b33c50d050 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ec77fe640 1 -- 172.21.15.103:0/3141406585 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4ee002a070 con 0x55b33c50d050 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 10 client.17001 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b33c518bf0 con 0x55b33c50d050 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ec77fe640 1 -- 172.21.15.103:0/3141406585 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed80318c0 con 0x55b33c50d050 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ec77fe640 1 -- 172.21.15.103:0/3141406585 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4ee000f9c0 con 0x55b33c50d050 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 10 client.17001 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 20 client.17001 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c58a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 4 client.17001 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.401+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b33c472450 con 0x55b33c51d0e0 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.402+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c58a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.402+0000 7f4ec77fe640 10 client.17001 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.402+0000 7f4ee7fff640 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c58a0e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d28a81abbcf807 server_cookie=1f58026298a9bb6c in_seq=0 out_seq=0 2024-08-19T01:42:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ec77fe640 1 -- 172.21.15.103:0/3141406585 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b33c472450 con 0x55b33c51d0e0 2024-08-19T01:42:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ec77fe640 10 client.17001 handle_command_reply: tid=0 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 client.17001 shutdown 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 msgr2=0x55b33c5172d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c5172d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005a50 tx=0x7f4ee00129d0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 shutdown_connections 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b33c51d0e0 0x55b33c58a0e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b33c50d050 0x55b33c5172d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b33c507fc0 0x55b33c5162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ec77fe640 0 client.17001 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 --2- 172.21.15.103:0/3141406585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b33c507680 0x55b33c515d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 >> 172.21.15.103:0/3141406585 conn(0x55b33c502cf0 msgr2=0x55b33c509170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 shutdown_connections 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 1 -- 172.21.15.103:0/3141406585 wait complete. 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.409+0000 7f4ef13e9740 20 client.17001 trim_cache size 0 max 16384 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:42:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:42:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4ed00a2cb0 msgr2=0x7f4ed00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4ed00a2cb0 0x7f4ed00a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802a930 tx=0x7f4ed80314a0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8102a60 msgr2=0x7f4ee8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8102a60 0x7f4ee8197f90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00e9f0 tx=0x7f4edc00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 shutdown_connections 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f4ed00a2cb0 0x7f4ed00a5170 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ee8103420 0x7f4ee819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee8102a60 0x7f4ee8197f90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 --2- 172.21.15.103:0/4281617071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ee8101870 0x7f4ee8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 >> 172.21.15.103:0/4281617071 conn(0x7f4ee80fd010 msgr2=0x7f4ee8105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 shutdown_connections 2024-08-19T01:42:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:02.410+0000 7f4eee54d640 1 -- 172.21.15.103:0/4281617071 wait complete. 2024-08-19T01:42:03.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:03 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:42:03.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:03 smithi103 ceph-mon[18436]: pgmap v527: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-08-19T01:42:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:03 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:42:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:03 smithi190 ceph-mon[22032]: pgmap v527: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-08-19T01:42:03.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:03 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:42:03.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:03 smithi146 ceph-mon[22098]: pgmap v527: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 25 op/s 2024-08-19T01:42:05.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:05 smithi103 ceph-mon[18436]: pgmap v528: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-08-19T01:42:05.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:05 smithi190 ceph-mon[22032]: pgmap v528: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-08-19T01:42:05.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:05 smithi146 ceph-mon[22098]: pgmap v528: 97 pgs: 97 active+clean; 9.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-08-19T01:42:07.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:07 smithi103 ceph-mon[18436]: pgmap v529: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:42:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:07 smithi190 ceph-mon[22032]: pgmap v529: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:42:07.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:07 smithi146 ceph-mon[22098]: pgmap v529: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-08-19T01:42:09.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:09 smithi103 ceph-mon[18436]: pgmap v530: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:42:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:09 smithi190 ceph-mon[22032]: pgmap v530: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:42:09.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:09 smithi146 ceph-mon[22098]: pgmap v530: 97 pgs: 97 active+clean; 9.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:42:11.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:11 smithi103 ceph-mon[18436]: pgmap v531: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:11.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:11 smithi190 ceph-mon[22032]: pgmap v531: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:11.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:11 smithi146 ceph-mon[22098]: pgmap v531: 97 pgs: 97 active+clean; 9.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:12.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:42:12.873 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:13.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:13 smithi103 ceph-mon[18436]: pgmap v532: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:42:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:13 smithi190 ceph-mon[22032]: pgmap v532: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:42:13.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:13 smithi146 ceph-mon[22098]: pgmap v532: 97 pgs: 97 active+clean; 9.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:42:15.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:15 smithi103 ceph-mon[18436]: pgmap v533: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:42:15.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:15 smithi190 ceph-mon[22032]: pgmap v533: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:42:15.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:15 smithi146 ceph-mon[22098]: pgmap v533: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 -- 172.21.15.103:0/656354046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc101960 msgr2=0x7f1efc101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 --2- 172.21.15.103:0/656354046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc101960 0x7f1efc101de0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009900 tx=0x7f1ef002a410 comp rx=0 tx=0).stop 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 -- 172.21.15.103:0/656354046 shutdown_connections 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 --2- 172.21.15.103:0/656354046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc102320 0x7f1efc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 --2- 172.21.15.103:0/656354046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc101960 0x7f1efc101de0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 --2- 172.21.15.103:0/656354046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1efc100770 0x7f1efc100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 -- 172.21.15.103:0/656354046 >> 172.21.15.103:0/656354046 conn(0x7f1efc0fbf10 msgr2=0x7f1efc0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 -- 172.21.15.103:0/656354046 shutdown_connections 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.505+0000 7f1f022ed640 1 -- 172.21.15.103:0/656354046 wait complete. 2024-08-19T01:42:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 Processor -- start 2024-08-19T01:42:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 -- start start 2024-08-19T01:42:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc100770 0x7f1efc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1efc101960 0x7f1efc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc102320 0x7f1efc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc10d3f0 con 0x7f1efc102320 2024-08-19T01:42:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc10d270 con 0x7f1efc100770 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f022ed640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc10d570 con 0x7f1efc101960 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.506+0000 7f1f0088a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc102320 0x7f1efc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc102320 0x7f1efc19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45666/0 (socket says 172.21.15.103:45666) 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 -- 172.21.15.103:0/1768171786 learned_addr learned my addr 172.21.15.103:0/1768171786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1efbfff640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc100770 0x7f1efc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1efc101960 0x7f1efc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 -- 172.21.15.103:0/1768171786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1efc101960 msgr2=0x7f1efc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1efc101960 0x7f1efc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 -- 172.21.15.103:0/1768171786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc100770 msgr2=0x7f1efc197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc100770 0x7f1efc197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 -- 172.21.15.103:0/1768171786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc19f5b0 con 0x7f1efc102320 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1efc101960 0x7f1efc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f0088a640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc102320 0x7f1efc19ef40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400cce0 tx=0x7f1ef400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1ef97fa640 1 -- 172.21.15.103:0/1768171786 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400f450 con 0x7f1efc102320 2024-08-19T01:42:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1ef97fa640 1 -- 172.21.15.103:0/1768171786 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef400faa0 con 0x7f1efc102320 2024-08-19T01:42:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1efc19f8a0 con 0x7f1efc102320 2024-08-19T01:42:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1ef97fa640 1 -- 172.21.15.103:0/1768171786 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4018b70 con 0x7f1efc102320 2024-08-19T01:42:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.507+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1efc19fda0 con 0x7f1efc102320 2024-08-19T01:42:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.510+0000 7f1ef97fa640 1 -- 172.21.15.103:0/1768171786 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1ef4018d10 con 0x7f1efc102320 2024-08-19T01:42:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.510+0000 7f1ef97fa640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1ee40a2c30 0x7f1ee40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.510+0000 7f1f050a8740 1 Processor -- start 2024-08-19T01:42:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1ef97fa640 1 -- 172.21.15.103:0/1768171786 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1ef40cf550 con 0x7f1efc102320 2024-08-19T01:42:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1f050a8740 1 -- start start 2024-08-19T01:42:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1efbfff640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1ee40a2c30 0x7f1ee40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53680 0x5590a5d53a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53fc0 0x5590a5d58a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5d5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1f050a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5590a5cb69d0 con 0x5590a5d53680 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1f050a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5590a5c5e820 con 0x5590a5d59050 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1f050a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5590a5cb2990 con 0x5590a5d53fc0 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.511+0000 7f1efbfff640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1ee40a2c30 0x7f1ee40a50f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1eec005db0 tx=0x7f1eec005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5d5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53fc0 0x5590a5d58a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5d5b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:42596/0 (socket says 172.21.15.103:42596) 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 -- 172.21.15.103:0/2546691507 learned_addr learned my addr 172.21.15.103:0/2546691507 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1efbfff640 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53680 0x5590a5d53a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 -- 172.21.15.103:0/2546691507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53fc0 msgr2=0x5590a5d58a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53fc0 0x5590a5d58a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 -- 172.21.15.103:0/2546691507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53680 msgr2=0x5590a5d53a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53680 0x5590a5d53a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 -- 172.21.15.103:0/2546691507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a59ca210 con 0x5590a5d59050 2024-08-19T01:42:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1f0088a640 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5d5b510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ef409bd70 tx=0x7f1ef409b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e92183fd8a485fb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.512+0000 7f1ee27fc640 1 -- 172.21.15.103:0/2546691507 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40a0190 con 0x5590a5d59050 2024-08-19T01:42:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.513+0000 7f1ee27fc640 1 -- 172.21.15.103:0/2546691507 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef40a4a50 con 0x5590a5d59050 2024-08-19T01:42:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.513+0000 7f1ee27fc640 1 -- 172.21.15.103:0/2546691507 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef409c040 con 0x5590a5d59050 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.513+0000 7f1f050a8740 1 -- 172.21.15.103:0/2546691507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 msgr2=0x5590a5d5b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.513+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5d5b510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ef409bd70 tx=0x7f1ef409b800 comp rx=0 tx=0).stop 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 -- 172.21.15.103:0/2546691507 shutdown_connections 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5d5b510 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53fc0 0x5590a5d58a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2546691507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53680 0x5590a5d53a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 -- 172.21.15.103:0/2546691507 >> 172.21.15.103:0/2546691507 conn(0x5590a5d4ecf0 msgr2=0x5590a5d4f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 -- 172.21.15.103:0/2546691507 shutdown_connections 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 -- 172.21.15.103:0/2546691507 wait complete. 2024-08-19T01:42:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 -- start start 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 Processor -- start 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.514+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53fc0 0x5590a5d62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d59050 0x5590a5d69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f050a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5590a5d603d0 con 0x5590a5d59050 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f050a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5590a5c72520 con 0x5590a5d53fc0 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f050a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5590a5d60550 con 0x5590a5d53680 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f050a8740 10 client.0 _ll_register_callbacks cb 0x5590a59c9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d59050 0x5590a5d69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d59050 0x5590a5d69160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45680/0 (socket says 172.21.15.103:45680) 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 -- 172.21.15.103:0/358370085 learned_addr learned my addr 172.21.15.103:0/358370085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1efbfff640 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53fc0 0x5590a5d62120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 -- 172.21.15.103:0/358370085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 msgr2=0x5590a5d61be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 -- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53fc0 msgr2=0x5590a5d62120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53fc0 0x5590a5d62120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 -- 172.21.15.103:0/358370085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a5a96620 con 0x5590a5d59050 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f0088a640 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d59050 0x5590a5d69160 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1ef409bae0 tx=0x7f1ef400af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6d7808c812c15e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1edb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1edb7fe640 1 -- 172.21.15.103:0/358370085 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4099e80 con 0x5590a5d59050 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1edb7fe640 1 -- 172.21.15.103:0/358370085 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1ef40a45c0 con 0x5590a5d59050 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.515+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef40a4a50 con 0x5590a5d59050 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1edb7fe640 1 -- 172.21.15.103:0/358370085 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1ef40a47c0 con 0x5590a5d59050 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 10 client.17013 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5590a5d6bf50 con 0x5590a5d59050 2024-08-19T01:42:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1edb7fe640 1 -- 172.21.15.103:0/358370085 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef40a4a50 con 0x5590a5d59050 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1edb7fe640 1 -- 172.21.15.103:0/358370085 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1ef409b230 con 0x5590a5d59050 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 10 client.17013 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 20 client.17013 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5d6d2a0 0x5590a5d6f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 4 client.17013 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a5cc4100 con 0x5590a5d6d2a0 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.516+0000 7f1efbfff640 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5d6d2a0 0x5590a5d6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.517+0000 7f1edb7fe640 10 client.17013 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.517+0000 7f1efbfff640 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5d6d2a0 0x5590a5d6f690 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcabb1016538c43f server_cookie=d0045c425c7c1896 in_seq=0 out_seq=0 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.517+0000 7f1edb7fe640 1 -- 172.21.15.103:0/358370085 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590a5cc4100 con 0x5590a5d6d2a0 2024-08-19T01:42:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.517+0000 7f1edb7fe640 10 client.17013 handle_command_reply: tid=0 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 client.17013 shutdown 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d59050 msgr2=0x5590a5d69160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d59050 0x5590a5d69160 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1ef409bae0 tx=0x7f1ef400af30 comp rx=0 tx=0).stop 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 shutdown_connections 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5d6d2a0 0x5590a5d6f690 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d59050 0x5590a5d69160 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1edb7fe640 0 client.17013 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53fc0 0x5590a5d62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 --2- 172.21.15.103:0/358370085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 >> 172.21.15.103:0/358370085 conn(0x5590a5d4ecf0 msgr2=0x5590a5d50500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 shutdown_connections 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 1 -- 172.21.15.103:0/358370085 wait complete. 2024-08-19T01:42:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.518+0000 7f1f050a8740 20 client.17013 trim_cache size 0 max 16384 2024-08-19T01:42:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 Processor -- start 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 -- start start 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d59050 0x5590a5d62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1a68a0 con 0x5590a5d53fc0 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1a83e0 con 0x5590a5d53680 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f050a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1a6f70 con 0x5590a5d59050 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d69120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45690/0 (socket says 172.21.15.103:45690) 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 -- 172.21.15.103:0/3787001166 learned_addr learned my addr 172.21.15.103:0/3787001166 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efbfff640 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 -- 172.21.15.103:0/3787001166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d59050 msgr2=0x5590a5d62120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1f0088a640 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d59050 0x5590a5d62120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d59050 0x5590a5d62120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 -- 172.21.15.103:0/3787001166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53680 msgr2=0x5590a5d61be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.537+0000 7f1efb7fe640 1 -- 172.21.15.103:0/3787001166 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a5d606b0 con 0x5590a5d53fc0 2024-08-19T01:42:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.538+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d69120 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1efc198ea0 tx=0x7f1ef0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae06544f8b0e859 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.538+0000 7f1ee1ffb640 1 -- 172.21.15.103:0/3787001166 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0035040 con 0x5590a5d53fc0 2024-08-19T01:42:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.538+0000 7f1ee1ffb640 1 -- 172.21.15.103:0/3787001166 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef0041070 con 0x5590a5d53fc0 2024-08-19T01:42:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.538+0000 7f1ee1ffb640 1 -- 172.21.15.103:0/3787001166 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0005500 con 0x5590a5d53fc0 2024-08-19T01:42:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 -- 172.21.15.103:0/3787001166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 msgr2=0x5590a5d69120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d69120 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1efc198ea0 tx=0x7f1ef0032f90 comp rx=0 tx=0).stop 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 -- 172.21.15.103:0/3787001166 shutdown_connections 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d59050 0x5590a5d62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d69120 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 --2- 172.21.15.103:0/3787001166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d53680 0x5590a5d61be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 -- 172.21.15.103:0/3787001166 >> 172.21.15.103:0/3787001166 conn(0x5590a5d4ecf0 msgr2=0x5590a5d551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 -- 172.21.15.103:0/3787001166 shutdown_connections 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 -- 172.21.15.103:0/3787001166 wait complete. 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.539+0000 7f1f050a8740 1 -- start start 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 1 Processor -- start 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d660d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d66610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5dd5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1a7d10 con 0x5590a5d53fc0 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1a7640 con 0x5590a5d59050 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1a4880 con 0x5590a5d53680 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f050a8740 10 client.0 _ll_register_callbacks cb 0x7f1eec014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d66610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d66610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45696/0 (socket says 172.21.15.103:45696) 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 -- 172.21.15.103:0/2407614026 learned_addr learned my addr 172.21.15.103:0/2407614026 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f0088a640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5dd5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efbfff640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 -- 172.21.15.103:0/2407614026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 msgr2=0x5590a5d660d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d660d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 -- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 msgr2=0x5590a5dd5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5dd5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efb7fe640 1 -- 172.21.15.103:0/2407614026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a5dd6360 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1f0088a640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5dd5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.540+0000 7f1efbfff640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1efb7fe640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d66610 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0000c00 tx=0x7f1ef0005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538bbac926fa3565 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1edb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1edb7fe640 1 -- 172.21.15.103:0/2407614026 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0035030 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef0041070 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1edb7fe640 1 -- 172.21.15.103:0/2407614026 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1ef00046c0 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1edb7fe640 1 -- 172.21.15.103:0/2407614026 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1ef0057070 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 10 client.17019 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5590a5dd75f0 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1edb7fe640 1 -- 172.21.15.103:0/2407614026 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef0004d70 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1edb7fe640 1 -- 172.21.15.103:0/2407614026 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1ef003da20 con 0x5590a5d53fc0 2024-08-19T01:42:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 10 client.17019 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 20 client.17019 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5dd7f20 0x5590a5dda340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 4 client.17019 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.541+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1ef0041070 con 0x5590a5dd7f20 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.542+0000 7f1efbfff640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5dd7f20 0x5590a5dda340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.542+0000 7f1edb7fe640 10 client.17019 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.542+0000 7f1efbfff640 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5dd7f20 0x5590a5dda340 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73661a5c9b02b2da server_cookie=25fc86f3b40e4726 in_seq=0 out_seq=0 2024-08-19T01:42:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.551+0000 7f1edb7fe640 1 -- 172.21.15.103:0/2407614026 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1ef0041070 con 0x5590a5dd7f20 2024-08-19T01:42:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.552+0000 7f1edb7fe640 10 client.17019 handle_command_reply: tid=0 2024-08-19T01:42:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.552+0000 7f1f050a8740 1 client.17019 shutdown 2024-08-19T01:42:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 msgr2=0x5590a5d66610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d66610 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0000c00 tx=0x7f1ef0005530 comp rx=0 tx=0).stop 2024-08-19T01:42:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 shutdown_connections 2024-08-19T01:42:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5590a5dd7f20 0x5590a5dda340 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1edb7fe640 0 client.17019 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5590a5d59050 0x5590a5dd5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5590a5d53fc0 0x5590a5d66610 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 --2- 172.21.15.103:0/2407614026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5590a5d53680 0x5590a5d660d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 >> 172.21.15.103:0/2407614026 conn(0x5590a5d4ecf0 msgr2=0x5590a5d61b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 shutdown_connections 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 1 -- 172.21.15.103:0/2407614026 wait complete. 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.553+0000 7f1f050a8740 20 client.17019 trim_cache size 0 max 16384 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:42:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:42:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1ee40a2c30 msgr2=0x7f1ee40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1ee40a2c30 0x7f1ee40a50f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1eec005db0 tx=0x7f1eec005d40 comp rx=0 tx=0).stop 2024-08-19T01:42:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc102320 msgr2=0x7f1efc19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc102320 0x7f1efc19ef40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400cce0 tx=0x7f1ef400a620 comp rx=0 tx=0).stop 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 shutdown_connections 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1ee40a2c30 0x7f1ee40a50f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1efc102320 0x7f1efc19ef40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1efc101960 0x7f1efc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 --2- 172.21.15.103:0/1768171786 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1efc100770 0x7f1efc197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.554+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 >> 172.21.15.103:0/1768171786 conn(0x7f1efc0fbf10 msgr2=0x7f1efc104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.555+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 shutdown_connections 2024-08-19T01:42:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:15.555+0000 7f1f022ed640 1 -- 172.21.15.103:0/1768171786 wait complete. 2024-08-19T01:42:17.257 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:17 smithi103 ceph-mon[18436]: pgmap v534: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:42:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:17 smithi190 ceph-mon[22032]: pgmap v534: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:42:17.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:17 smithi146 ceph-mon[22098]: pgmap v534: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 -- 172.21.15.103:0/1053001781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2301067a0 msgr2=0x7fc230106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 --2- 172.21.15.103:0/1053001781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2301067a0 0x7fc230106b80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc220009930 tx=0x7fc22002a420 comp rx=0 tx=0).stop 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 -- 172.21.15.103:0/1053001781 shutdown_connections 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 --2- 172.21.15.103:0/1053001781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc230101140 0x7fc23010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 --2- 172.21.15.103:0/1053001781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2301007a0 0x7fc230100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 --2- 172.21.15.103:0/1053001781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2301067a0 0x7fc230106b80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 -- 172.21.15.103:0/1053001781 >> 172.21.15.103:0/1053001781 conn(0x7fc2300fc4c0 msgr2=0x7fc2300fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:17.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 -- 172.21.15.103:0/1053001781 shutdown_connections 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.764+0000 7fc234b8b640 1 -- 172.21.15.103:0/1053001781 wait complete. 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 Processor -- start 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 -- start start 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2301007a0 0x7fc230102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc230101140 0x7fc2301028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2301067a0 0x7fc230102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc230110a10 con 0x7fc230101140 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc230110890 con 0x7fc2301067a0 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc234b8b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc230110b90 con 0x7fc2301007a0 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.765+0000 7fc22dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc230101140 0x7fc2301028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc230101140 0x7fc2301028b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45712/0 (socket says 172.21.15.103:45712) 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3558081042 learned_addr learned my addr 172.21.15.103:0/3558081042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:17.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2301007a0 0x7fc230102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2301067a0 0x7fc230102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3558081042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2301007a0 msgr2=0x7fc230102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2301007a0 0x7fc230102370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3558081042 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2301067a0 msgr2=0x7fc230102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2301067a0 0x7fc230102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3558081042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2301aaa30 con 0x7fc230101140 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2301067a0 0x7fc230102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2301007a0 0x7fc230102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc230101140 0x7fc2301028b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c920 tx=0x7fc22400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc21f7fe640 1 -- 172.21.15.103:0/3558081042 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224007c70 con 0x7fc230101140 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2301aac60 con 0x7fc230101140 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.766+0000 7fc21f7fe640 1 -- 172.21.15.103:0/3558081042 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc224007e10 con 0x7fc230101140 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.767+0000 7fc21f7fe640 1 -- 172.21.15.103:0/3558081042 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224018820 con 0x7fc230101140 2024-08-19T01:42:17.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.767+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2301ab1a0 con 0x7fc230101140 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 Processor -- start 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 -- start start 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc21f7fe640 1 -- 172.21.15.103:0/3558081042 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc2240189c0 con 0x7fc230101140 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426355f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb4263feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb426427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb4259a980 con 0x55eb426403f0 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55eb42532a70 con 0x55eb42635210 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc236f89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eb42596940 con 0x55eb42635b30 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc22ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc22ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426355f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:42666/0 (socket says 172.21.15.103:42666) 2024-08-19T01:42:17.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc22dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb426427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc22ed9d640 1 -- 172.21.15.103:0/1398485143 learned_addr learned my addr 172.21.15.103:0/1398485143 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.770+0000 7fc22e59c640 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb4263feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc22ed9d640 1 -- 172.21.15.103:0/1398485143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 msgr2=0x55eb4263feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb4263feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc22ed9d640 1 -- 172.21.15.103:0/1398485143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 msgr2=0x55eb426427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb426427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc22ed9d640 1 -- 172.21.15.103:0/1398485143 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb425956e0 con 0x55eb42635210 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc21f7fe640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2180a2b80 0x7fc2180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426355f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc228009fd0 tx=0x7fc22800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24e9ca1cce01aae6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc21f7fe640 1 -- 172.21.15.103:0/3558081042 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc2240cf1e0 con 0x7fc230101140 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2180a2b80 0x7fc2180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc21cff9640 1 -- 172.21.15.103:0/1398485143 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228015070 con 0x55eb42635210 2024-08-19T01:42:17.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc21cff9640 1 -- 172.21.15.103:0/1398485143 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc228011bb0 con 0x55eb42635210 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.771+0000 7fc21cff9640 1 -- 172.21.15.103:0/1398485143 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228010b40 con 0x55eb42635210 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 -- 172.21.15.103:0/1398485143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 msgr2=0x55eb426355f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426355f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc228009fd0 tx=0x7fc22800f7f0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2180a2b80 0x7fc2180a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc220009900 tx=0x7fc220005c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 -- 172.21.15.103:0/1398485143 shutdown_connections 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb426427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb4263feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 --2- 172.21.15.103:0/1398485143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426355f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 -- 172.21.15.103:0/1398485143 >> 172.21.15.103:0/1398485143 conn(0x55eb426317a0 msgr2=0x55eb42631b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.772+0000 7fc236f89740 1 -- 172.21.15.103:0/1398485143 shutdown_connections 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 -- 172.21.15.103:0/1398485143 wait complete. 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 -- start start 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 Processor -- start 2024-08-19T01:42:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635210 0x55eb4263d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635b30 0x55eb4263d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb426403f0 0x55eb4263dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb42642fc0 con 0x55eb42635b30 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55eb42555aa0 con 0x55eb426403f0 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55eb42643140 con 0x55eb42635210 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc236f89740 10 client.0 _ll_register_callbacks cb 0x55eb42544e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.773+0000 7fc22dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635b30 0x55eb4263d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635b30 0x55eb4263d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45738/0 (socket says 172.21.15.103:45738) 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3709568160 learned_addr learned my addr 172.21.15.103:0/3709568160 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635210 0x55eb4263d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb426403f0 0x55eb4263dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3709568160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635210 msgr2=0x55eb4263d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635210 0x55eb4263d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb426403f0 msgr2=0x55eb4263dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb426403f0 0x55eb4263dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 -- 172.21.15.103:0/3709568160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb42539650 con 0x55eb42635b30 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb426403f0 0x55eb4263dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635b30 0x55eb4263d810 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc224007590 tx=0x7fc22400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb74a5e180b4085 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc215ffb640 1 -- 172.21.15.103:0/3709568160 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22409b5b0 con 0x55eb42635b30 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc215ffb640 1 -- 172.21.15.103:0/3709568160 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc2240a4250 con 0x55eb42635b30 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.774+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc228011bb0 con 0x55eb42635b30 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc215ffb640 1 -- 172.21.15.103:0/3709568160 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc224018cb0 con 0x55eb42635b30 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 10 client.17031 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:17.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55eb4230c3c0 con 0x55eb42635b30 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc215ffb640 1 -- 172.21.15.103:0/3709568160 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc228011bb0 con 0x55eb42635b30 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc215ffb640 1 -- 172.21.15.103:0/3709568160 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc2240977c0 con 0x55eb42635b30 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 10 client.17031 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 20 client.17031 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb42652350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 4 client.17031 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb425a7ef0 con 0x55eb4264ff60 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.775+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb42652350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.776+0000 7fc215ffb640 10 client.17031 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.776+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb42652350 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19535bfcfa2b69c3 server_cookie=633612c9d94038c1 in_seq=0 out_seq=0 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.776+0000 7fc215ffb640 1 -- 172.21.15.103:0/3709568160 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb425a7ef0 con 0x55eb4264ff60 2024-08-19T01:42:17.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.776+0000 7fc215ffb640 10 client.17031 handle_command_reply: tid=0 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.776+0000 7fc236f89740 1 client.17031 shutdown 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635b30 msgr2=0x55eb4263d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635b30 0x55eb4263d810 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc224007590 tx=0x7fc22400a560 comp rx=0 tx=0).stop 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 shutdown_connections 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb42652350 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb426403f0 0x55eb4263dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc215ffb640 0 client.17031 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635b30 0x55eb4263d810 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 --2- 172.21.15.103:0/3709568160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635210 0x55eb4263d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 >> 172.21.15.103:0/3709568160 conn(0x55eb426317a0 msgr2=0x55eb42637480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 shutdown_connections 2024-08-19T01:42:17.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 1 -- 172.21.15.103:0/3709568160 wait complete. 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.777+0000 7fc236f89740 20 client.17031 trim_cache size 0 max 16384 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 Processor -- start 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 -- start start 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635210 0x55eb42631b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635b30 0x55eb42657bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb426403f0 0x55eb42658110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b3740 con 0x55eb42635210 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b5280 con 0x55eb42635b30 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc236f89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b3e10 con 0x55eb426403f0 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc22e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635210 0x55eb42631b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc22e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635210 0x55eb42631b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45750/0 (socket says 172.21.15.103:45750) 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc22e59c640 1 -- 172.21.15.103:0/3691127719 learned_addr learned my addr 172.21.15.103:0/3691127719 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635b30 0x55eb42657bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.812+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb426403f0 0x55eb42658110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc22e59c640 1 -- 172.21.15.103:0/3691127719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb426403f0 msgr2=0x55eb42658110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb426403f0 0x55eb42658110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc22e59c640 1 -- 172.21.15.103:0/3691127719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635b30 msgr2=0x55eb42657bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635b30 0x55eb42657bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc22e59c640 1 -- 172.21.15.103:0/3691127719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb426315d0 con 0x55eb42635210 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb426403f0 0x55eb42658110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc22e59c640 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635210 0x55eb42631b80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc220000c00 tx=0x7fc22003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e601eda97a72c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc217fff640 1 -- 172.21.15.103:0/3691127719 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220043070 con 0x55eb42635210 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc217fff640 1 -- 172.21.15.103:0/3691127719 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc220003d00 con 0x55eb42635210 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.813+0000 7fc217fff640 1 -- 172.21.15.103:0/3691127719 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220035040 con 0x55eb42635210 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 -- 172.21.15.103:0/3691127719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635210 msgr2=0x55eb42631b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635210 0x55eb42631b80 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc220000c00 tx=0x7fc22003f000 comp rx=0 tx=0).stop 2024-08-19T01:42:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 -- 172.21.15.103:0/3691127719 shutdown_connections 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb426403f0 0x55eb42658110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635b30 0x55eb42657bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 --2- 172.21.15.103:0/3691127719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb42635210 0x55eb42631b80 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 -- 172.21.15.103:0/3691127719 >> 172.21.15.103:0/3691127719 conn(0x55eb426317a0 msgr2=0x55eb4263f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 -- 172.21.15.103:0/3691127719 shutdown_connections 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.814+0000 7fc236f89740 1 -- 172.21.15.103:0/3691127719 wait complete. 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 -- start start 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 Processor -- start 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb42648bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb42643b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b4bb0 con 0x55eb426403f0 2024-08-19T01:42:17.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b44e0 con 0x55eb42635210 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b1720 con 0x55eb42635b30 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc236f89740 10 client.0 _ll_register_callbacks cb 0x7fc22400c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc22ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb42643b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc22ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb42643b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45764/0 (socket says 172.21.15.103:45764) 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc22ed9d640 1 -- 172.21.15.103:0/142495262 learned_addr learned my addr 172.21.15.103:0/142495262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc22e59c640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.815+0000 7fc22dd9b640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb42648bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc22ed9d640 1 -- 172.21.15.103:0/142495262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 msgr2=0x55eb42648bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb42648bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc22ed9d640 1 -- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 msgr2=0x55eb426486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc22ed9d640 1 -- 172.21.15.103:0/142495262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb42644330 con 0x55eb426403f0 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc22e59c640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:17.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc22ed9d640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb42643b00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc228006010 tx=0x7fc228009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d560e788c48173f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc215ffb640 1 -- 172.21.15.103:0/142495262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228015040 con 0x55eb426403f0 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc215ffb640 1 -- 172.21.15.103:0/142495262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc228004c70 con 0x55eb426403f0 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc220003d00 con 0x55eb426403f0 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc215ffb640 1 -- 172.21.15.103:0/142495262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc228003970 con 0x55eb426403f0 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc236f89740 10 client.17037 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:42:17.805 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55eb426ba720 con 0x55eb426403f0 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc215ffb640 1 -- 172.21.15.103:0/142495262 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc220003d00 con 0x55eb426403f0 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.816+0000 7fc215ffb640 1 -- 172.21.15.103:0/142495262 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc228005200 con 0x55eb426403f0 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc236f89740 10 client.17037 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc236f89740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc236f89740 20 client.17037 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:17.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc236f89740 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb426bcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc236f89740 4 client.17037 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb425a2400 con 0x55eb4264ff60 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc22e59c640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb426bcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc22e59c640 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb426bcb20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1c843e55eaf99a server_cookie=e841ea465df34a9b in_seq=0 out_seq=0 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.817+0000 7fc215ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc215ffb640 1 -- 172.21.15.103:0/142495262 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eb425a2400 con 0x55eb4264ff60 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc215ffb640 10 client.17037 handle_command_reply: tid=0 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 client.17037 shutdown 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 msgr2=0x55eb42643b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb42643b00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc228006010 tx=0x7fc228009d10 comp rx=0 tx=0).stop 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 shutdown_connections 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55eb4264ff60 0x55eb426bcb20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb426403f0 0x55eb42643b00 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55eb42635b30 0x55eb42648bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc215ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 --2- 172.21.15.103:0/142495262 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55eb42635210 0x55eb426486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 >> 172.21.15.103:0/142495262 conn(0x55eb426317a0 msgr2=0x55eb426300e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 shutdown_connections 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 1 -- 172.21.15.103:0/142495262 wait complete. 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.818+0000 7fc236f89740 20 client.17037 trim_cache size 0 max 16384 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2180a2b80 msgr2=0x7fc2180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2180a2b80 0x7fc2180a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc220009900 tx=0x7fc220005c00 comp rx=0 tx=0).stop 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc230101140 msgr2=0x7fc2301028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc230101140 0x7fc2301028b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c920 tx=0x7fc22400cde0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 shutdown_connections 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc2180a2b80 0x7fc2180a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2301067a0 0x7fc230102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc230101140 0x7fc2301028b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 --2- 172.21.15.103:0/3558081042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2301007a0 0x7fc230102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 >> 172.21.15.103:0/3558081042 conn(0x7fc2300fc4c0 msgr2=0x7fc230104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 shutdown_connections 2024-08-19T01:42:17.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:16.819+0000 7fc234b8b640 1 -- 172.21.15.103:0/3558081042 wait complete. 2024-08-19T01:42:19.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:19 smithi103 ceph-mon[18436]: pgmap v535: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:42:19.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:19 smithi190 ceph-mon[22032]: pgmap v535: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:42:19.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:19 smithi146 ceph-mon[22098]: pgmap v535: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:42:21.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:21 smithi103 ceph-mon[18436]: pgmap v536: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:21 smithi190 ceph-mon[22032]: pgmap v536: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:21.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:21 smithi146 ceph-mon[22098]: pgmap v536: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:22.421 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:42:22.422 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:42:22.709 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:23.136 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:23 smithi103 ceph-mon[18436]: pgmap v537: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 -- 172.21.15.103:0/4260116168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901018e0 msgr2=0x7f4990101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 --2- 172.21.15.103:0/4260116168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901018e0 0x7f4990101d40 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4978009930 tx=0x7f497802a420 comp rx=0 tx=0).stop 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 -- 172.21.15.103:0/4260116168 shutdown_connections 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 --2- 172.21.15.103:0/4260116168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4990102280 0x7f499010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 --2- 172.21.15.103:0/4260116168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901018e0 0x7f4990101d40 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 --2- 172.21.15.103:0/4260116168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49901078e0 0x7f4990107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 -- 172.21.15.103:0/4260116168 >> 172.21.15.103:0/4260116168 conn(0x7f49900fd600 msgr2=0x7f49900ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 -- 172.21.15.103:0/4260116168 shutdown_connections 2024-08-19T01:42:23.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.225+0000 7f4996a45640 1 -- 172.21.15.103:0/4260116168 wait complete. 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.226+0000 7f4996a45640 1 Processor -- start 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.226+0000 7f4996a45640 1 -- start start 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.226+0000 7f4996a45640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49901018e0 0x7f4990198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.226+0000 7f4996a45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4990102280 0x7f49901985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.226+0000 7f4996a45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901078e0 0x7f499019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.226+0000 7f4996a45640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4990111b90 con 0x7f49901078e0 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.226+0000 7f4996a45640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4990111a10 con 0x7f49901018e0 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4996a45640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4990111d10 con 0x7f4990102280 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901078e0 0x7f499019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901078e0 0x7f499019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52624/0 (socket says 172.21.15.103:52624) 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 -- 172.21.15.103:0/3099021494 learned_addr learned my addr 172.21.15.103:0/3099021494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:23.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4987fff640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4990102280 0x7f49901985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f498ffff640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49901018e0 0x7f4990198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 -- 172.21.15.103:0/3099021494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4990102280 msgr2=0x7f49901985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4990102280 0x7f49901985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 -- 172.21.15.103:0/3099021494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49901018e0 msgr2=0x7f4990198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49901018e0 0x7f4990198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 -- 172.21.15.103:0/3099021494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499019d040 con 0x7f49901078e0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4987fff640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4990102280 0x7f49901985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f4994fe2640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901078e0 0x7f499019c960 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f498800cce0 tx=0x7f498800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f498dffb640 1 -- 172.21.15.103:0/3099021494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498800f450 con 0x7f49901078e0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.227+0000 7f498dffb640 1 -- 172.21.15.103:0/3099021494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f498800faa0 con 0x7f49901078e0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.228+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499019d330 con 0x7f49901078e0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.228+0000 7f498dffb640 1 -- 172.21.15.103:0/3099021494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988018b70 con 0x7f49901078e0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.228+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49901a4c10 con 0x7f49901078e0 2024-08-19T01:42:23.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.229+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4990103020 con 0x7f49901078e0 2024-08-19T01:42:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.230+0000 7f498dffb640 1 -- 172.21.15.103:0/3099021494 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4988018d10 con 0x7f49901078e0 2024-08-19T01:42:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.231+0000 7f498dffb640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f497c0a2b80 0x7f497c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.232+0000 7f498ffff640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f497c0a2b80 0x7f497c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.232+0000 7f498dffb640 1 -- 172.21.15.103:0/3099021494 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f49880cf630 con 0x7f49901078e0 2024-08-19T01:42:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.232+0000 7f498ffff640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f497c0a2b80 0x7f497c0a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4980009ae0 tx=0x7f4980005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:23.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.235+0000 7f498dffb640 1 -- 172.21.15.103:0/3099021494 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4988014030 con 0x7f49901078e0 2024-08-19T01:42:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:23 smithi190 ceph-mon[22032]: pgmap v537: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:42:23.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:23 smithi146 ceph-mon[22098]: pgmap v537: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:42:23.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.472+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4990101d40 con 0x7f49901078e0 2024-08-19T01:42:23.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.473+0000 7f498dffb640 1 -- 172.21.15.103:0/3099021494 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11989 (secure 0 0 0) 0x7f4990101d40 con 0x7f49901078e0 2024-08-19T01:42:23.475 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:42:23.476 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":36,"btime":"2024-08-19T01:42:02:004677+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:42:01.572010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:42:23.477 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 36 2024-08-19T01:42:23.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f497c0a2b80 msgr2=0x7f497c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:23.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f497c0a2b80 0x7f497c0a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4980009ae0 tx=0x7f4980005f90 comp rx=0 tx=0).stop 2024-08-19T01:42:23.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901078e0 msgr2=0x7f499019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:23.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901078e0 0x7f499019c960 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f498800cce0 tx=0x7f498800a620 comp rx=0 tx=0).stop 2024-08-19T01:42:23.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 shutdown_connections 2024-08-19T01:42:23.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f497c0a2b80 0x7f497c0a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49901078e0 0x7f499019c960 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4990102280 0x7f49901985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 --2- 172.21.15.103:0/3099021494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49901018e0 0x7f4990198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:23.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 >> 172.21.15.103:0/3099021494 conn(0x7f49900fd600 msgr2=0x7f49900fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:23.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 shutdown_connections 2024-08-19T01:42:23.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:23.476+0000 7f4996a45640 1 -- 172.21.15.103:0/3099021494 wait complete. 2024-08-19T01:42:24.196 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:24 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3099021494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:42:24.225 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:42:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:24 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3099021494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:42:24.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:24 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3099021494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:42:24.513 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:25.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 -- 172.21.15.103:0/178433354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c101090 msgr2=0x7fcc2c10dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 --2- 172.21.15.103:0/178433354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c101090 0x7fcc2c10dcc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009900 tx=0x7fcc2002a410 comp rx=0 tx=0).stop 2024-08-19T01:42:25.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 -- 172.21.15.103:0/178433354 shutdown_connections 2024-08-19T01:42:25.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 --2- 172.21.15.103:0/178433354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c10e200 0x7fcc2c110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 --2- 172.21.15.103:0/178433354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c101090 0x7fcc2c10dcc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 --2- 172.21.15.103:0/178433354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc2c1006e0 0x7fcc2c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 -- 172.21.15.103:0/178433354 >> 172.21.15.103:0/178433354 conn(0x7fcc2c0fc500 msgr2=0x7fcc2c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 -- 172.21.15.103:0/178433354 shutdown_connections 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.097+0000 7fcc3132d640 1 -- 172.21.15.103:0/178433354 wait complete. 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 Processor -- start 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 -- start start 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc2c1006e0 0x7fcc2c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c101090 0x7fcc2c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c10e200 0x7fcc2c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c113e20 con 0x7fcc2c10e200 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c113ca0 con 0x7fcc2c101090 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.098+0000 7fcc3132d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c113fa0 con 0x7fcc2c1006e0 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c10e200 0x7fcc2c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c10e200 0x7fcc2c19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52640/0 (socket says 172.21.15.103:52640) 2024-08-19T01:42:25.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 -- 172.21.15.103:0/394933202 learned_addr learned my addr 172.21.15.103:0/394933202 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2affd640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc2c1006e0 0x7fcc2c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c101090 0x7fcc2c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 -- 172.21.15.103:0/394933202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc2c1006e0 msgr2=0x7fcc2c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc2c1006e0 0x7fcc2c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 -- 172.21.15.103:0/394933202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c101090 msgr2=0x7fcc2c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c101090 0x7fcc2c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 -- 172.21.15.103:0/394933202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c19f3d0 con 0x7fcc2c10e200 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc2b7fe640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c10e200 0x7fcc2c19ecf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009fd0 tx=0x7fcc2400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc13fff640 1 -- 172.21.15.103:0/394933202 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24017070 con 0x7fcc2c10e200 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c19f6c0 con 0x7fcc2c10e200 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc13fff640 1 -- 172.21.15.103:0/394933202 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc24003e20 con 0x7fcc2c10e200 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.099+0000 7fcc13fff640 1 -- 172.21.15.103:0/394933202 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24010460 con 0x7fcc2c10e200 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.100+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1a6fa0 con 0x7fcc2c10e200 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.102+0000 7fcc3372b740 1 Processor -- start 2024-08-19T01:42:25.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.102+0000 7fcc13fff640 1 -- 172.21.15.103:0/394933202 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcc24003970 con 0x7fcc2c10e200 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.102+0000 7fcc3372b740 1 -- start start 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eb910 0x5628241f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc3372b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562824150a40 con 0x5628241f61d0 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc3372b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5628240e89d0 con 0x5628241eb910 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc3372b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56282414ca00 con 0x5628241eaff0 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52644/0 (socket says 172.21.15.103:52644) 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/172913845 learned_addr learned my addr 172.21.15.103:0/172913845 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2affd640 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eb910 0x5628241f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc13fff640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc140a2b80 0x7fcc140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/172913845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 msgr2=0x5628241eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/172913845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eb910 msgr2=0x5628241f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eb910 0x5628241f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/172913845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628240ef650 con 0x5628241f61d0 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.103+0000 7fcc2b7fe640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc140a2b80 0x7fcc140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.104+0000 7fcc13fff640 1 -- 172.21.15.103:0/394933202 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fcc240ced60 con 0x7fcc2c10e200 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.104+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241f85c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fcc2002a930 tx=0x7fcc20005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f43138efc7c12a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.104+0000 7fcc117fa640 1 -- 172.21.15.103:0/172913845 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20035070 con 0x5628241f61d0 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.104+0000 7fcc117fa640 1 -- 172.21.15.103:0/172913845 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc20045070 con 0x5628241f61d0 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.104+0000 7fcc117fa640 1 -- 172.21.15.103:0/172913845 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20003c10 con 0x5628241f61d0 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 -- 172.21.15.103:0/172913845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 msgr2=0x5628241f85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc2b7fe640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc140a2b80 0x7fcc140a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc2409b3a0 tx=0x7fcc24066320 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241f85c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fcc2002a930 tx=0x7fcc20005b50 comp rx=0 tx=0).stop 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 -- 172.21.15.103:0/172913845 shutdown_connections 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241f85c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eb910 0x5628241f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 --2- 172.21.15.103:0/172913845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 -- 172.21.15.103:0/172913845 >> 172.21.15.103:0/172913845 conn(0x5628241e7580 msgr2=0x5628241e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 -- 172.21.15.103:0/172913845 shutdown_connections 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 -- 172.21.15.103:0/172913845 wait complete. 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 -- start start 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.105+0000 7fcc3372b740 1 Processor -- start 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241eb910 0x5628241f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241f61d0 0x5628241f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc3372b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5628241f8da0 con 0x5628241eb910 2024-08-19T01:42:25.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc3372b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56282410bb20 con 0x5628241f61d0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc3372b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5628241f8f20 con 0x5628241eaff0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc3372b740 10 client.0 _ll_register_callbacks cb 0x562824109bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241eb910 0x5628241f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241eb910 0x5628241f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52658/0 (socket says 172.21.15.103:52658) 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 -- 172.21.15.103:0/517485711 learned_addr learned my addr 172.21.15.103:0/517485711 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241f61d0 0x5628241f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 -- 172.21.15.103:0/517485711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 msgr2=0x5628241f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241f30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 -- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241f61d0 msgr2=0x5628241f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241f61d0 0x5628241f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 -- 172.21.15.103:0/517485711 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562823ec23c0 con 0x5628241eb910 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc2affd640 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241eb910 0x5628241f3600 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b950 tx=0x7fcc1c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e29985dc125a13 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.106+0000 7fcc0eff5640 1 -- 172.21.15.103:0/517485711 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c01b440 con 0x5628241eb910 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc0eff5640 1 -- 172.21.15.103:0/517485711 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fcc1c01b5e0 con 0x5628241eb910 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc20045070 con 0x5628241eb910 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc0eff5640 1 -- 172.21.15.103:0/517485711 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c022070 con 0x5628241eb910 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 10 client.17055 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562824204a60 con 0x5628241eb910 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc0eff5640 1 -- 172.21.15.103:0/517485711 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc20045070 con 0x5628241eb910 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc0eff5640 1 -- 172.21.15.103:0/517485711 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fcc1c019020 con 0x5628241eb910 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 10 client.17055 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 10 client.17055 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 20 client.17055 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5628242060b0 0x5628242084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 4 client.17055 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56282415dfb0 con 0x5628242060b0 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.107+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5628242060b0 0x5628242084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.108+0000 7fcc0eff5640 10 client.17055 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.108+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5628242060b0 0x5628242084a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71ec8d0f7a917215 server_cookie=1e181e298a6896aa in_seq=0 out_seq=0 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc0eff5640 1 -- 172.21.15.103:0/517485711 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56282415dfb0 con 0x5628242060b0 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc0eff5640 10 client.17055 handle_command_reply: tid=0 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 client.17055 shutdown 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241eb910 msgr2=0x5628241f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241eb910 0x5628241f3600 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b950 tx=0x7fcc1c00be10 comp rx=0 tx=0).stop 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 shutdown_connections 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x5628242060b0 0x5628242084a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241f61d0 0x5628241f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241eb910 0x5628241f3600 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 --2- 172.21.15.103:0/517485711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eaff0 0x5628241f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 >> 172.21.15.103:0/517485711 conn(0x5628241e7580 msgr2=0x5628241ed260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc0eff5640 0 client.17055 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:42:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 shutdown_connections 2024-08-19T01:42:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 1 -- 172.21.15.103:0/517485711 wait complete. 2024-08-19T01:42:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.109+0000 7fcc3372b740 20 client.17055 trim_cache size 0 max 16384 2024-08-19T01:42:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 Processor -- start 2024-08-19T01:42:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 -- start start 2024-08-19T01:42:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x562824208690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x56282420dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1af1d0 con 0x5628241f61d0 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b0d10 con 0x5628241eaff0 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.124+0000 7fcc3372b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1af8a0 con 0x5628241eb910 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x56282420dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x56282420dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52668/0 (socket says 172.21.15.103:52668) 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/1842216523 learned_addr learned my addr 172.21.15.103:0/1842216523 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x562824208690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2affd640 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/1842216523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 msgr2=0x5628241f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/1842216523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 msgr2=0x562824208690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x562824208690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/1842216523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628241e8710 con 0x5628241f61d0 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2affd640 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x56282420dcb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009850 tx=0x7fcc20005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241a62bc7027a852 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc10ff9640 1 -- 172.21.15.103:0/1842216523 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20035070 con 0x5628241f61d0 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc10ff9640 1 -- 172.21.15.103:0/1842216523 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc20045030 con 0x5628241f61d0 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.125+0000 7fcc10ff9640 1 -- 172.21.15.103:0/1842216523 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20031620 con 0x5628241f61d0 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 -- 172.21.15.103:0/1842216523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 msgr2=0x56282420dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x56282420dcb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009850 tx=0x7fcc20005b50 comp rx=0 tx=0).stop 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 -- 172.21.15.103:0/1842216523 shutdown_connections 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x56282420dcb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 --2- 172.21.15.103:0/1842216523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x562824208690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 -- 172.21.15.103:0/1842216523 >> 172.21.15.103:0/1842216523 conn(0x5628241e7580 msgr2=0x562824207f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 -- 172.21.15.103:0/1842216523 shutdown_connections 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 -- 172.21.15.103:0/1842216523 wait complete. 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.126+0000 7fcc3372b740 1 -- start start 2024-08-19T01:42:25.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 1 Processor -- start 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x5628241fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b0640 con 0x5628241f61d0 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1aff70 con 0x5628241eaff0 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1ad1b0 con 0x5628241eb910 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc3372b740 10 client.0 _ll_register_callbacks cb 0x7fcc1c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241ef6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52682/0 (socket says 172.21.15.103:52682) 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2a7fc640 1 -- 172.21.15.103:0/4087129967 learned_addr learned my addr 172.21.15.103:0/4087129967 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2affd640 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x5628241fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2affd640 1 -- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 msgr2=0x5628241fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2affd640 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x5628241fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2affd640 1 -- 172.21.15.103:0/4087129967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 msgr2=0x5628241ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2affd640 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241ef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.127+0000 7fcc2affd640 1 -- 172.21.15.103:0/4087129967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628241efc30 con 0x5628241eb910 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc2affd640 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241fe6b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c011a20 tx=0x7fcc1c011ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6441be399f5a1834 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc0eff5640 1 -- 172.21.15.103:0/4087129967 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c011440 con 0x5628241eb910 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc20045030 con 0x5628241eb910 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc0eff5640 1 -- 172.21.15.103:0/4087129967 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fcc1c0115e0 con 0x5628241eb910 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc0eff5640 1 -- 172.21.15.103:0/4087129967 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c027480 con 0x5628241eb910 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc3372b740 10 client.26320 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562824270330 con 0x5628241eb910 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.128+0000 7fcc0eff5640 1 -- 172.21.15.103:0/4087129967 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc20045030 con 0x5628241eb910 2024-08-19T01:42:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc0eff5640 1 -- 172.21.15.103:0/4087129967 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fcc1c030060 con 0x5628241eb910 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc3372b740 10 client.26320 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc3372b740 10 client.26320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc3372b740 20 client.26320 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc3372b740 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x562824270b00 0x562824272f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc3372b740 4 client.26320 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628241584c0 con 0x562824270b00 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x562824270b00 0x562824272f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc2a7fc640 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x562824270b00 0x562824272f20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=466f73d9e4811837 server_cookie=cdd4e5be711e4830 in_seq=0 out_seq=0 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.129+0000 7fcc0eff5640 10 client.26320 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc0eff5640 1 -- 172.21.15.103:0/4087129967 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628241584c0 con 0x562824270b00 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc0eff5640 10 client.26320 handle_command_reply: tid=0 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 client.26320 shutdown 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 msgr2=0x5628241fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241fe6b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c011a20 tx=0x7fcc1c011ea0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 shutdown_connections 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x562824270b00 0x562824272f20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628241f61d0 0x5628241ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628241eb910 0x5628241fe6b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc0eff5640 0 client.26320 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 --2- 172.21.15.103:0/4087129967 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5628241eaff0 0x5628241fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 >> 172.21.15.103:0/4087129967 conn(0x5628241e7580 msgr2=0x5628241e5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 shutdown_connections 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 1 -- 172.21.15.103:0/4087129967 wait complete. 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.130+0000 7fcc3372b740 20 client.26320 trim_cache size 0 max 16384 2024-08-19T01:42:25.142 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:42:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc140a2b80 msgr2=0x7fcc140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc140a2b80 0x7fcc140a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcc2409b3a0 tx=0x7fcc24066320 comp rx=0 tx=0).stop 2024-08-19T01:42:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c10e200 msgr2=0x7fcc2c19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c10e200 0x7fcc2c19ecf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009fd0 tx=0x7fcc2400b260 comp rx=0 tx=0).stop 2024-08-19T01:42:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 shutdown_connections 2024-08-19T01:42:25.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fcc140a2b80 0x7fcc140a5040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc2c10e200 0x7fcc2c19ecf0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc2c101090 0x7fcc2c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 --2- 172.21.15.103:0/394933202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc2c1006e0 0x7fcc2c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:25.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 >> 172.21.15.103:0/394933202 conn(0x7fcc2c0fc500 msgr2=0x7fcc2c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:25.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 shutdown_connections 2024-08-19T01:42:25.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:25.131+0000 7fcc3132d640 1 -- 172.21.15.103:0/394933202 wait complete. 2024-08-19T01:42:25.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:25 smithi103 ceph-mon[18436]: pgmap v538: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-08-19T01:42:25.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:25 smithi190 ceph-mon[22032]: pgmap v538: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-08-19T01:42:25.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:25 smithi146 ceph-mon[22098]: pgmap v538: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-08-19T01:42:25.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:42:26.139 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:26.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 -- 172.21.15.103:0/2383245100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 msgr2=0x7fe32010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 --2- 172.21.15.103:0/2383245100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 0x7fe32010e970 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe318009cd0 tx=0x7fe31802a8e0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 -- 172.21.15.103:0/2383245100 shutdown_connections 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 --2- 172.21.15.103:0/2383245100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 0x7fe32010e970 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 --2- 172.21.15.103:0/2383245100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320103410 0x7fe3201078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 --2- 172.21.15.103:0/2383245100 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe320102a60 0x7fe320102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 -- 172.21.15.103:0/2383245100 >> 172.21.15.103:0/2383245100 conn(0x7fe3200fc500 msgr2=0x7fe3200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 -- 172.21.15.103:0/2383245100 shutdown_connections 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.736+0000 7fe327272640 1 -- 172.21.15.103:0/2383245100 wait complete. 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.737+0000 7fe327272640 1 Processor -- start 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.737+0000 7fe327272640 1 -- start start 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.737+0000 7fe327272640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320102a60 0x7fe32019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.737+0000 7fe327272640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe320103410 0x7fe32019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.737+0000 7fe327272640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 0x7fe32019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.737+0000 7fe327272640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe320111c80 con 0x7fe320107fd0 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe327272640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe320111b00 con 0x7fe320103410 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe327272640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe320111e00 con 0x7fe320102a60 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32580f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 0x7fe32019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320102a60 0x7fe32019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320102a60 0x7fe32019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:52488/0 (socket says 172.21.15.103:52488) 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 -- 172.21.15.103:0/1310805993 learned_addr learned my addr 172.21.15.103:0/1310805993 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32480d640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe320103410 0x7fe32019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 -- 172.21.15.103:0/1310805993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe320103410 msgr2=0x7fe32019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe320103410 0x7fe32019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 -- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 msgr2=0x7fe32019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 0x7fe32019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 -- 172.21.15.103:0/1310805993 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32019f350 con 0x7fe320102a60 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32580f640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 0x7fe32019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe32500e640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320102a60 0x7fe32019a3a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d950 tx=0x7fe31c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe3167fc640 1 -- 172.21.15.103:0/1310805993 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c017030 con 0x7fe320102a60 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.738+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32019f640 con 0x7fe320102a60 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.739+0000 7fe3167fc640 1 -- 172.21.15.103:0/1310805993 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe31c004020 con 0x7fe320102a60 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.739+0000 7fe3167fc640 1 -- 172.21.15.103:0/1310805993 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0047a0 con 0x7fe320102a60 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.739+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3201a6f20 con 0x7fe320102a60 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.741+0000 7fe3167fc640 1 -- 172.21.15.103:0/1310805993 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe31c0041d0 con 0x7fe320102a60 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.741+0000 7fe329670740 1 Processor -- start 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe329670740 1 -- start start 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe329670740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93aff0 0x55f19a93b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe329670740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a945c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe329670740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a9485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe329670740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f19a8a0a40 con 0x55f19a93b910 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe329670740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f19a8389d0 con 0x55f19a93aff0 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe329670740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f19a89ca00 con 0x55f19a9461d0 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe3167fc640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe30c0a2b80 0x7fe30c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32500e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a945c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32580f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93aff0 0x55f19a93b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32500e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a945c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52712/0 (socket says 172.21.15.103:52712) 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32500e640 1 -- 172.21.15.103:0/3364806971 learned_addr learned my addr 172.21.15.103:0/3364806971 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32480d640 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a9485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32580f640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe30c0a2b80 0x7fe30c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32580f640 1 -- 172.21.15.103:0/3364806971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 msgr2=0x55f19a9485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32580f640 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a9485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32580f640 1 -- 172.21.15.103:0/3364806971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 msgr2=0x55f19a945c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.742+0000 7fe32580f640 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a945c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.743+0000 7fe32580f640 1 -- 172.21.15.103:0/3364806971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19a83f650 con 0x55f19a93aff0 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.743+0000 7fe3167fc640 1 -- 172.21.15.103:0/1310805993 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe31c0ceb00 con 0x7fe320102a60 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.743+0000 7fe32580f640 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93aff0 0x55f19a93b3d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe318006010 tx=0x7fe318031bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f44d5753320ec55e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.743+0000 7fe32580f640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe30c0a2b80 0x7fe30c0a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe318009ca0 tx=0x7fe318048000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.743+0000 7fe30b7fe640 1 -- 172.21.15.103:0/3364806971 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31804d070 con 0x55f19a93aff0 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.743+0000 7fe30b7fe640 1 -- 172.21.15.103:0/3364806971 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe318036040 con 0x55f19a93aff0 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.743+0000 7fe30b7fe640 1 -- 172.21.15.103:0/3364806971 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180305a0 con 0x55f19a93aff0 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 -- 172.21.15.103:0/3364806971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93aff0 msgr2=0x55f19a93b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93aff0 0x55f19a93b3d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe318006010 tx=0x7fe318031bb0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 -- 172.21.15.103:0/3364806971 shutdown_connections 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a9485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a945c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 --2- 172.21.15.103:0/3364806971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93aff0 0x55f19a93b3d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 -- 172.21.15.103:0/3364806971 >> 172.21.15.103:0/3364806971 conn(0x55f19a937580 msgr2=0x55f19a937960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 -- 172.21.15.103:0/3364806971 shutdown_connections 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 -- 172.21.15.103:0/3364806971 wait complete. 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 -- start start 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.744+0000 7fe329670740 1 Processor -- start 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe329670740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a9430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe329670740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93b910 0x55f19a943610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe329670740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a943b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe329670740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f19a948da0 con 0x55f19a93aff0 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe329670740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f19a85bb20 con 0x55f19a9461d0 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe329670740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f19a948f20 con 0x55f19a93b910 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe329670740 10 client.0 _ll_register_callbacks cb 0x55f19a859bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a9430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a9430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52720/0 (socket says 172.21.15.103:52720) 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 -- 172.21.15.103:0/241525329 learned_addr learned my addr 172.21.15.103:0/241525329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:26.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32580f640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a943b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32480d640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93b910 0x55f19a943610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 -- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93b910 msgr2=0x55f19a943610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93b910 0x55f19a943610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 -- 172.21.15.103:0/241525329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 msgr2=0x55f19a943b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a943b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32500e640 1 -- 172.21.15.103:0/241525329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19a6123c0 con 0x55f19a93aff0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.745+0000 7fe32580f640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a943b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe32500e640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a9430d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0ceda0 tx=0x7fe31c097670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d22a54486972905 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe308ff9640 1 -- 172.21.15.103:0/241525329 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c09ac90 con 0x55f19a93aff0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe318036040 con 0x55f19a93aff0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe308ff9640 1 -- 172.21.15.103:0/241525329 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe31c0a3b10 con 0x55f19a93aff0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe308ff9640 1 -- 172.21.15.103:0/241525329 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe31c0a3fa0 con 0x55f19a93aff0 2024-08-19T01:42:26.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 10 client.17073 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f19a954a80 con 0x55f19a93aff0 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe308ff9640 1 -- 172.21.15.103:0/241525329 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe318036040 con 0x55f19a93aff0 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe308ff9640 1 -- 172.21.15.103:0/241525329 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe31c0b4120 con 0x55f19a93aff0 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 10 client.17073 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 10 client.17073 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 20 client.17073 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9561d0 0x55f19a958650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 4 client.17073 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.746+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f19a8adfb0 con 0x55f19a9561d0 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.747+0000 7fe32480d640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9561d0 0x55f19a958650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.747+0000 7fe308ff9640 10 client.17073 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.747+0000 7fe32480d640 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9561d0 0x55f19a958650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e329356d4043f3a7 server_cookie=9b93b3c98db47fdf in_seq=0 out_seq=0 2024-08-19T01:42:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe308ff9640 1 -- 172.21.15.103:0/241525329 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f19a8adfb0 con 0x55f19a9561d0 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe308ff9640 10 client.17073 handle_command_reply: tid=0 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 client.17073 shutdown 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 msgr2=0x55f19a9430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a9430d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0ceda0 tx=0x7fe31c097670 comp rx=0 tx=0).stop 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 shutdown_connections 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9561d0 0x55f19a958650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a943b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93b910 0x55f19a943610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 --2- 172.21.15.103:0/241525329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a9430d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 >> 172.21.15.103:0/241525329 conn(0x55f19a937580 msgr2=0x55f19a93d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe308ff9640 0 client.17073 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 shutdown_connections 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.748+0000 7fe329670740 1 -- 172.21.15.103:0/241525329 wait complete. 2024-08-19T01:42:26.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.749+0000 7fe329670740 20 client.17073 trim_cache size 0 max 16384 2024-08-19T01:42:26.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 Processor -- start 2024-08-19T01:42:26.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 -- start start 2024-08-19T01:42:26.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a93daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93b910 0x55f19a951880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a940350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201af3d0 con 0x55f19a93aff0 2024-08-19T01:42:26.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201b0f10 con 0x55f19a93b910 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe329670740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201afaa0 con 0x55f19a9461d0 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32500e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a93daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93b910 0x55f19a951880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93b910 0x55f19a951880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:52778/0 (socket says 172.21.15.103:52778) 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 -- 172.21.15.103:0/508649776 learned_addr learned my addr 172.21.15.103:0/508649776 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32580f640 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a940350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 -- 172.21.15.103:0/508649776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 msgr2=0x55f19a940350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a940350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 -- 172.21.15.103:0/508649776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 msgr2=0x55f19a93daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a93daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.769+0000 7fe32480d640 1 -- 172.21.15.103:0/508649776 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19a9386f0 con 0x55f19a93b910 2024-08-19T01:42:26.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.770+0000 7fe32480d640 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93b910 0x55f19a951880 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3100045f0 tx=0x7fe31000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c821705276280d1f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.770+0000 7fe30affd640 1 -- 172.21.15.103:0/508649776 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310003ca0 con 0x55f19a93b910 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.770+0000 7fe30affd640 1 -- 172.21.15.103:0/508649776 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe310003e40 con 0x55f19a93b910 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.770+0000 7fe30affd640 1 -- 172.21.15.103:0/508649776 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310011440 con 0x55f19a93b910 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 -- 172.21.15.103:0/508649776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93b910 msgr2=0x55f19a951880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93b910 0x55f19a951880 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3100045f0 tx=0x7fe31000b390 comp rx=0 tx=0).stop 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 -- 172.21.15.103:0/508649776 shutdown_connections 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a9461d0 0x55f19a940350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a93b910 0x55f19a951880 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 --2- 172.21.15.103:0/508649776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93aff0 0x55f19a93daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 -- 172.21.15.103:0/508649776 >> 172.21.15.103:0/508649776 conn(0x55f19a937580 msgr2=0x55f19a951170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 -- 172.21.15.103:0/508649776 shutdown_connections 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 -- 172.21.15.103:0/508649776 wait complete. 2024-08-19T01:42:26.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 -- start start 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.771+0000 7fe329670740 1 Processor -- start 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe329670740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93aff0 0x55f19a95b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe329670740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a95b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe329670740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a9bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe329670740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201b0840 con 0x55f19a93b910 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe329670740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201b0170 con 0x55f19a9461d0 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe329670740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201ad3b0 con 0x55f19a93aff0 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a95b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe329670740 10 client.0 _ll_register_callbacks cb 0x7fe31c0050c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a95b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52744/0 (socket says 172.21.15.103:52744) 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 -- 172.21.15.103:0/3317500701 learned_addr learned my addr 172.21.15.103:0/3317500701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32580f640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a9bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32500e640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93aff0 0x55f19a95b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 -- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93aff0 msgr2=0x55f19a95b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93aff0 0x55f19a95b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 -- 172.21.15.103:0/3317500701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 msgr2=0x55f19a9bee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a9bee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 -- 172.21.15.103:0/3317500701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19a9bf380 con 0x55f19a93b910 2024-08-19T01:42:26.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32500e640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93aff0 0x55f19a95b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe32480d640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a95b9a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c00 tx=0x7fe310003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9429d3803b8db97 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.772+0000 7fe308ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe308ff9640 1 -- 172.21.15.103:0/3317500701 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310004730 con 0x55f19a93b910 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe308ff9640 1 -- 172.21.15.103:0/3317500701 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe3100048d0 con 0x55f19a93b910 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe310003e40 con 0x55f19a93b910 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe308ff9640 1 -- 172.21.15.103:0/3317500701 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe310004e60 con 0x55f19a93b910 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 10 client.17079 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f19a9c0660 con 0x55f19a93b910 2024-08-19T01:42:26.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe308ff9640 1 -- 172.21.15.103:0/3317500701 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe310003e40 con 0x55f19a93b910 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe308ff9640 1 -- 172.21.15.103:0/3317500701 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe3100117b0 con 0x55f19a93b910 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 10 client.17079 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 10 client.17079 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 20 client.17079 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9c0da0 0x55f19a9c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 4 client.17079 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.773+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f19a8a84c0 con 0x55f19a9c0da0 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe32500e640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9c0da0 0x55f19a9c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe308ff9640 10 client.17079 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:42:26.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe32500e640 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9c0da0 0x55f19a9c31c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eed7df030188c6ff server_cookie=ba1a2adb2f818209 in_seq=0 out_seq=0 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe308ff9640 1 -- 172.21.15.103:0/3317500701 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f19a8a84c0 con 0x55f19a9c0da0 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe308ff9640 10 client.17079 handle_command_reply: tid=0 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe329670740 1 client.17079 shutdown 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 msgr2=0x55f19a95b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe329670740 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a95b9a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c00 tx=0x7fe310003ac0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.774+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 shutdown_connections 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f19a9c0da0 0x55f19a9c31c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f19a9461d0 0x55f19a9bee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f19a93b910 0x55f19a95b9a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 1 --2- 172.21.15.103:0/3317500701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f19a93aff0 0x55f19a95b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 >> 172.21.15.103:0/3317500701 conn(0x55f19a937580 msgr2=0x55f19a951990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe308ff9640 0 client.17079 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 shutdown_connections 2024-08-19T01:42:26.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 1 -- 172.21.15.103:0/3317500701 wait complete. 2024-08-19T01:42:26.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe329670740 20 client.17079 trim_cache size 0 max 16384 2024-08-19T01:42:26.783 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:42:26.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe30c0a2b80 msgr2=0x7fe30c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe30c0a2b80 0x7fe30c0a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe318009ca0 tx=0x7fe318048000 comp rx=0 tx=0).stop 2024-08-19T01:42:26.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320102a60 msgr2=0x7fe32019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:26.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320102a60 0x7fe32019a3a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d950 tx=0x7fe31c00de10 comp rx=0 tx=0).stop 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 shutdown_connections 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe30c0a2b80 0x7fe30c0a5040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe320107fd0 0x7fe32019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe320103410 0x7fe32019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.775+0000 7fe327272640 1 --2- 172.21.15.103:0/1310805993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe320102a60 0x7fe32019a3a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.776+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 >> 172.21.15.103:0/1310805993 conn(0x7fe3200fc500 msgr2=0x7fe3201009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.776+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 shutdown_connections 2024-08-19T01:42:26.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:26.776+0000 7fe327272640 1 -- 172.21.15.103:0/1310805993 wait complete. 2024-08-19T01:42:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:27 smithi103 ceph-mon[18436]: pgmap v539: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:42:27.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:27 smithi190 ceph-mon[22032]: pgmap v539: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:42:27.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:27 smithi146 ceph-mon[22098]: pgmap v539: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-08-19T01:42:27.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:42:28.064 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 -- 172.21.15.103:0/277208599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c102c20 msgr2=0x7fa09c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/277208599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c102c20 0x7fa09c10d770 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa094009ca0 tx=0x7fa09402a950 comp rx=0 tx=0).stop 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 -- 172.21.15.103:0/277208599 shutdown_connections 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/277208599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c102c20 0x7fa09c10d770 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/277208599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c1022a0 0x7fa09c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/277208599 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa09c100b50 0x7fa09c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 -- 172.21.15.103:0/277208599 >> 172.21.15.103:0/277208599 conn(0x7fa09c0fc500 msgr2=0x7fa09c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 -- 172.21.15.103:0/277208599 shutdown_connections 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.590+0000 7fa0a27ca640 1 -- 172.21.15.103:0/277208599 wait complete. 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 Processor -- start 2024-08-19T01:42:28.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 -- start start 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c100b50 0x7fa09c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c1022a0 0x7fa09c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa09c102c20 0x7fa09c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c110b60 con 0x7fa09c1022a0 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1109e0 con 0x7fa09c102c20 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.591+0000 7fa0a27ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c110ce0 con 0x7fa09c100b50 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c1022a0 0x7fa09c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c1022a0 0x7fa09c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52756/0 (socket says 172.21.15.103:52756) 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 -- 172.21.15.103:0/443617605 learned_addr learned my addr 172.21.15.103:0/443617605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09bfff640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c100b50 0x7fa09c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa09c102c20 0x7fa09c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 -- 172.21.15.103:0/443617605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c100b50 msgr2=0x7fa09c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c100b50 0x7fa09c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 -- 172.21.15.103:0/443617605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa09c102c20 msgr2=0x7fa09c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa09c102c20 0x7fa09c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 -- 172.21.15.103:0/443617605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c19d040 con 0x7fa09c1022a0 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa09c102c20 0x7fa09c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa09b7fe640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c1022a0 0x7fa09c1985d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa09000d950 tx=0x7fa09000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.592+0000 7fa0997fa640 1 -- 172.21.15.103:0/443617605 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090017070 con 0x7fa09c1022a0 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.593+0000 7fa0997fa640 1 -- 172.21.15.103:0/443617605 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa090009c30 con 0x7fa09c1022a0 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.593+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09c19d330 con 0x7fa09c1022a0 2024-08-19T01:42:28.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.593+0000 7fa0997fa640 1 -- 172.21.15.103:0/443617605 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09000f620 con 0x7fa09c1022a0 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.593+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09c0fc8e0 con 0x7fa09c1022a0 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.595+0000 7fa0997fa640 1 -- 172.21.15.103:0/443617605 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa09000f7c0 con 0x7fa09c1022a0 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 Processor -- start 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 -- start start 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e8910 0x55abac8f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8f31d0 0x55abac8f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abac84da40 con 0x55abac8e7ff0 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55abac7e59d0 con 0x55abac8e8910 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55abac849a00 con 0x55abac8f31d0 2024-08-19T01:42:28.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a0d67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a0d67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8e83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52760/0 (socket says 172.21.15.103:52760) 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.596+0000 7fa0a0d67640 1 -- 172.21.15.103:0/292622423 learned_addr learned my addr 172.21.15.103:0/292622423 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa09bfff640 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e8910 0x55abac8f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa09b7fe640 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8f31d0 0x55abac8f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0a0d67640 1 -- 172.21.15.103:0/292622423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8f31d0 msgr2=0x55abac8f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8f31d0 0x55abac8f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0a0d67640 1 -- 172.21.15.103:0/292622423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e8910 msgr2=0x55abac8f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e8910 0x55abac8f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0997fa640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0840a2cd0 0x7fa0840a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0a0d67640 1 -- 172.21.15.103:0/292622423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abac7ec650 con 0x55abac8e7ff0 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa09b7fe640 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8f31d0 0x55abac8f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:28.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0997fa640 1 -- 172.21.15.103:0/443617605 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa0900ceb90 con 0x7fa09c1022a0 2024-08-19T01:42:28.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8e83d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa094006010 tx=0x7fa094009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ee147ab54fa563 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa09b7fe640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0840a2cd0 0x7fa0840a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.597+0000 7fa0827fc640 1 -- 172.21.15.103:0/292622423 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094034050 con 0x55abac8e7ff0 2024-08-19T01:42:28.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0827fc640 1 -- 172.21.15.103:0/292622423 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa09402fac0 con 0x55abac8e7ff0 2024-08-19T01:42:28.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0827fc640 1 -- 172.21.15.103:0/292622423 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940043c0 con 0x55abac8e7ff0 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/292622423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 msgr2=0x55abac8e83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8e83d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa094006010 tx=0x7fa094009a20 comp rx=0 tx=0).stop 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa09b7fe640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0840a2cd0 0x7fa0840a5190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa0900cca30 tx=0x7fa090097040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/292622423 shutdown_connections 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8f31d0 0x55abac8f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e8910 0x55abac8f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/292622423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8e83d0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/292622423 >> 172.21.15.103:0/292622423 conn(0x55abac8e4580 msgr2=0x55abac8e4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/292622423 shutdown_connections 2024-08-19T01:42:28.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.598+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/292622423 wait complete. 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.599+0000 7fa0a4bc8740 1 -- start start 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.599+0000 7fa0a4bc8740 1 Processor -- start 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.599+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e7ff0 0x55abac8f0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.599+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.599+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8f31d0 0x55abac8f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abac8f5da0 con 0x55abac8f31d0 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55abac808b20 con 0x55abac8e7ff0 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55abac8f5f20 con 0x55abac8e8910 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a4bc8740 10 client.0 _ll_register_callbacks cb 0x55abac806bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a0d67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8f31d0 0x55abac8f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a0d67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8f31d0 0x55abac8f0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52764/0 (socket says 172.21.15.103:52764) 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1361358161 learned_addr learned my addr 172.21.15.103:0/1361358161 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e7ff0 0x55abac8f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa09bfff640 1 -- 172.21.15.103:0/1361358161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e7ff0 msgr2=0x55abac8f0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e7ff0 0x55abac8f0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa09bfff640 1 -- 172.21.15.103:0/1361358161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8f31d0 msgr2=0x55abac8f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8f31d0 0x55abac8f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa09bfff640 1 -- 172.21.15.103:0/1361358161 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abac5bf3c0 con 0x55abac8e8910 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f0560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0126d0 tx=0x7fa08c012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c3ee81afb0916d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.600+0000 7fa07b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1361358161 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00f8c0 con 0x55abac8e8910 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09402fac0 con 0x55abac8e8910 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1361358161 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa08c00fa60 con 0x55abac8e8910 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1361358161 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa08c00fc60 con 0x55abac8e8910 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa0a4bc8740 10 client.26347 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:28.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55abac901950 con 0x55abac8e8910 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1361358161 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa09402fac0 con 0x55abac8e8910 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.601+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1361358161 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa08c027420 con 0x55abac8e8910 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a4bc8740 10 client.26347 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a4bc8740 10 client.26347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a4bc8740 20 client.26347 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac902e20 0x55abac905240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a4bc8740 4 client.26347 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abac85afb0 con 0x55abac902e20 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac902e20 0x55abac905240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac902e20 0x55abac905240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0e32f83e2ff0aca server_cookie=e84147ff0aba0d15 in_seq=0 out_seq=0 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.602+0000 7fa07b7fe640 10 client.26347 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.603+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1361358161 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abac85afb0 con 0x55abac902e20 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.603+0000 7fa07b7fe640 10 client.26347 handle_command_reply: tid=0 2024-08-19T01:42:28.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.603+0000 7fa0a4bc8740 1 client.26347 shutdown 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.603+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 msgr2=0x55abac8f0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.603+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f0560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0126d0 tx=0x7fa08c012b90 comp rx=0 tx=0).stop 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 shutdown_connections 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac902e20 0x55abac905240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8f31d0 0x55abac8f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f0560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1361358161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8e7ff0 0x55abac8f0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa07b7fe640 0 client.26347 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 >> 172.21.15.103:0/1361358161 conn(0x55abac8e4580 msgr2=0x55abac8ea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 shutdown_connections 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1361358161 wait complete. 2024-08-19T01:42:28.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.604+0000 7fa0a4bc8740 20 client.26347 trim_cache size 0 max 16384 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 Processor -- start 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 -- start start 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac90ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1af1f0 con 0x55abac8e7ff0 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1b0d30 con 0x55abac8f31d0 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1af8c0 con 0x55abac8e8910 2024-08-19T01:42:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8f08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52766/0 (socket says 172.21.15.103:52766) 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1236962745 learned_addr learned my addr 172.21.15.103:0/1236962745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac90ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1236962745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 msgr2=0x55abac8f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1236962745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 msgr2=0x55abac90ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac90ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1236962745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abac8e5630 con 0x55abac8e7ff0 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.620+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.621+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8f08f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa094009430 tx=0x7fa094004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be21da1248747f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.621+0000 7fa081ffb640 1 -- 172.21.15.103:0/1236962745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094034050 con 0x55abac8e7ff0 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.621+0000 7fa081ffb640 1 -- 172.21.15.103:0/1236962745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa09402f690 con 0x55abac8e7ff0 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.621+0000 7fa081ffb640 1 -- 172.21.15.103:0/1236962745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094004630 con 0x55abac8e7ff0 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1236962745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 msgr2=0x55abac8f08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8f08f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa094009430 tx=0x7fa094004a40 comp rx=0 tx=0).stop 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1236962745 shutdown_connections 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac90ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e8910 0x55abac8f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1236962745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e7ff0 0x55abac8f08f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1236962745 >> 172.21.15.103:0/1236962745 conn(0x55abac8e4580 msgr2=0x55abac8f1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1236962745 shutdown_connections 2024-08-19T01:42:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1236962745 wait complete. 2024-08-19T01:42:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.622+0000 7fa0a4bc8740 1 -- start start 2024-08-19T01:42:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 1 Processor -- start 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e7ff0 0x55abac8fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e8910 0x55abac8eca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac8ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1b0660 con 0x55abac8e8910 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1aff90 con 0x55abac8f31d0 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1ad1d0 con 0x55abac8e7ff0 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a4bc8740 10 client.0 _ll_register_callbacks cb 0x7fa08c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa09bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e8910 0x55abac8eca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa09bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e8910 0x55abac8eca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52768/0 (socket says 172.21.15.103:52768) 2024-08-19T01:42:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa09bfff640 1 -- 172.21.15.103:0/1505281058 learned_addr learned my addr 172.21.15.103:0/1505281058 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.623+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e7ff0 0x55abac8fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac8ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1505281058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 msgr2=0x55abac8ecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac8ecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1505281058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e8910 msgr2=0x55abac8eca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e8910 0x55abac8eca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a0d67640 1 -- 172.21.15.103:0/1505281058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abac8ed580 con 0x55abac8e7ff0 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a0d67640 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e7ff0 0x55abac8fb5f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa094054040 tx=0x7fa094004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c537c1f88a6b05f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa07b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1505281058 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094060020 con 0x55abac8e7ff0 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09402f690 con 0x55abac8e7ff0 2024-08-19T01:42:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1505281058 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa094004d80 con 0x55abac8e7ff0 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1505281058 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa094034030 con 0x55abac8e7ff0 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a4bc8740 10 client.26353 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.624+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55abac9760a0 con 0x55abac8e7ff0 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1505281058 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa09402f690 con 0x55abac8e7ff0 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1505281058 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa09404e040 con 0x55abac8e7ff0 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa0a4bc8740 10 client.26353 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa0a4bc8740 10 client.26353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa0a4bc8740 20 client.26353 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac976270 0x55abac978660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa0a4bc8740 4 client.26353 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abac8554c0 con 0x55abac976270 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.625+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac976270 0x55abac978660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.626+0000 7fa07b7fe640 10 client.26353 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:42:28.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.626+0000 7fa09bfff640 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac976270 0x55abac978660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c47e4fb3ef4f8013 server_cookie=166ececaafda0058 in_seq=0 out_seq=0 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.626+0000 7fa07b7fe640 1 -- 172.21.15.103:0/1505281058 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abac8554c0 con 0x55abac976270 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.626+0000 7fa07b7fe640 10 client.26353 handle_command_reply: tid=0 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.626+0000 7fa0a4bc8740 1 client.26353 shutdown 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e7ff0 msgr2=0x55abac8fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e7ff0 0x55abac8fb5f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa094054040 tx=0x7fa094004210 comp rx=0 tx=0).stop 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 shutdown_connections 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55abac976270 0x55abac978660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abac8f31d0 0x55abac8ecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abac8e8910 0x55abac8eca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 --2- 172.21.15.103:0/1505281058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abac8e7ff0 0x55abac8fb5f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 >> 172.21.15.103:0/1505281058 conn(0x55abac8e4580 msgr2=0x55abac8f1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa07b7fe640 0 client.26353 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:42:28.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 shutdown_connections 2024-08-19T01:42:28.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 1 -- 172.21.15.103:0/1505281058 wait complete. 2024-08-19T01:42:28.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.627+0000 7fa0a4bc8740 20 client.26353 trim_cache size 0 max 16384 2024-08-19T01:42:28.634 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:42:28.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0840a2cd0 msgr2=0x7fa0840a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0840a2cd0 0x7fa0840a5190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa0900cca30 tx=0x7fa090097040 comp rx=0 tx=0).stop 2024-08-19T01:42:28.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c1022a0 msgr2=0x7fa09c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c1022a0 0x7fa09c1985d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa09000d950 tx=0x7fa09000de10 comp rx=0 tx=0).stop 2024-08-19T01:42:28.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 shutdown_connections 2024-08-19T01:42:28.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa0840a2cd0 0x7fa0840a5190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa09c102c20 0x7fa09c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa09c1022a0 0x7fa09c1985d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 --2- 172.21.15.103:0/443617605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa09c100b50 0x7fa09c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.628+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 >> 172.21.15.103:0/443617605 conn(0x7fa09c0fc500 msgr2=0x7fa09c069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.629+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 shutdown_connections 2024-08-19T01:42:28.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:28.629+0000 7fa0a27ca640 1 -- 172.21.15.103:0/443617605 wait complete. 2024-08-19T01:42:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 -- 172.21.15.103:0/2603324953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 msgr2=0x7f28f4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 --2- 172.21.15.103:0/2603324953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 0x7f28f4109cd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009ca0 tx=0x7f28ec02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 -- 172.21.15.103:0/2603324953 shutdown_connections 2024-08-19T01:42:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 --2- 172.21.15.103:0/2603324953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 0x7f28f4109cd0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 --2- 172.21.15.103:0/2603324953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28f4102a80 0x7f28f4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 --2- 172.21.15.103:0/2603324953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4101890 0x7f28f4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 -- 172.21.15.103:0/2603324953 >> 172.21.15.103:0/2603324953 conn(0x7f28f40fcff0 msgr2=0x7f28f40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 -- 172.21.15.103:0/2603324953 shutdown_connections 2024-08-19T01:42:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.662+0000 7f28f992c640 1 -- 172.21.15.103:0/2603324953 wait complete. 2024-08-19T01:42:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.663+0000 7f28f992c640 1 Processor -- start 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.663+0000 7f28f992c640 1 -- start start 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f992c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28f4101890 0x7f28f4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f992c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4102a80 0x7f28f4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f992c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 0x7f28f419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f992c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28f410d3c0 con 0x7f28f4103440 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f992c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28f410d240 con 0x7f28f4101890 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f992c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28f410d540 con 0x7f28f4102a80 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4102a80 0x7f28f4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 0x7f28f419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4102a80 0x7f28f4197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:52596/0 (socket says 172.21.15.103:52596) 2024-08-19T01:42:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f27fc640 1 -- 172.21.15.103:0/3341172970 learned_addr learned my addr 172.21.15.103:0/3341172970 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28f4101890 0x7f28f4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f37fe640 1 -- 172.21.15.103:0/3341172970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4102a80 msgr2=0x7f28f4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f37fe640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4102a80 0x7f28f4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f37fe640 1 -- 172.21.15.103:0/3341172970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28f4101890 msgr2=0x7f28f4197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f37fe640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28f4101890 0x7f28f4197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f37fe640 1 -- 172.21.15.103:0/3341172970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f4074150 con 0x7f28f4103440 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.664+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28f4101890 0x7f28f4197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.665+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4102a80 0x7f28f4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.665+0000 7f28f37fe640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 0x7f28f419eff0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009ca0 tx=0x7f28ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.665+0000 7f28f892a640 1 -- 172.21.15.103:0/3341172970 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec003c30 con 0x7f28f4103440 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.665+0000 7f28f892a640 1 -- 172.21.15.103:0/3341172970 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28ec003dd0 con 0x7f28f4103440 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.665+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f40743e0 con 0x7f28f4103440 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.665+0000 7f28f892a640 1 -- 172.21.15.103:0/3341172970 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0309e0 con 0x7f28f4103440 2024-08-19T01:42:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.665+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f4074980 con 0x7f28f4103440 2024-08-19T01:42:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 Processor -- start 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28f892a640 1 -- 172.21.15.103:0/3341172970 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f28ec004580 con 0x7f28f4103440 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 -- start start 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597680 0x561f29597a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f2959ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f2959d050 0x561f2959f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f294fa9d0 con 0x561f29597680 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561f294a2820 con 0x561f2959d050 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561f294f6990 con 0x561f29597fc0 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597680 0x561f29597a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597680 0x561f29597a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52816/0 (socket says 172.21.15.103:52816) 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.668+0000 7f28f27fc640 1 -- 172.21.15.103:0/3923340514 learned_addr learned my addr 172.21.15.103:0/3923340514 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f2959ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f892a640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28dc0a2cb0 0x7f28dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f27fc640 1 -- 172.21.15.103:0/3923340514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 msgr2=0x561f2959ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f2959ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f27fc640 1 -- 172.21.15.103:0/3923340514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f2959d050 msgr2=0x561f2959f510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f2959d050 0x561f2959f510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f27fc640 1 -- 172.21.15.103:0/3923340514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2920e210 con 0x561f29597680 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f37fe640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28dc0a2cb0 0x7f28dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f892a640 1 -- 172.21.15.103:0/3341172970 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f28ec0f0430 con 0x7f28f4103440 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597680 0x561f29597a80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f28e80115c0 tx=0x7f28e8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd3fbb4ed162caf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.669+0000 7f28d9ffb640 1 -- 172.21.15.103:0/3923340514 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8004030 con 0x561f29597680 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28d9ffb640 1 -- 172.21.15.103:0/3923340514 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28e80041d0 con 0x561f29597680 2024-08-19T01:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28d9ffb640 1 -- 172.21.15.103:0/3923340514 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e80038f0 con 0x561f29597680 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3923340514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597680 msgr2=0x561f29597a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28f37fe640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28dc0a2cb0 0x7f28dc0a5170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0b9120 tx=0x7f28ec087940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597680 0x561f29597a80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f28e80115c0 tx=0x7f28e8011a80 comp rx=0 tx=0).stop 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3923340514 shutdown_connections 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f2959d050 0x561f2959f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f2959ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3923340514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597680 0x561f29597a80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.670+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3923340514 >> 172.21.15.103:0/3923340514 conn(0x561f29592cf0 msgr2=0x561f295930f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3923340514 shutdown_connections 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3923340514 wait complete. 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 -- start start 2024-08-19T01:42:28.678 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 Processor -- start 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f295a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f295a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f295a0020 con 0x561f2959d050 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561f294b6520 con 0x561f29597680 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561f295a01a0 con 0x561f29597fc0 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28fc7c9740 10 client.0 _ll_register_callbacks cb 0x561f2920da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52826/0 (socket says 172.21.15.103:52826) 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.671+0000 7f28f27fc640 1 -- 172.21.15.103:0/3603076358 learned_addr learned my addr 172.21.15.103:0/3603076358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f295a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f295a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f27fc640 1 -- 172.21.15.103:0/3603076358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 msgr2=0x561f295a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f295a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f27fc640 1 -- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 msgr2=0x561f295a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f295a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f27fc640 1 -- 172.21.15.103:0/3603076358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f292da620 con 0x561f2959d050 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f295a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28f27fc640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a8e20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f28e801b040 tx=0x7f28e8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64121d38325c9c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28d2ffd640 1 -- 172.21.15.103:0/3603076358 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e802a060 con 0x561f2959d050 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28d2ffd640 1 -- 172.21.15.103:0/3603076358 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f28e8005260 con 0x561f2959d050 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e80041d0 con 0x561f2959d050 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28d2ffd640 1 -- 172.21.15.103:0/3603076358 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f28e8005460 con 0x561f2959d050 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28fc7c9740 10 client.17109 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.672+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561f295abc10 con 0x561f2959d050 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28d2ffd640 1 -- 172.21.15.103:0/3603076358 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28e80041d0 con 0x561f2959d050 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28d2ffd640 1 -- 172.21.15.103:0/3603076358 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f28e80196b0 con 0x561f2959d050 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28fc7c9740 10 client.17109 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28fc7c9740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28fc7c9740 20 client.17109 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f295ad070 0x561f295af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28fc7c9740 4 client.17109 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f29508100 con 0x561f295ad070 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.673+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f295ad070 0x561f295af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.674+0000 7f28d2ffd640 10 client.17109 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.674+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f295ad070 0x561f295af4a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d71d926c51d630 server_cookie=24c18b97aa61894b in_seq=0 out_seq=0 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.674+0000 7f28d2ffd640 1 -- 172.21.15.103:0/3603076358 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f29508100 con 0x561f295ad070 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.674+0000 7f28d2ffd640 10 client.17109 handle_command_reply: tid=0 2024-08-19T01:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.674+0000 7f28fc7c9740 1 client.17109 shutdown 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 msgr2=0x561f295a8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a8e20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f28e801b040 tx=0x7f28e8009310 comp rx=0 tx=0).stop 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 shutdown_connections 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f295ad070 0x561f295af4a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a8e20 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597fc0 0x561f295a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/3603076358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f295a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 >> 172.21.15.103:0/3603076358 conn(0x561f29592cf0 msgr2=0x561f29594500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28d2ffd640 0 client.17109 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 shutdown_connections 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 1 -- 172.21.15.103:0/3603076358 wait complete. 2024-08-19T01:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.675+0000 7f28fc7c9740 20 client.17109 trim_cache size 0 max 16384 2024-08-19T01:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 Processor -- start 2024-08-19T01:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 -- start start 2024-08-19T01:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f29591680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597fc0 0x561f295a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f2959d050 0x561f29591bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41aada0 con 0x561f29597fc0 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41ac8e0 con 0x561f29597680 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41ab470 con 0x561f2959d050 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597fc0 0x561f295a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597fc0 0x561f295a8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52830/0 (socket says 172.21.15.103:52830) 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 -- 172.21.15.103:0/2010275854 learned_addr learned my addr 172.21.15.103:0/2010275854 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f27fc640 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f29591680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 -- 172.21.15.103:0/2010275854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f2959d050 msgr2=0x561f29591bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f2959d050 0x561f29591bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 -- 172.21.15.103:0/2010275854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 msgr2=0x561f29591680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f29591680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f2ffd640 1 -- 172.21.15.103:0/2010275854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f29592b10 con 0x561f29597fc0 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.695+0000 7f28f27fc640 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f29591680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.696+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597fc0 0x561f295a8d40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x561f295989f0 tx=0x7f28e4012c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee593c6e8fbedde server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.696+0000 7f28d97fa640 1 -- 172.21.15.103:0/2010275854 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4012310 con 0x561f29597fc0 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.696+0000 7f28d97fa640 1 -- 172.21.15.103:0/2010275854 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28e40124b0 con 0x561f29597fc0 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.696+0000 7f28d97fa640 1 -- 172.21.15.103:0/2010275854 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4010e60 con 0x561f29597fc0 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2010275854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597fc0 msgr2=0x561f295a8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597fc0 0x561f295a8d40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x561f295989f0 tx=0x7f28e4012c20 comp rx=0 tx=0).stop 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2010275854 shutdown_connections 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f2959d050 0x561f29591bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f29597fc0 0x561f295a8d40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2010275854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597680 0x561f29591680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2010275854 >> 172.21.15.103:0/2010275854 conn(0x561f29592cf0 msgr2=0x561f295a18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2010275854 shutdown_connections 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2010275854 wait complete. 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 -- start start 2024-08-19T01:42:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.697+0000 7f28fc7c9740 1 Processor -- start 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597680 0x561f295a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597fc0 0x561f295a6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28fc7c9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41ac210 con 0x561f2959d050 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41abb40 con 0x561f29597fc0 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28fc7c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28f41a8d80 con 0x561f29597680 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28fc7c9740 10 client.0 _ll_register_callbacks cb 0x7f28e4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52840/0 (socket says 172.21.15.103:52840) 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 -- 172.21.15.103:0/2304955025 learned_addr learned my addr 172.21.15.103:0/2304955025 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597680 0x561f295a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597fc0 0x561f295a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 -- 172.21.15.103:0/2304955025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597680 msgr2=0x561f295a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597680 0x561f295a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 -- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597fc0 msgr2=0x561f295a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597fc0 0x561f295a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 -- 172.21.15.103:0/2304955025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f295a7980 con 0x561f2959d050 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28f27fc640 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a7370 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f28e8019bf0 tx=0x7f28e8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ae275bf07dd48b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.698+0000 7f28d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28d2ffd640 1 -- 172.21.15.103:0/2304955025 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8035020 con 0x561f2959d050 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e40124b0 con 0x561f2959d050 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28d2ffd640 1 -- 172.21.15.103:0/2304955025 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f28e8011270 con 0x561f2959d050 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28d2ffd640 1 -- 172.21.15.103:0/2304955025 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f28e802a040 con 0x561f2959d050 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 10 client.17115 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561f295a8c90 con 0x561f2959d050 2024-08-19T01:42:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28d2ffd640 1 -- 172.21.15.103:0/2304955025 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28e40124b0 con 0x561f2959d050 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28d2ffd640 1 -- 172.21.15.103:0/2304955025 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f28e80048c0 con 0x561f2959d050 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 10 client.17115 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 20 client.17115 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f29618020 0x561f2961a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 4 client.17115 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f29502450 con 0x561f29618020 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.699+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f29618020 0x561f2961a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.700+0000 7f28d2ffd640 10 client.17115 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.700+0000 7f28f2ffd640 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f29618020 0x561f2961a440 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a436bcab7d0c6b server_cookie=c7a33148b767b5b9 in_seq=0 out_seq=0 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.706+0000 7f28d2ffd640 1 -- 172.21.15.103:0/2304955025 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f29502450 con 0x561f29618020 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.706+0000 7f28d2ffd640 10 client.17115 handle_command_reply: tid=0 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.706+0000 7f28fc7c9740 1 client.17115 shutdown 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 msgr2=0x561f295a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a7370 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f28e8019bf0 tx=0x7f28e8011a80 comp rx=0 tx=0).stop 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 shutdown_connections 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561f29618020 0x561f2961a440 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f2959d050 0x561f295a7370 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561f29597fc0 0x561f295a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28d2ffd640 0 client.17115 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 --2- 172.21.15.103:0/2304955025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f29597680 0x561f295a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 >> 172.21.15.103:0/2304955025 conn(0x561f29592cf0 msgr2=0x561f29591680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 shutdown_connections 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 1 -- 172.21.15.103:0/2304955025 wait complete. 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28fc7c9740 20 client.17115 trim_cache size 0 max 16384 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:42:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:42:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28dc0a2cb0 msgr2=0x7f28dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28dc0a2cb0 0x7f28dc0a5170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0b9120 tx=0x7f28ec087940 comp rx=0 tx=0).stop 2024-08-19T01:42:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 msgr2=0x7f28f419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 0x7f28f419eff0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009ca0 tx=0x7f28ec004a40 comp rx=0 tx=0).stop 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 shutdown_connections 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f28dc0a2cb0 0x7f28dc0a5170 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28f4103440 0x7f28f419eff0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28f4102a80 0x7f28f4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 --2- 172.21.15.103:0/3341172970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28f4101890 0x7f28f4197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.707+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 >> 172.21.15.103:0/3341172970 conn(0x7f28f40fcff0 msgr2=0x7f28f4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.708+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 shutdown_connections 2024-08-19T01:42:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:28.708+0000 7f28f992c640 1 -- 172.21.15.103:0/3341172970 wait complete. 2024-08-19T01:42:29.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:29 smithi103 ceph-mon[18436]: pgmap v540: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:42:29.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:42:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:29 smithi190 ceph-mon[22032]: pgmap v540: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:42:29.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:29 smithi146 ceph-mon[22098]: pgmap v540: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:42:29.649 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:30.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 -- 172.21.15.103:0/3392787198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4107e80 msgr2=0x7f0ac410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/3392787198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4107e80 0x7f0ac410e7b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009ca0 tx=0x7f0abc02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 -- 172.21.15.103:0/3392787198 shutdown_connections 2024-08-19T01:42:30.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/3392787198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4107e80 0x7f0ac410e7b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/3392787198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4101000 0x7f0ac4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/3392787198 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ac4100650 0x7f0ac4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 -- 172.21.15.103:0/3392787198 >> 172.21.15.103:0/3392787198 conn(0x7f0ac4075ca0 msgr2=0x7f0ac40760b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 -- 172.21.15.103:0/3392787198 shutdown_connections 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.174+0000 7f0aca4f8640 1 -- 172.21.15.103:0/3392787198 wait complete. 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.175+0000 7f0aca4f8640 1 Processor -- start 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.175+0000 7f0aca4f8640 1 -- start start 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0aca4f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4100650 0x7f0ac41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0aca4f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4101000 0x7f0ac4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0aca4f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ac4107e80 0x7f0ac419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0aca4f8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4111bb0 con 0x7f0ac4101000 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0aca4f8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4111a30 con 0x7f0ac4100650 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0aca4f8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4111d30 con 0x7f0ac4107e80 2024-08-19T01:42:30.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4101000 0x7f0ac4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4101000 0x7f0ac4198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52854/0 (socket says 172.21.15.103:52854) 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 -- 172.21.15.103:0/1753871646 learned_addr learned my addr 172.21.15.103:0/1753871646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ac4107e80 0x7f0ac419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4100650 0x7f0ac41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 -- 172.21.15.103:0/1753871646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ac4107e80 msgr2=0x7f0ac419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ac4107e80 0x7f0ac419c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 -- 172.21.15.103:0/1753871646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4100650 msgr2=0x7f0ac41980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4100650 0x7f0ac41980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 -- 172.21.15.103:0/1753871646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac419d080 con 0x7f0ac4101000 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4100650 0x7f0ac41980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:30.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.176+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4101000 0x7f0ac4198610 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b4a0 tx=0x7f0ab800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:30.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.177+0000 7f0ac17fa640 1 -- 172.21.15.103:0/1753871646 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8014070 con 0x7f0ac4101000 2024-08-19T01:42:30.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.177+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac419d370 con 0x7f0ac4101000 2024-08-19T01:42:30.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.177+0000 7f0ac17fa640 1 -- 172.21.15.103:0/1753871646 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab8010ac0 con 0x7f0ac4101000 2024-08-19T01:42:30.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.177+0000 7f0ac17fa640 1 -- 172.21.15.103:0/1753871646 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8017c30 con 0x7f0ac4101000 2024-08-19T01:42:30.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.177+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41a4c50 con 0x7f0ac4101000 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.179+0000 7f0ac17fa640 1 -- 172.21.15.103:0/1753871646 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0ab8017070 con 0x7f0ac4101000 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 Processor -- start 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 -- start start 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b49ff0 0x560034b4a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b4a910 0x560034b54c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560034aafa40 con 0x560034b4a910 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560034a479d0 con 0x560034b49ff0 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560034aaba00 con 0x560034b551d0 2024-08-19T01:42:30.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0ac17fa640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0aac0a2b80 0x7f0aac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0ac8a95640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b49ff0 0x560034b4a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0ac8a95640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b49ff0 0x560034b4a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:52890/0 (socket says 172.21.15.103:52890) 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0ac8a95640 1 -- 172.21.15.103:0/2066225584 learned_addr learned my addr 172.21.15.103:0/2066225584 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.180+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b4a910 0x560034b54c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0aac0a2b80 0x7f0aac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac17fa640 1 -- 172.21.15.103:0/1753871646 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0ab80ce710 con 0x7f0ac4101000 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac3fff640 1 -- 172.21.15.103:0/2066225584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 msgr2=0x560034b575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac3fff640 1 -- 172.21.15.103:0/2066225584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b49ff0 msgr2=0x560034b4a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b49ff0 0x560034b4a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac3fff640 1 -- 172.21.15.103:0/2066225584 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560034a4e650 con 0x560034b4a910 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b4a910 0x560034b54c90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005fd0 tx=0x7f0ab400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610d6691e95dc65a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0aac0a2b80 0x7f0aac0a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0319e0 tx=0x7f0abc009860 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0aaa7fc640 1 -- 172.21.15.103:0/2066225584 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4004030 con 0x560034b4a910 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.181+0000 7f0aaa7fc640 1 -- 172.21.15.103:0/2066225584 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab40054d0 con 0x560034b4a910 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0aaa7fc640 1 -- 172.21.15.103:0/2066225584 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4004a70 con 0x560034b4a910 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2066225584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b4a910 msgr2=0x560034b54c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b4a910 0x560034b54c90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005fd0 tx=0x7f0ab400f730 comp rx=0 tx=0).stop 2024-08-19T01:42:30.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2066225584 shutdown_connections 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b4a910 0x560034b54c90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2066225584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b49ff0 0x560034b4a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2066225584 >> 172.21.15.103:0/2066225584 conn(0x560034b46580 msgr2=0x560034b46960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2066225584 shutdown_connections 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.182+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2066225584 wait complete. 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 -- start start 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 Processor -- start 2024-08-19T01:42:30.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b54c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b51fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560034b5c200 con 0x560034b49ff0 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560034a6ab20 con 0x560034b4a910 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560034b5c380 con 0x560034b551d0 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b54c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.183+0000 7f0acc8f6740 10 client.0 _ll_register_callbacks cb 0x560034a68bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b54c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52866/0 (socket says 172.21.15.103:52866) 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 -- 172.21.15.103:0/3847991903 learned_addr learned my addr 172.21.15.103:0/3847991903 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b51fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 -- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 msgr2=0x560034b52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 -- 172.21.15.103:0/3847991903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 msgr2=0x560034b51fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b51fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 -- 172.21.15.103:0/3847991903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600348213c0 con 0x560034b49ff0 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b51fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b54c80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4004580 tx=0x7f0ab4005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402da54ee8ca3c9e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.184+0000 7f0aa37fe640 1 -- 172.21.15.103:0/3847991903 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab401b450 con 0x560034b49ff0 2024-08-19T01:42:30.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0aa37fe640 1 -- 172.21.15.103:0/3847991903 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0ab401b5f0 con 0x560034b49ff0 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab40054d0 con 0x560034b49ff0 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0aa37fe640 1 -- 172.21.15.103:0/3847991903 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0ab401b7f0 con 0x560034b49ff0 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 10 client.17127 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560034b67ca0 con 0x560034b49ff0 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0aa37fe640 1 -- 172.21.15.103:0/3847991903 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab40054d0 con 0x560034b49ff0 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0aa37fe640 1 -- 172.21.15.103:0/3847991903 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab4011b40 con 0x560034b49ff0 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 10 client.17127 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 10 client.17127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 20 client.17127 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:30.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034b69100 0x560034b6b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 4 client.17127 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.185+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560034abcfb0 con 0x560034b69100 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.186+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034b69100 0x560034b6b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.186+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034b69100 0x560034b6b4f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a93e7dd46676d7b8 server_cookie=a23cfa4299b31254 in_seq=0 out_seq=0 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.186+0000 7f0aa37fe640 10 client.17127 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0aa37fe640 1 -- 172.21.15.103:0/3847991903 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560034abcfb0 con 0x560034b69100 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0aa37fe640 10 client.17127 handle_command_reply: tid=0 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 client.17127 shutdown 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 msgr2=0x560034b54c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b54c80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4004580 tx=0x7f0ab4005680 comp rx=0 tx=0).stop 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 shutdown_connections 2024-08-19T01:42:30.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034b69100 0x560034b6b4f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0aa37fe640 0 client.17127 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b551d0 0x560034b52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b51fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/3847991903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b54c80 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 >> 172.21.15.103:0/3847991903 conn(0x560034b46580 msgr2=0x560034b4c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 shutdown_connections 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 1 -- 172.21.15.103:0/3847991903 wait complete. 2024-08-19T01:42:30.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.187+0000 7f0acc8f6740 20 client.17127 trim_cache size 0 max 16384 2024-08-19T01:42:30.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 Processor -- start 2024-08-19T01:42:30.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 -- start start 2024-08-19T01:42:30.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b49ff0 0x560034b4d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b4d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b551d0 0x560034b70cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41ab020 con 0x560034b551d0 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41acb60 con 0x560034b4a910 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41ab6f0 con 0x560034b49ff0 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.203+0000 7f0ac8a95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b551d0 0x560034b70cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b551d0 0x560034b70cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52872/0 (socket says 172.21.15.103:52872) 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 -- 172.21.15.103:0/1901094748 learned_addr learned my addr 172.21.15.103:0/1901094748 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b4d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b49ff0 0x560034b4d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 -- 172.21.15.103:0/1901094748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b49ff0 msgr2=0x560034b4d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b49ff0 0x560034b4d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 -- 172.21.15.103:0/1901094748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 msgr2=0x560034b4d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b4d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 -- 172.21.15.103:0/1901094748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560034b52ad0 con 0x560034b551d0 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b4d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b551d0 0x560034b70cf0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009ca0 tx=0x7f0abc042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783e7f0a3d14dc1f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0aa9ffb640 1 -- 172.21.15.103:0/1901094748 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc031c40 con 0x560034b551d0 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0aa9ffb640 1 -- 172.21.15.103:0/1901094748 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0abc050070 con 0x560034b551d0 2024-08-19T01:42:30.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.204+0000 7f0aa9ffb640 1 -- 172.21.15.103:0/1901094748 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc032050 con 0x560034b551d0 2024-08-19T01:42:30.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 -- 172.21.15.103:0/1901094748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b551d0 msgr2=0x560034b70cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b551d0 0x560034b70cf0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009ca0 tx=0x7f0abc042000 comp rx=0 tx=0).stop 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 -- 172.21.15.103:0/1901094748 shutdown_connections 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b551d0 0x560034b70cf0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b4a910 0x560034b4d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/1901094748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b49ff0 0x560034b4d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 -- 172.21.15.103:0/1901094748 >> 172.21.15.103:0/1901094748 conn(0x560034b46580 msgr2=0x560034b411b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 -- 172.21.15.103:0/1901094748 shutdown_connections 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.205+0000 7f0acc8f6740 1 -- 172.21.15.103:0/1901094748 wait complete. 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 -- start start 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 Processor -- start 2024-08-19T01:42:30.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b4a910 0x560034b61b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b551d0 0x560034b4e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41ac490 con 0x560034b49ff0 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41abdc0 con 0x560034b551d0 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac41a9000 con 0x560034b4a910 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0acc8f6740 10 client.0 _ll_register_callbacks cb 0x7f0ab4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52874/0 (socket says 172.21.15.103:52874) 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0ac3fff640 1 -- 172.21.15.103:0/2736252713 learned_addr learned my addr 172.21.15.103:0/2736252713 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.206+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b4a910 0x560034b61b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b551d0 0x560034b4e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac3fff640 1 -- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b4a910 msgr2=0x560034b61b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b4a910 0x560034b61b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac3fff640 1 -- 172.21.15.103:0/2736252713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b551d0 msgr2=0x560034b4e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b551d0 0x560034b4e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac3fff640 1 -- 172.21.15.103:0/2736252713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560034b4a740 con 0x560034b49ff0 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b4a910 0x560034b61b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac8a95640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b551d0 0x560034b4e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0ac3fff640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b615d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4011f50 tx=0x7f0ab4032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4e24f96984bb29 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0aa37fe640 1 -- 172.21.15.103:0/2736252713 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4036070 con 0x560034b49ff0 2024-08-19T01:42:30.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0abc050070 con 0x560034b49ff0 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0aa37fe640 1 -- 172.21.15.103:0/2736252713 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0ab4032ba0 con 0x560034b49ff0 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0aa37fe640 1 -- 172.21.15.103:0/2736252713 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0ab4042050 con 0x560034b49ff0 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0acc8f6740 10 client.17133 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.207+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560034bd3120 con 0x560034b49ff0 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0aa37fe640 1 -- 172.21.15.103:0/2736252713 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0abc050070 con 0x560034b49ff0 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0aa37fe640 1 -- 172.21.15.103:0/2736252713 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab40115b0 con 0x560034b49ff0 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0acc8f6740 10 client.17133 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0acc8f6740 10 client.17133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0acc8f6740 20 client.17133 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034bd3810 0x560034bd5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0acc8f6740 4 client.17133 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560034ab74c0 con 0x560034bd3810 2024-08-19T01:42:30.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.208+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034bd3810 0x560034bd5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0ac37fe640 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034bd3810 0x560034bd5c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6eeb6d2e8918a6a server_cookie=10d38dba9eadba52 in_seq=0 out_seq=0 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0aa37fe640 10 client.17133 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0aa37fe640 1 -- 172.21.15.103:0/2736252713 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560034ab74c0 con 0x560034bd3810 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0aa37fe640 10 client.17133 handle_command_reply: tid=0 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 client.17133 shutdown 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 msgr2=0x560034b615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b615d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4011f50 tx=0x7f0ab4032590 comp rx=0 tx=0).stop 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 shutdown_connections 2024-08-19T01:42:30.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x560034bd3810 0x560034bd5c30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560034b551d0 0x560034b4e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560034b4a910 0x560034b61b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 --2- 172.21.15.103:0/2736252713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560034b49ff0 0x560034b615d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 >> 172.21.15.103:0/2736252713 conn(0x560034b46580 msgr2=0x560034b46960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0aa37fe640 0 client.17133 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 shutdown_connections 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 1 -- 172.21.15.103:0/2736252713 wait complete. 2024-08-19T01:42:30.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.209+0000 7f0acc8f6740 20 client.17133 trim_cache size 0 max 16384 2024-08-19T01:42:30.219 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:42:30.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0aac0a2b80 msgr2=0x7f0aac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0aac0a2b80 0x7f0aac0a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0319e0 tx=0x7f0abc009860 comp rx=0 tx=0).stop 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4101000 msgr2=0x7f0ac4198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4101000 0x7f0ac4198610 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b4a0 tx=0x7f0ab800b960 comp rx=0 tx=0).stop 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 shutdown_connections 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0aac0a2b80 0x7f0aac0a5040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ac4107e80 0x7f0ac419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ac4101000 0x7f0ac4198610 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 --2- 172.21.15.103:0/1753871646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ac4100650 0x7f0ac41980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 >> 172.21.15.103:0/1753871646 conn(0x7f0ac4075ca0 msgr2=0x7f0ac4105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 shutdown_connections 2024-08-19T01:42:30.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:30.210+0000 7f0aca4f8640 1 -- 172.21.15.103:0/1753871646 wait complete. 2024-08-19T01:42:31.034 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:42:31.058 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:31 smithi103 ceph-mon[18436]: pgmap v541: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:31.338 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:31 smithi190 ceph-mon[22032]: pgmap v541: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:31.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:31 smithi146 ceph-mon[22098]: pgmap v541: 97 pgs: 97 active+clean; 9.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:42:31.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 -- 172.21.15.103:0/4266835698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 msgr2=0x7fd630107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 --2- 172.21.15.103:0/4266835698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd630107c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009930 tx=0x7fd62c02a420 comp rx=0 tx=0).stop 2024-08-19T01:42:31.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 -- 172.21.15.103:0/4266835698 shutdown_connections 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 --2- 172.21.15.103:0/4266835698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd630102240 0x7fd63010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 --2- 172.21.15.103:0/4266835698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6301018a0 0x7fd630101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 --2- 172.21.15.103:0/4266835698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd630107c80 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 -- 172.21.15.103:0/4266835698 >> 172.21.15.103:0/4266835698 conn(0x7fd6300fd5e0 msgr2=0x7fd6300ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 -- 172.21.15.103:0/4266835698 shutdown_connections 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.877+0000 7fd637d33640 1 -- 172.21.15.103:0/4266835698 wait complete. 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 Processor -- start 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 -- start start 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6301018a0 0x7fd630198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd630102240 0x7fd6301986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd63019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd630111b90 con 0x7fd6301078a0 2024-08-19T01:42:31.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd630111a10 con 0x7fd630102240 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd637d33640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd630111d10 con 0x7fd6301018a0 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd6362d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd63019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd6362d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd63019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59326/0 (socket says 172.21.15.103:59326) 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd6362d0640 1 -- 172.21.15.103:0/2257697326 learned_addr learned my addr 172.21.15.103:0/2257697326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.878+0000 7fd635acf640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6301018a0 0x7fd630198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd6352ce640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd630102240 0x7fd6301986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd6362d0640 1 -- 172.21.15.103:0/2257697326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6301018a0 msgr2=0x7fd630198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd6362d0640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6301018a0 0x7fd630198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd6362d0640 1 -- 172.21.15.103:0/2257697326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd630102240 msgr2=0x7fd6301986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd6362d0640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd630102240 0x7fd6301986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd6362d0640 1 -- 172.21.15.103:0/2257697326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63019d140 con 0x7fd6301078a0 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd6362d0640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd63019ca60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd62800cad0 tx=0x7fd62800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd626ffd640 1 -- 172.21.15.103:0/2257697326 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62800ae20 con 0x7fd6301078a0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63019d430 con 0x7fd6301078a0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd626ffd640 1 -- 172.21.15.103:0/2257697326 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd628005020 con 0x7fd6301078a0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd626ffd640 1 -- 172.21.15.103:0/2257697326 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62800f6b0 con 0x7fd6301078a0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.879+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301a4d10 con 0x7fd6301078a0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.882+0000 7fd626ffd640 1 -- 172.21.15.103:0/2257697326 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd6280051d0 con 0x7fd6301078a0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.882+0000 7fd63a131740 1 Processor -- start 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd63a131740 1 -- start start 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd63a131740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd63a131740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd60c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd63a131740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd63a131740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635cbcbba40 con 0x5635cbd56910 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd63a131740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5635cbc539d0 con 0x5635cbd611d0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd63a131740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635cbcb7a00 con 0x5635cbd55ff0 2024-08-19T01:42:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd635acf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd60c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd635acf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd60c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59330/0 (socket says 172.21.15.103:59330) 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd635acf640 1 -- 172.21.15.103:0/1045821395 learned_addr learned my addr 172.21.15.103:0/1045821395 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd6352ce640 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd6352ce640 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd626ffd640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd61c0a2b80 0x7fd61c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd6352ce640 1 -- 172.21.15.103:0/1045821395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 msgr2=0x5635cbd635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd6352ce640 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.883+0000 7fd6352ce640 1 -- 172.21.15.103:0/1045821395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 msgr2=0x5635cbd60c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd6352ce640 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd60c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd6352ce640 1 -- 172.21.15.103:0/1045821395 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635cbc5a650 con 0x5635cbd55ff0 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd626ffd640 1 -- 172.21.15.103:0/2257697326 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd6280cf090 con 0x7fd6301078a0 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd635acf640 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd60c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd6362d0640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd61c0a2b80 0x7fd61c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd6352ce640 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd563d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd6300ffd10 tx=0x7fd62000a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5ddab7700f8a74 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd61bfff640 1 -- 172.21.15.103:0/1045821395 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620016dc0 con 0x5635cbd55ff0 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd61bfff640 1 -- 172.21.15.103:0/1045821395 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6200188e0 con 0x5635cbd55ff0 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.884+0000 7fd61bfff640 1 -- 172.21.15.103:0/1045821395 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6200177d0 con 0x5635cbd55ff0 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 -- 172.21.15.103:0/1045821395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 msgr2=0x5635cbd563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd6362d0640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd61c0a2b80 0x7fd61c0a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd628007850 tx=0x7fd6280077e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd563d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd6300ffd10 tx=0x7fd62000a230 comp rx=0 tx=0).stop 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 -- 172.21.15.103:0/1045821395 shutdown_connections 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd60c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 --2- 172.21.15.103:0/1045821395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd563d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 -- 172.21.15.103:0/1045821395 >> 172.21.15.103:0/1045821395 conn(0x5635cbd52580 msgr2=0x5635cbd52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 -- 172.21.15.103:0/1045821395 shutdown_connections 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 -- 172.21.15.103:0/1045821395 wait complete. 2024-08-19T01:42:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 -- start start 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.885+0000 7fd63a131740 1 Processor -- start 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd63a131740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd55ff0 0x5635cbd5e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd63a131740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd56910 0x5635cbd5e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd63a131740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd63a131740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635cbd63da0 con 0x5635cbd55ff0 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd63a131740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5635cbc76b20 con 0x5635cbd611d0 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd63a131740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5635cbd63f20 con 0x5635cbd56910 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd63a131740 10 client.0 _ll_register_callbacks cb 0x5635cbc74bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd55ff0 0x5635cbd5e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd55ff0 0x5635cbd5e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59336/0 (socket says 172.21.15.103:59336) 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 -- 172.21.15.103:0/3947497697 learned_addr learned my addr 172.21.15.103:0/3947497697 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd635acf640 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd56910 0x5635cbd5e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 -- 172.21.15.103:0/3947497697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd56910 msgr2=0x5635cbd5e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd56910 0x5635cbd5e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 -- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 msgr2=0x5635cbd5eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.886+0000 7fd6352ce640 1 -- 172.21.15.103:0/3947497697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635cba2d3c0 con 0x5635cbd55ff0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd55ff0 0x5635cbd5e0c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd620016ff0 tx=0x7fd620006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fcf9e39b00f299 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd6197fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd6197fa640 1 -- 172.21.15.103:0/3947497697 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620018a90 con 0x5635cbd55ff0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd6197fa640 1 -- 172.21.15.103:0/3947497697 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd620018c30 con 0x5635cbd55ff0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6200188e0 con 0x5635cbd55ff0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd6197fa640 1 -- 172.21.15.103:0/3947497697 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd620018e30 con 0x5635cbd55ff0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd63a131740 10 client.17145 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5635cbd6f9b0 con 0x5635cbd55ff0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd6197fa640 1 -- 172.21.15.103:0/3947497697 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6200188e0 con 0x5635cbd55ff0 2024-08-19T01:42:31.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd6197fa640 1 -- 172.21.15.103:0/3947497697 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd620024040 con 0x5635cbd55ff0 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd63a131740 10 client.17145 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.887+0000 7fd63a131740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.888+0000 7fd63a131740 20 client.17145 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.888+0000 7fd63a131740 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbd70d00 0x5635cbd730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.888+0000 7fd63a131740 4 client.17145 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.888+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635cbcc8fb0 con 0x5635cbd70d00 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.888+0000 7fd635acf640 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbd70d00 0x5635cbd730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.888+0000 7fd635acf640 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbd70d00 0x5635cbd730f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793ee3eabad0e78b server_cookie=43b43a90cc122b50 in_seq=0 out_seq=0 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.888+0000 7fd6197fa640 10 client.17145 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd6197fa640 1 -- 172.21.15.103:0/3947497697 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635cbcc8fb0 con 0x5635cbd70d00 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd6197fa640 10 client.17145 handle_command_reply: tid=0 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 client.17145 shutdown 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd55ff0 msgr2=0x5635cbd5e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd55ff0 0x5635cbd5e0c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd620016ff0 tx=0x7fd620006a30 comp rx=0 tx=0).stop 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 shutdown_connections 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbd70d00 0x5635cbd730f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd6197fa640 0 client.17145 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd56910 0x5635cbd5e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 --2- 172.21.15.103:0/3947497697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd55ff0 0x5635cbd5e0c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 >> 172.21.15.103:0/3947497697 conn(0x5635cbd52580 msgr2=0x5635cbd582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 shutdown_connections 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 1 -- 172.21.15.103:0/3947497697 wait complete. 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.889+0000 7fd63a131740 20 client.17145 trim_cache size 0 max 16384 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 Processor -- start 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 -- start start 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd55ff0 0x5635cbd5d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd78690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd611d0 0x5635cbd78bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ad280 con 0x5635cbd56910 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301aedc0 con 0x5635cbd55ff0 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd63a131740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ad950 con 0x5635cbd611d0 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd635acf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd78690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd635acf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd78690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59342/0 (socket says 172.21.15.103:59342) 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd635acf640 1 -- 172.21.15.103:0/3134150037 learned_addr learned my addr 172.21.15.103:0/3134150037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.904+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd55ff0 0x5635cbd5d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd611d0 0x5635cbd78bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd635acf640 1 -- 172.21.15.103:0/3134150037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd611d0 msgr2=0x5635cbd78bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd635acf640 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd611d0 0x5635cbd78bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd635acf640 1 -- 172.21.15.103:0/3134150037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd55ff0 msgr2=0x5635cbd5d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd635acf640 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd55ff0 0x5635cbd5d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd635acf640 1 -- 172.21.15.103:0/3134150037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635cbd53210 con 0x5635cbd56910 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd611d0 0x5635cbd78bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd635acf640 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd78690 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd62c02a940 tx=0x7fd62c02aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6051737205a05fff server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd61b7fe640 1 -- 172.21.15.103:0/3134150037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c035040 con 0x5635cbd56910 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd61b7fe640 1 -- 172.21.15.103:0/3134150037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd62c004380 con 0x5635cbd56910 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.905+0000 7fd61b7fe640 1 -- 172.21.15.103:0/3134150037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0035f0 con 0x5635cbd56910 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 -- 172.21.15.103:0/3134150037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 msgr2=0x5635cbd78690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd78690 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd62c02a940 tx=0x7fd62c02aae0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 -- 172.21.15.103:0/3134150037 shutdown_connections 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd611d0 0x5635cbd78bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd56910 0x5635cbd78690 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 --2- 172.21.15.103:0/3134150037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd55ff0 0x5635cbd5d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 -- 172.21.15.103:0/3134150037 >> 172.21.15.103:0/3134150037 conn(0x5635cbd52580 msgr2=0x5635cbd515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 -- 172.21.15.103:0/3134150037 shutdown_connections 2024-08-19T01:42:31.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 -- 172.21.15.103:0/3134150037 wait complete. 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 -- start start 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.906+0000 7fd63a131740 1 Processor -- start 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd63a131740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd69570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd63a131740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd63a131740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd5af90 0x5635cbd5b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd63a131740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ae6f0 con 0x5635cbd5af90 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd63a131740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ae020 con 0x5635cbd611d0 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd63a131740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ab260 con 0x5635cbd55ff0 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd63a131740 10 client.0 _ll_register_callbacks cb 0x7fd62c005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd6352ce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd5af90 0x5635cbd5b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd6352ce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd5af90 0x5635cbd5b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59358/0 (socket says 172.21.15.103:59358) 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd6352ce640 1 -- 172.21.15.103:0/3545784396 learned_addr learned my addr 172.21.15.103:0/3545784396 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd635acf640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd69570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd635acf640 1 -- 172.21.15.103:0/3545784396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 msgr2=0x5635cbd69570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd635acf640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd69570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd635acf640 1 -- 172.21.15.103:0/3545784396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd5af90 msgr2=0x5635cbd5b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd635acf640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd5af90 0x5635cbd5b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd635acf640 1 -- 172.21.15.103:0/3545784396 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635cbd69e10 con 0x5635cbd611d0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.907+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd69570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd635acf640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5aa50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5635cbd522a0 tx=0x7fd62c032f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b676b0f47adf0c87 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd6197fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd6197fa640 1 -- 172.21.15.103:0/3545784396 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c058070 con 0x5635cbd611d0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62c004380 con 0x5635cbd611d0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd6197fa640 1 -- 172.21.15.103:0/3545784396 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd62c0037f0 con 0x5635cbd611d0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd6197fa640 1 -- 172.21.15.103:0/3545784396 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd62c064050 con 0x5635cbd611d0 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd63a131740 10 client.26456 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:31.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5635cbddb6c0 con 0x5635cbd611d0 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd6197fa640 1 -- 172.21.15.103:0/3545784396 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd62c004380 con 0x5635cbd611d0 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.908+0000 7fd6197fa640 1 -- 172.21.15.103:0/3545784396 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd62c0324d0 con 0x5635cbd611d0 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 10 client.26456 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 20 client.26456 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbddbd30 0x5635cbdde120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 4 client.26456 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635cbcc34c0 con 0x5635cbddbd30 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbddbd30 0x5635cbdde120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd6197fa640 10 client.26456 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd6352ce640 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbddbd30 0x5635cbdde120 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc3d06ea244d863 server_cookie=3b81031037dd149b in_seq=0 out_seq=0 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd6197fa640 1 -- 172.21.15.103:0/3545784396 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635cbcc34c0 con 0x5635cbddbd30 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd6197fa640 10 client.26456 handle_command_reply: tid=0 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 1 client.26456 shutdown 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 msgr2=0x5635cbd5aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.909+0000 7fd63a131740 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5aa50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5635cbd522a0 tx=0x7fd62c032f10 comp rx=0 tx=0).stop 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 shutdown_connections 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5635cbddbd30 0x5635cbdde120 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635cbd5af90 0x5635cbd5b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5635cbd611d0 0x5635cbd5aa50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 --2- 172.21.15.103:0/3545784396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5635cbd55ff0 0x5635cbd69570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 >> 172.21.15.103:0/3545784396 conn(0x5635cbd52580 msgr2=0x5635cbd5f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd6197fa640 0 client.26456 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 shutdown_connections 2024-08-19T01:42:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 1 -- 172.21.15.103:0/3545784396 wait complete. 2024-08-19T01:42:31.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd63a131740 20 client.26456 trim_cache size 0 max 16384 2024-08-19T01:42:31.928 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd61c0a2b80 msgr2=0x7fd61c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd637d33640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd61c0a2b80 0x7fd61c0a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd628007850 tx=0x7fd6280077e0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 msgr2=0x7fd63019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.910+0000 7fd637d33640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd63019ca60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fd62800cad0 tx=0x7fd62800cf90 comp rx=0 tx=0).stop 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 shutdown_connections 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd61c0a2b80 0x7fd61c0a5040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6301078a0 0x7fd63019ca60 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd630102240 0x7fd6301986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 --2- 172.21.15.103:0/2257697326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6301018a0 0x7fd630198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 >> 172.21.15.103:0/2257697326 conn(0x7fd6300fd5e0 msgr2=0x7fd6300fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 shutdown_connections 2024-08-19T01:42:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:31.911+0000 7fd637d33640 1 -- 172.21.15.103:0/2257697326 wait complete. 2024-08-19T01:42:33.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:33 smithi103 ceph-mon[18436]: pgmap v542: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:42:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:33 smithi190 ceph-mon[22032]: pgmap v542: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:42:33.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:33 smithi146 ceph-mon[22098]: pgmap v542: 97 pgs: 97 active+clean; 9.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:42:33.783 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:42:33.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:42:34.097 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 -- 172.21.15.103:0/3125822329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0238100b50 msgr2=0x7f0238100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 --2- 172.21.15.103:0/3125822329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0238100b50 0x7f0238100f30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0234009900 tx=0x7f023402a480 comp rx=0 tx=0).stop 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 -- 172.21.15.103:0/3125822329 shutdown_connections 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 --2- 172.21.15.103:0/3125822329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0238102c20 0x7f023810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 --2- 172.21.15.103:0/3125822329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02381022a0 0x7f02381026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 --2- 172.21.15.103:0/3125822329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0238100b50 0x7f0238100f30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 -- 172.21.15.103:0/3125822329 >> 172.21.15.103:0/3125822329 conn(0x7f02380fc500 msgr2=0x7f02380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 -- 172.21.15.103:0/3125822329 shutdown_connections 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.639+0000 7f0240535640 1 -- 172.21.15.103:0/3125822329 wait complete. 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 Processor -- start 2024-08-19T01:42:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 -- start start 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0238100b50 0x7f0238198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02381022a0 0x7f0238199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0238102c20 0x7f02381a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0238110b60 con 0x7f02381022a0 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02381109e0 con 0x7f0238102c20 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f0240535640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0238110ce0 con 0x7f0238100b50 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f023dad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02381022a0 0x7f0238199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f023dad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02381022a0 0x7f0238199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59370/0 (socket says 172.21.15.103:59370) 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f023dad0640 1 -- 172.21.15.103:0/567582114 learned_addr learned my addr 172.21.15.103:0/567582114 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f023ead2640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0238102c20 0x7f02381a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.640+0000 7f023e2d1640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0238100b50 0x7f0238198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f023dad0640 1 -- 172.21.15.103:0/567582114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0238100b50 msgr2=0x7f0238198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f023dad0640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0238100b50 0x7f0238198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f023dad0640 1 -- 172.21.15.103:0/567582114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0238102c20 msgr2=0x7f02381a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f023dad0640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0238102c20 0x7f02381a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f023dad0640 1 -- 172.21.15.103:0/567582114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02381a45b0 con 0x7f02381022a0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f023ead2640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0238102c20 0x7f02381a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f023dad0640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02381022a0 0x7f0238199530 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f022800bd30 tx=0x7f0228009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f022f7fe640 1 -- 172.21.15.103:0/567582114 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228004220 con 0x7f02381022a0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f022f7fe640 1 -- 172.21.15.103:0/567582114 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02280043c0 con 0x7f02381022a0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0238196de0 con 0x7f02381022a0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f022f7fe640 1 -- 172.21.15.103:0/567582114 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280047a0 con 0x7f02381022a0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.641+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0238197320 con 0x7f02381022a0 2024-08-19T01:42:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.644+0000 7f022f7fe640 1 -- 172.21.15.103:0/567582114 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0228011050 con 0x7f02381022a0 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.644+0000 7f0242933740 1 Processor -- start 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f0242933740 1 -- start start 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f0242933740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f0242933740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a00fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f0242933740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00fc3e0 0x5587a00fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f0242933740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587a00b4e40 con 0x5587a00fc3e0 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f0242933740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5587a00567f0 con 0x5587a00f1200 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f0242933740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55879fffdfb0 con 0x5587a00f1b20 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f023ead2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f023ead2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:41250/0 (socket says 172.21.15.103:41250) 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f023ead2640 1 -- 172.21.15.103:0/3631394610 learned_addr learned my addr 172.21.15.103:0/3631394610 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f023e2d1640 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a00fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.645+0000 7f023dad0640 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00fc3e0 0x5587a00fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f022f7fe640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02240a2c00 0x7f02240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023ead2640 1 -- 172.21.15.103:0/3631394610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 msgr2=0x5587a00fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023ead2640 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a00fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023ead2640 1 -- 172.21.15.103:0/3631394610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00fc3e0 msgr2=0x5587a00fe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023ead2640 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00fc3e0 0x5587a00fe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023ead2640 1 -- 172.21.15.103:0/3631394610 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879fdc86a0 con 0x5587a00f1200 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023dad0640 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00fc3e0 0x5587a00fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f022f7fe640 1 -- 172.21.15.103:0/567582114 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f02280d28f0 con 0x7f02381022a0 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023ead2640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02240a2c00 0x7f02240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.646+0000 7f023ead2640 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f15e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0230010710 tx=0x7f0230010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce107547fcec863 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.647+0000 7f023ead2640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02240a2c00 0x7f02240a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0230014000 tx=0x7f0230015000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.647+0000 7f022cff9640 1 -- 172.21.15.103:0/3631394610 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230012aa0 con 0x5587a00f1200 2024-08-19T01:42:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.647+0000 7f022cff9640 1 -- 172.21.15.103:0/3631394610 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0230012c40 con 0x5587a00f1200 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.647+0000 7f022cff9640 1 -- 172.21.15.103:0/3631394610 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02300113e0 con 0x5587a00f1200 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 -- 172.21.15.103:0/3631394610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 msgr2=0x5587a00f15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f15e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0230010710 tx=0x7f0230010b10 comp rx=0 tx=0).stop 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 -- 172.21.15.103:0/3631394610 shutdown_connections 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00fc3e0 0x5587a00fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 --2- 172.21.15.103:0/3631394610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f15e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 -- 172.21.15.103:0/3631394610 >> 172.21.15.103:0/3631394610 conn(0x5587a00ed790 msgr2=0x5587a00edb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 -- 172.21.15.103:0/3631394610 shutdown_connections 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 -- 172.21.15.103:0/3631394610 wait complete. 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.648+0000 7f0242933740 1 -- start start 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 1 Processor -- start 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a00f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 0x5587a00f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587a0011cb0 con 0x5587a00f1b20 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5587a00527b0 con 0x5587a00f1200 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5587a00ff050 con 0x5587a00fc3e0 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f0242933740 10 client.0 _ll_register_callbacks cb 0x55879fd65ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f023dad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a00f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f023dad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a00f9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59396/0 (socket says 172.21.15.103:59396) 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f023dad0640 1 -- 172.21.15.103:0/2087949383 learned_addr learned my addr 172.21.15.103:0/2087949383 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f023e2d1640 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f023dad0640 1 -- 172.21.15.103:0/2087949383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 msgr2=0x5587a00f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.649+0000 7f023dad0640 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 0x5587a00f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f023dad0640 1 -- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 msgr2=0x5587a00f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f023dad0640 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f023dad0640 1 -- 172.21.15.103:0/2087949383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a0010bc0 con 0x5587a00f1b20 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f023e2d1640 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f023dad0640 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a00f9860 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f02280d0420 tx=0x7f022809b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ce90f679e4f27 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0221ffb640 1 -- 172.21.15.103:0/2087949383 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228020040 con 0x5587a00f1b20 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0230012c40 con 0x5587a00f1b20 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0221ffb640 1 -- 172.21.15.103:0/2087949383 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0228011340 con 0x5587a00f1b20 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0221ffb640 1 -- 172.21.15.103:0/2087949383 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f022809eb90 con 0x5587a00f1b20 2024-08-19T01:42:34.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0242933740 10 client.17163 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5587a0113820 con 0x5587a00f1b20 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.650+0000 7f0221ffb640 1 -- 172.21.15.103:0/2087949383 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0230012c40 con 0x5587a00f1b20 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.651+0000 7f0221ffb640 1 -- 172.21.15.103:0/2087949383 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f02280a7960 con 0x5587a00f1b20 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.651+0000 7f0242933740 10 client.17163 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.651+0000 7f0242933740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.651+0000 7f0242933740 20 client.17163 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.651+0000 7f0242933740 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a01151f0 0x5587a01176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.651+0000 7f0242933740 4 client.17163 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.651+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a0063d60 con 0x5587a01151f0 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.652+0000 7f023e2d1640 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a01151f0 0x5587a01176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.652+0000 7f023e2d1640 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a01151f0 0x5587a01176b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e6c27e163c718d server_cookie=cce400e8e662e102 in_seq=0 out_seq=0 2024-08-19T01:42:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.652+0000 7f0221ffb640 10 client.17163 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0221ffb640 1 -- 172.21.15.103:0/2087949383 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587a0063d60 con 0x5587a01151f0 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0221ffb640 10 client.17163 handle_command_reply: tid=0 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 client.17163 shutdown 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 msgr2=0x5587a00f9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a00f9860 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f02280d0420 tx=0x7f022809b4d0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 shutdown_connections 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a01151f0 0x5587a01176b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 0x5587a00f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a00f9860 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 --2- 172.21.15.103:0/2087949383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a00f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 >> 172.21.15.103:0/2087949383 conn(0x5587a00ed790 msgr2=0x5587a00f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0221ffb640 0 client.17163 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 shutdown_connections 2024-08-19T01:42:34.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 1 -- 172.21.15.103:0/2087949383 wait complete. 2024-08-19T01:42:34.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.653+0000 7f0242933740 20 client.17163 trim_cache size 0 max 16384 2024-08-19T01:42:34.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 Processor -- start 2024-08-19T01:42:34.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 -- start start 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a011cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a011d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 0x5587a00ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02381aae60 con 0x5587a00f1b20 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02381ac9a0 con 0x5587a00f1200 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f0242933740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f02381ab530 con 0x5587a00fc3e0 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.674+0000 7f023dad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a011d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a011d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59398/0 (socket says 172.21.15.103:59398) 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 -- 172.21.15.103:0/3774041860 learned_addr learned my addr 172.21.15.103:0/3774041860 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:34.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023e2d1640 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a011cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023ead2640 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 0x5587a00ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 -- 172.21.15.103:0/3774041860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 msgr2=0x5587a00ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 0x5587a00ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 -- 172.21.15.103:0/3774041860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 msgr2=0x5587a011cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a011cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 -- 172.21.15.103:0/3774041860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a00f78b0 con 0x5587a00f1b20 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023e2d1640 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a011cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f023dad0640 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a011d500 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0228009b10 tx=0x7f022809d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abded71eb685609 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f0223fff640 1 -- 172.21.15.103:0/3774041860 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022809f000 con 0x5587a00f1b20 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f0223fff640 1 -- 172.21.15.103:0/3774041860 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f022809f230 con 0x5587a00f1b20 2024-08-19T01:42:34.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.675+0000 7f0223fff640 1 -- 172.21.15.103:0/3774041860 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228011340 con 0x5587a00f1b20 2024-08-19T01:42:34.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 -- 172.21.15.103:0/3774041860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 msgr2=0x5587a011d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a011d500 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0228009b10 tx=0x7f022809d740 comp rx=0 tx=0).stop 2024-08-19T01:42:34.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 -- 172.21.15.103:0/3774041860 shutdown_connections 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00fc3e0 0x5587a00ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1b20 0x5587a011d500 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 --2- 172.21.15.103:0/3774041860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00f1200 0x5587a011cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 -- 172.21.15.103:0/3774041860 >> 172.21.15.103:0/3774041860 conn(0x5587a00ed790 msgr2=0x5587a00f7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 -- 172.21.15.103:0/3774041860 shutdown_connections 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.676+0000 7f0242933740 1 -- 172.21.15.103:0/3774041860 wait complete. 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 -- start start 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 Processor -- start 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1200 0x5587a010cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a010d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00fc3e0 0x5587a00f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02381ac2d0 con 0x5587a00f1200 2024-08-19T01:42:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f02381abc00 con 0x5587a00fc3e0 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f02381a8e40 con 0x5587a00f1b20 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f0242933740 10 client.0 _ll_register_callbacks cb 0x7f0228005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f023e2d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1200 0x5587a010cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f023e2d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1200 0x5587a010cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59406/0 (socket says 172.21.15.103:59406) 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f023e2d1640 1 -- 172.21.15.103:0/1198295683 learned_addr learned my addr 172.21.15.103:0/1198295683 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.677+0000 7f023ead2640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00fc3e0 0x5587a00f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023dad0640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a010d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023e2d1640 1 -- 172.21.15.103:0/1198295683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 msgr2=0x5587a010d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023e2d1640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a010d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023e2d1640 1 -- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00fc3e0 msgr2=0x5587a00f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023e2d1640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00fc3e0 0x5587a00f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023e2d1640 1 -- 172.21.15.103:0/1198295683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a00f6450 con 0x5587a00f1200 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023dad0640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a010d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f023e2d1640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1200 0x5587a010cd80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0234009230 tx=0x7f0234004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44d10d52de7366a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f0221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f0221ffb640 1 -- 172.21.15.103:0/1198295683 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234049030 con 0x5587a00f1200 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f0221ffb640 1 -- 172.21.15.103:0/1198295683 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0234031e00 con 0x5587a00f1200 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022809f230 con 0x5587a00f1200 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f0221ffb640 1 -- 172.21.15.103:0/1198295683 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0234035040 con 0x5587a00f1200 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f0242933740 10 client.17169 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.678+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5587a017fad0 con 0x5587a00f1200 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0221ffb640 1 -- 172.21.15.103:0/1198295683 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f022809f230 con 0x5587a00f1200 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0221ffb640 1 -- 172.21.15.103:0/1198295683 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f02340047a0 con 0x5587a00f1200 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0242933740 10 client.17169 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0242933740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0242933740 20 client.17169 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0242933740 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a0180270 0x5587a0182690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0242933740 4 client.17169 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:42:34.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587a005e270 con 0x5587a0180270 2024-08-19T01:42:34.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f023dad0640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a0180270 0x5587a0182690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:34.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f023dad0640 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a0180270 0x5587a0182690 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879585c634af53da server_cookie=22d7b517f4060654 in_seq=0 out_seq=0 2024-08-19T01:42:34.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.679+0000 7f0221ffb640 10 client.17169 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:34.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0221ffb640 1 -- 172.21.15.103:0/1198295683 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587a005e270 con 0x5587a0180270 2024-08-19T01:42:34.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0221ffb640 10 client.17169 handle_command_reply: tid=0 2024-08-19T01:42:34.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 client.17169 shutdown 2024-08-19T01:42:34.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1200 msgr2=0x5587a010cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1200 0x5587a010cd80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0234009230 tx=0x7f0234004330 comp rx=0 tx=0).stop 2024-08-19T01:42:34.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 shutdown_connections 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5587a0180270 0x5587a0182690 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5587a00fc3e0 0x5587a00f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5587a00f1b20 0x5587a010d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 --2- 172.21.15.103:0/1198295683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587a00f1200 0x5587a010cd80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 >> 172.21.15.103:0/1198295683 conn(0x5587a00ed790 msgr2=0x5587a00f2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0221ffb640 0 client.17169 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 shutdown_connections 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 1 -- 172.21.15.103:0/1198295683 wait complete. 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.681+0000 7f0242933740 20 client.17169 trim_cache size 0 max 16384 2024-08-19T01:42:34.687 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:42:34.688 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:42:34.688 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "dc06bd78-2122-465c-b572-b5393e857aa3", 2024-08-19T01:42:34.688 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:42:34.688 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:42:34.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02240a2c00 msgr2=0x7f02240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02240a2c00 0x7f02240a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0230014000 tx=0x7f0230015000 comp rx=0 tx=0).stop 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02381022a0 msgr2=0x7f0238199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02381022a0 0x7f0238199530 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f022800bd30 tx=0x7f0228009610 comp rx=0 tx=0).stop 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 shutdown_connections 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f02240a2c00 0x7f02240a50c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0238102c20 0x7f02381a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02381022a0 0x7f0238199530 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 --2- 172.21.15.103:0/567582114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0238100b50 0x7f0238198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 >> 172.21.15.103:0/567582114 conn(0x7f02380fc500 msgr2=0x7f02380fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 shutdown_connections 2024-08-19T01:42:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:34.682+0000 7f0240535640 1 -- 172.21.15.103:0/567582114 wait complete. 2024-08-19T01:42:35.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:35 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:42:35.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:35 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:35.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:35 smithi103 ceph-mon[18436]: pgmap v543: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:42:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:35 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:42:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:35 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:35 smithi190 ceph-mon[22032]: pgmap v543: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:42:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:35 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:42:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:35 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:35.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:35 smithi146 ceph-mon[22098]: pgmap v543: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:42:35.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:42:35.835 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:42:36.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:42:36.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.280 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:42:36.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:42:36.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:42:36.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:36 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:42:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:42:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:42:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:42:36.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:36 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 -- 172.21.15.103:0/1714139071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781018c0 msgr2=0x7fa278101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 --2- 172.21.15.103:0/1714139071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781018c0 0x7fa278101d20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa268009930 tx=0x7fa26802a420 comp rx=0 tx=0).stop 2024-08-19T01:42:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 -- 172.21.15.103:0/1714139071 shutdown_connections 2024-08-19T01:42:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 --2- 172.21.15.103:0/1714139071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa278102260 0x7fa27810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 --2- 172.21.15.103:0/1714139071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781018c0 0x7fa278101d20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 --2- 172.21.15.103:0/1714139071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2781078c0 0x7fa278107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 -- 172.21.15.103:0/1714139071 >> 172.21.15.103:0/1714139071 conn(0x7fa2780fd620 msgr2=0x7fa2780ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.374+0000 7fa280357640 1 -- 172.21.15.103:0/1714139071 shutdown_connections 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 -- 172.21.15.103:0/1714139071 wait complete. 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 Processor -- start 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 -- start start 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2781018c0 0x7fa27819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278102260 0x7fa27819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781078c0 0x7fa27819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa278111b70 con 0x7fa2781078c0 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781119f0 con 0x7fa2781018c0 2024-08-19T01:42:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.375+0000 7fa280357640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa278111cf0 con 0x7fa278102260 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781078c0 0x7fa27819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781078c0 0x7fa27819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59412/0 (socket says 172.21.15.103:59412) 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 -- 172.21.15.103:0/3690217426 learned_addr learned my addr 172.21.15.103:0/3690217426 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2781018c0 0x7fa27819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278102260 0x7fa27819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 -- 172.21.15.103:0/3690217426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278102260 msgr2=0x7fa27819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278102260 0x7fa27819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 -- 172.21.15.103:0/3690217426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2781018c0 msgr2=0x7fa27819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2781018c0 0x7fa27819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 -- 172.21.15.103:0/3690217426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27819f2e0 con 0x7fa2781078c0 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa27e8f4640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781078c0 0x7fa27819ec00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa27000eaa0 tx=0x7fa27000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa26f7fe640 1 -- 172.21.15.103:0/3690217426 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27000cdb0 con 0x7fa2781078c0 2024-08-19T01:42:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa26f7fe640 1 -- 172.21.15.103:0/3690217426 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa270005020 con 0x7fa2781078c0 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27819f5d0 con 0x7fa2781078c0 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.376+0000 7fa26f7fe640 1 -- 172.21.15.103:0/3690217426 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2700116d0 con 0x7fa2781078c0 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.377+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781a6eb0 con 0x7fa2781078c0 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.379+0000 7fa26f7fe640 1 -- 172.21.15.103:0/3690217426 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa2700051d0 con 0x7fa2781078c0 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.379+0000 7fa282755740 1 Processor -- start 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.379+0000 7fa282755740 1 -- start start 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa282755740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219c210 0x55bde219c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa282755740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219cb30 0x55bde21a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa282755740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21a73f0 0x55bde21a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa282755740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bde2101980 con 0x55bde219cb30 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa282755740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bde2099a70 con 0x55bde21a73f0 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa282755740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bde20fd940 con 0x55bde219c210 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219cb30 0x55bde21a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219cb30 0x55bde21a6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59426/0 (socket says 172.21.15.103:59426) 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 -- 172.21.15.103:0/3431679826 learned_addr learned my addr 172.21.15.103:0/3431679826 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219c210 0x55bde219c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa26f7fe640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa2640a2c00 0x7fa2640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21a73f0 0x55bde21a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 -- 172.21.15.103:0/3431679826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219c210 msgr2=0x55bde219c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219c210 0x55bde219c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 -- 172.21.15.103:0/3431679826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21a73f0 msgr2=0x55bde21a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21a73f0 0x55bde21a97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e0f3640 1 -- 172.21.15.103:0/3431679826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde20fc6e0 con 0x55bde219cb30 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa27e8f4640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa2640a2c00 0x7fa2640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.380+0000 7fa26f7fe640 1 -- 172.21.15.103:0/3690217426 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa2700cee50 con 0x7fa2781078c0 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.381+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21a73f0 0x55bde21a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:36.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.381+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219cb30 0x55bde21a6eb0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa27400bc90 tx=0x7fa2740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4f231b6dd7f99c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.381+0000 7fa26cdea640 1 -- 172.21.15.103:0/3431679826 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27401c480 con 0x55bde219cb30 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.381+0000 7fa26cdea640 1 -- 172.21.15.103:0/3431679826 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa27401cad0 con 0x55bde219cb30 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.381+0000 7fa26cdea640 1 -- 172.21.15.103:0/3431679826 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274011b50 con 0x55bde219cb30 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.381+0000 7fa27e8f4640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa2640a2c00 0x7fa2640a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa27009b380 tx=0x7fa2700073b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 -- 172.21.15.103:0/3431679826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219cb30 msgr2=0x55bde21a6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219cb30 0x55bde21a6eb0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa27400bc90 tx=0x7fa2740107a0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 -- 172.21.15.103:0/3431679826 shutdown_connections 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21a73f0 0x55bde21a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219cb30 0x55bde21a6eb0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 --2- 172.21.15.103:0/3431679826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219c210 0x55bde219c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 -- 172.21.15.103:0/3431679826 >> 172.21.15.103:0/3431679826 conn(0x55bde21987a0 msgr2=0x55bde2198b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 -- 172.21.15.103:0/3431679826 shutdown_connections 2024-08-19T01:42:36.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 -- 172.21.15.103:0/3431679826 wait complete. 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 -- start start 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.382+0000 7fa282755740 1 Processor -- start 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa282755740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219c210 0x55bde21a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa282755740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa282755740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21a73f0 0x55bde21a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa282755740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bde21a9fc0 con 0x55bde219c210 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa282755740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bde20bcaa0 con 0x55bde219cb30 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa282755740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bde21aa140 con 0x55bde21a73f0 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219c210 0x55bde21a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219c210 0x55bde21a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59440/0 (socket says 172.21.15.103:59440) 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa282755740 10 client.0 _ll_register_callbacks cb 0x55bde20abe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 -- 172.21.15.103:0/3232479167 learned_addr learned my addr 172.21.15.103:0/3232479167 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21a73f0 0x55bde21a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 -- 172.21.15.103:0/3232479167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21a73f0 msgr2=0x55bde21a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21a73f0 0x55bde21a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 -- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 msgr2=0x55bde21a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.383+0000 7fa27d8f2640 1 -- 172.21.15.103:0/3232479167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde20a0650 con 0x55bde219c210 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219c210 0x55bde21a4350 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa268005c40 tx=0x7fa268009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7b5881e5f4d7a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa261ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa261ffb640 1 -- 172.21.15.103:0/3232479167 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268035040 con 0x55bde219c210 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa261ffb640 1 -- 172.21.15.103:0/3232479167 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa2680035f0 con 0x55bde219c210 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27401cad0 con 0x55bde219c210 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa261ffb640 1 -- 172.21.15.103:0/3232479167 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa2680037f0 con 0x55bde219c210 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa282755740 10 client.17181 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.384+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bde1e733c0 con 0x55bde219c210 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa261ffb640 1 -- 172.21.15.103:0/3232479167 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa27401cad0 con 0x55bde219c210 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa261ffb640 1 -- 172.21.15.103:0/3232479167 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa26803a040 con 0x55bde219c210 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa282755740 10 client.17181 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa282755740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa282755740 20 client.17181 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa282755740 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde21b6f30 0x55bde21b9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa282755740 4 client.17181 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bde210eef0 con 0x55bde21b6f30 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.385+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde21b6f30 0x55bde21b9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.386+0000 7fa261ffb640 10 client.17181 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.386+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde21b6f30 0x55bde21b9320 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf993db74ce38f44 server_cookie=b55c3f03a0ebc8bb in_seq=0 out_seq=0 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.386+0000 7fa261ffb640 1 -- 172.21.15.103:0/3232479167 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bde210eef0 con 0x55bde21b6f30 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.386+0000 7fa261ffb640 10 client.17181 handle_command_reply: tid=0 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.386+0000 7fa282755740 1 client.17181 shutdown 2024-08-19T01:42:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219c210 msgr2=0x55bde21a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219c210 0x55bde21a4350 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa268005c40 tx=0x7fa268009260 comp rx=0 tx=0).stop 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 shutdown_connections 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde21b6f30 0x55bde21b9320 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21a73f0 0x55bde21a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 --2- 172.21.15.103:0/3232479167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde219c210 0x55bde21a4350 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 >> 172.21.15.103:0/3232479167 conn(0x55bde21987a0 msgr2=0x55bde219e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa261ffb640 0 client.17181 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 shutdown_connections 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 1 -- 172.21.15.103:0/3232479167 wait complete. 2024-08-19T01:42:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.387+0000 7fa282755740 20 client.17181 trim_cache size 0 max 16384 2024-08-19T01:42:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.413+0000 7fa282755740 1 Processor -- start 2024-08-19T01:42:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa282755740 1 -- start start 2024-08-19T01:42:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa282755740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa282755740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde219ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa282755740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21bebd0 0x55bde21bf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa282755740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781af4d0 con 0x55bde21a73f0 2024-08-19T01:42:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa282755740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781b1010 con 0x55bde219cb30 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa282755740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781afba0 con 0x55bde21bebd0 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde219ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde219ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59450/0 (socket says 172.21.15.103:59450) 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 -- 172.21.15.103:0/2010336083 learned_addr learned my addr 172.21.15.103:0/2010336083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21bebd0 0x55bde21bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 -- 172.21.15.103:0/2010336083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21bebd0 msgr2=0x55bde21bf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21bebd0 0x55bde21bf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 -- 172.21.15.103:0/2010336083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 msgr2=0x55bde21a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.414+0000 7fa27e0f3640 1 -- 172.21.15.103:0/2010336083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bde21a3360 con 0x55bde21a73f0 2024-08-19T01:42:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.415+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde219ec70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x55bde2199600 tx=0x7fa2740058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca2ada3e5e37935 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.415+0000 7fa263fff640 1 -- 172.21.15.103:0/2010336083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274023040 con 0x55bde21a73f0 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.415+0000 7fa263fff640 1 -- 172.21.15.103:0/2010336083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa274011520 con 0x55bde21a73f0 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.415+0000 7fa263fff640 1 -- 172.21.15.103:0/2010336083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274005520 con 0x55bde21a73f0 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.415+0000 7fa282755740 1 -- 172.21.15.103:0/2010336083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 msgr2=0x55bde219ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.415+0000 7fa282755740 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde219ec70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x55bde2199600 tx=0x7fa2740058b0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 -- 172.21.15.103:0/2010336083 shutdown_connections 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde21bebd0 0x55bde21bf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde219ec70 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 --2- 172.21.15.103:0/2010336083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde219cb30 0x55bde21a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 -- 172.21.15.103:0/2010336083 >> 172.21.15.103:0/2010336083 conn(0x55bde21987a0 msgr2=0x55bde21a62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 -- 172.21.15.103:0/2010336083 shutdown_connections 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 -- 172.21.15.103:0/2010336083 wait complete. 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 -- start start 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.416+0000 7fa282755740 1 Processor -- start 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa282755740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219cb30 0x55bde21af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa282755740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde21afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa282755740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21bebd0 0x55bde21ac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa282755740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781b0940 con 0x55bde21a73f0 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa282755740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781b0270 con 0x55bde21bebd0 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa282755740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781ad4b0 con 0x55bde219cb30 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa282755740 10 client.0 _ll_register_callbacks cb 0x7fa268031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27d8f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219cb30 0x55bde21af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde21afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27d8f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219cb30 0x55bde21af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:50430/0 (socket says 172.21.15.103:50430) 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde21afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59458/0 (socket says 172.21.15.103:59458) 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 -- 172.21.15.103:0/1965093921 learned_addr learned my addr 172.21.15.103:0/1965093921 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21bebd0 0x55bde21ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 -- 172.21.15.103:0/1965093921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219cb30 msgr2=0x55bde21af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219cb30 0x55bde21af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 -- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21bebd0 msgr2=0x55bde21ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:36.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21bebd0 0x55bde21ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27e0f3640 1 -- 172.21.15.103:0/1965093921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bde21acb90 con 0x55bde21a73f0 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.417+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21bebd0 0x55bde21ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa27e0f3640 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde21afab0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa274011010 tx=0x7fa27400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e64bf4b7742337 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa261ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa261ffb640 1 -- 172.21.15.103:0/1965093921 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274036020 con 0x55bde21a73f0 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa261ffb640 1 -- 172.21.15.103:0/1965093921 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fa2740051c0 con 0x55bde21a73f0 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa274011520 con 0x55bde21a73f0 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa261ffb640 1 -- 172.21.15.103:0/1965093921 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa274023030 con 0x55bde21a73f0 2024-08-19T01:42:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa282755740 10 client.17187 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.418+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bde22226e0 con 0x55bde21a73f0 2024-08-19T01:42:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa261ffb640 1 -- 172.21.15.103:0/1965093921 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa274011520 con 0x55bde21a73f0 2024-08-19T01:42:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa261ffb640 1 -- 172.21.15.103:0/1965093921 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa2740035f0 con 0x55bde21a73f0 2024-08-19T01:42:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa282755740 10 client.17187 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa282755740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa282755740 20 client.17187 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa282755740 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde2222ba0 0x55bde2224f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:36.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa282755740 4 client.17187 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:42:36.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bde2109400 con 0x55bde2222ba0 2024-08-19T01:42:36.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde2222ba0 0x55bde2224f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:36.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa27d8f2640 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde2222ba0 0x55bde2224f90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828989a51d338a23 server_cookie=9cf06566e163714c in_seq=0 out_seq=0 2024-08-19T01:42:36.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.419+0000 7fa261ffb640 10 client.17187 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa261ffb640 1 -- 172.21.15.103:0/1965093921 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bde2109400 con 0x55bde2222ba0 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa261ffb640 10 client.17187 handle_command_reply: tid=0 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 client.17187 shutdown 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 msgr2=0x55bde21afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde21afab0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa274011010 tx=0x7fa27400bd80 comp rx=0 tx=0).stop 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 shutdown_connections 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bde2222ba0 0x55bde2224f90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bde21bebd0 0x55bde21ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bde21a73f0 0x55bde21afab0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 --2- 172.21.15.103:0/1965093921 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bde219cb30 0x55bde21af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa261ffb640 0 client.17187 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 >> 172.21.15.103:0/1965093921 conn(0x55bde21987a0 msgr2=0x55bde2197900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 shutdown_connections 2024-08-19T01:42:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 1 -- 172.21.15.103:0/1965093921 wait complete. 2024-08-19T01:42:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.420+0000 7fa282755740 20 client.17187 trim_cache size 0 max 16384 2024-08-19T01:42:36.429 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:42:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:42:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:42:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "dc06bd78-2122-465c-b572-b5393e857aa3": { 2024-08-19T01:42:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:42:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "dc06bd78-2122-465c-b572-b5393e857aa3", 2024-08-19T01:42:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:42:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:42:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:42:36.430 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:42:36.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa2640a2c00 msgr2=0x7fa2640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa2640a2c00 0x7fa2640a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa27009b380 tx=0x7fa2700073b0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781078c0 msgr2=0x7fa27819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781078c0 0x7fa27819ec00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa27000eaa0 tx=0x7fa27000ef60 comp rx=0 tx=0).stop 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 shutdown_connections 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fa2640a2c00 0x7fa2640a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2781078c0 0x7fa27819ec00 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa278102260 0x7fa27819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 --2- 172.21.15.103:0/3690217426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2781018c0 0x7fa27819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:36.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 >> 172.21.15.103:0/3690217426 conn(0x7fa2780fd620 msgr2=0x7fa27810ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:36.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 shutdown_connections 2024-08-19T01:42:36.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:42:36.421+0000 7fa280357640 1 -- 172.21.15.103:0/3690217426 wait complete. 2024-08-19T01:42:36.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:42:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:42:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:42:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:42:36.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:36 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:42:37.235 INFO:tasks.cephfs.filesystem:scrub status for tag:dc06bd78-2122-465c-b572-b5393e857aa3 - {'path': '/', 'tag': 'dc06bd78-2122-465c-b572-b5393e857aa3', 'options': 'recursive,force'} 2024-08-19T01:42:37.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:37 smithi103 ceph-mon[18436]: pgmap v544: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 34 op/s 2024-08-19T01:42:37.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:37 smithi190 ceph-mon[22032]: pgmap v544: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 34 op/s 2024-08-19T01:42:37.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:37 smithi146 ceph-mon[22098]: pgmap v544: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 34 op/s 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi146 to 176.0M 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: Unable to set osd_memory_target on smithi146 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: Unable to set osd_memory_target on smithi190 to 184600371: error parsing value: Value '184600371' is below minimum 939524096 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: Unable to set osd_memory_target on smithi103 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:42:38.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:42:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:42:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi146 to 176.0M 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: Unable to set osd_memory_target on smithi146 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: Unable to set osd_memory_target on smithi190 to 184600371: error parsing value: Value '184600371' is below minimum 939524096 2024-08-19T01:42:38.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: Unable to set osd_memory_target on smithi103 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-08-19T01:42:38.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:42:38.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:42:38.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:42:38.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-19T01:42:38.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi146 to 176.0M 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi190 to 176.0M 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: Unable to set osd_memory_target on smithi146 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: Unable to set osd_memory_target on smithi190 to 184600371: error parsing value: Value '184600371' is below minimum 939524096 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: Unable to set osd_memory_target on smithi103 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:42:38.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:42:39.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:39 smithi103 ceph-mon[18436]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 46 op/s 2024-08-19T01:42:39.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:39 smithi190 ceph-mon[22032]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 46 op/s 2024-08-19T01:42:39.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:39 smithi146 ceph-mon[22098]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 46 op/s 2024-08-19T01:42:41.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:41 smithi103 ceph-mon[18436]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:42:41.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:41 smithi190 ceph-mon[22032]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:42:41.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:41 smithi146 ceph-mon[22098]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 MiB/s wr, 48 op/s 2024-08-19T01:42:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 -- 172.21.15.103:0/4276103782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0102320 msgr2=0x7efcf0108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 --2- 172.21.15.103:0/4276103782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0102320 0x7efcf0108bb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7efce8009ca0 tx=0x7efce802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 -- 172.21.15.103:0/4276103782 shutdown_connections 2024-08-19T01:42:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 --2- 172.21.15.103:0/4276103782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0102320 0x7efcf0108bb0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 --2- 172.21.15.103:0/4276103782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf0101960 0x7efcf0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 --2- 172.21.15.103:0/4276103782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcf0100770 0x7efcf0100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 -- 172.21.15.103:0/4276103782 >> 172.21.15.103:0/4276103782 conn(0x7efcf00fbf10 msgr2=0x7efcf00fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 -- 172.21.15.103:0/4276103782 shutdown_connections 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 -- 172.21.15.103:0/4276103782 wait complete. 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.815+0000 7efcf7037640 1 Processor -- start 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf7037640 1 -- start start 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf7037640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf0100770 0x7efcf006a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf7037640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0101960 0x7efcf019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf7037640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcf0102320 0x7efcf019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf7037640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcf010c2a0 con 0x7efcf0101960 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf7037640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcf010c120 con 0x7efcf0102320 2024-08-19T01:42:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf7037640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efcf010c420 con 0x7efcf0100770 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0101960 0x7efcf019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0101960 0x7efcf019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59472/0 (socket says 172.21.15.103:59472) 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 -- 172.21.15.103:0/3635259777 learned_addr learned my addr 172.21.15.103:0/3635259777 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf0100770 0x7efcf006a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efcf55d4640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcf0102320 0x7efcf019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 -- 172.21.15.103:0/3635259777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf0100770 msgr2=0x7efcf006a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf0100770 0x7efcf006a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 -- 172.21.15.103:0/3635259777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcf0102320 msgr2=0x7efcf019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcf0102320 0x7efcf019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 -- 172.21.15.103:0/3635259777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf019f650 con 0x7efcf0101960 2024-08-19T01:42:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.816+0000 7efceffff640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0101960 0x7efcf019c720 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00d770 tx=0x7efcdc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.817+0000 7efcedffb640 1 -- 172.21.15.103:0/3635259777 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc017070 con 0x7efcf0101960 2024-08-19T01:42:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.817+0000 7efcedffb640 1 -- 172.21.15.103:0/3635259777 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc0054d0 con 0x7efcf0101960 2024-08-19T01:42:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.817+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf019f8e0 con 0x7efcf0101960 2024-08-19T01:42:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.817+0000 7efcedffb640 1 -- 172.21.15.103:0/3635259777 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc004cf0 con 0x7efcf0101960 2024-08-19T01:42:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.817+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf019fde0 con 0x7efcf0101960 2024-08-19T01:42:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.819+0000 7efcedffb640 1 -- 172.21.15.103:0/3635259777 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efcdc004e90 con 0x7efcf0101960 2024-08-19T01:42:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 Processor -- start 2024-08-19T01:42:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 -- start start 2024-08-19T01:42:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcedffb640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efce00a2c30 0x7efce00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bcc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef40b239d0 con 0x55ef40bca460 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ef40acb820 con 0x55ef40bc4a90 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ef40b1f990 con 0x55ef40bc53d0 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf55d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bcc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efceffff640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efce00a2c30 0x7efce00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf55d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bcc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59480/0 (socket says 172.21.15.103:59480) 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf55d4640 1 -- 172.21.15.103:0/2267380681 learned_addr learned my addr 172.21.15.103:0/2267380681 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.820+0000 7efcedffb640 1 -- 172.21.15.103:0/3635259777 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efcdc0ce9e0 con 0x7efcf0101960 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcf55d4640 1 -- 172.21.15.103:0/2267380681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 msgr2=0x55ef40bc9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcf55d4640 1 -- 172.21.15.103:0/2267380681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 msgr2=0x55ef40bc4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcf55d4640 1 -- 172.21.15.103:0/2267380681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef40837210 con 0x55ef40bca460 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efceffff640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efce00a2c30 0x7efce00a50f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efcdc09b200 tx=0x7efcdc0bb040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bcc920 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efce8005cc0 tx=0x7efce8005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e87f6e586b1d10 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcdaffd640 1 -- 172.21.15.103:0/2267380681 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8032e90 con 0x55ef40bca460 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcdaffd640 1 -- 172.21.15.103:0/2267380681 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efce8036040 con 0x55ef40bca460 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.821+0000 7efcdaffd640 1 -- 172.21.15.103:0/2267380681 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8004040 con 0x55ef40bca460 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2267380681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 msgr2=0x55ef40bcc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bcc920 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efce8005cc0 tx=0x7efce8005c30 comp rx=0 tx=0).stop 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2267380681 shutdown_connections 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bcc920 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2267380681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2267380681 >> 172.21.15.103:0/2267380681 conn(0x55ef40bbbcf0 msgr2=0x55ef40bbc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2267380681 shutdown_connections 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2267380681 wait complete. 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 -- start start 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.822+0000 7efcf9dfa740 1 Processor -- start 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bc79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc53d0 0x55ef40bc7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bc8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef40bc0500 con 0x55ef40bca460 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ef40adf520 con 0x55ef40bc53d0 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ef40bc0680 con 0x55ef40bc4a90 2024-08-19T01:42:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf9dfa740 10 client.0 _ll_register_callbacks cb 0x55ef40836a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bc8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bc8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59484/0 (socket says 172.21.15.103:59484) 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 -- 172.21.15.103:0/1586953672 learned_addr learned my addr 172.21.15.103:0/1586953672 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bc79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc53d0 0x55ef40bc7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 -- 172.21.15.103:0/1586953672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 msgr2=0x55ef40bc79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bc79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 -- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc53d0 msgr2=0x55ef40bc7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc53d0 0x55ef40bc7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 -- 172.21.15.103:0/1586953672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef40903620 con 0x55ef40bca460 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efcf55d4640 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bc8f00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efce8009600 tx=0x7efce8031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d9906585c8fc16 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.823+0000 7efccffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efccffff640 1 -- 172.21.15.103:0/1586953672 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8003970 con 0x55ef40bca460 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efccffff640 1 -- 172.21.15.103:0/1586953672 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efce8002030 con 0x55ef40bca460 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efce8036040 con 0x55ef40bca460 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efccffff640 1 -- 172.21.15.103:0/1586953672 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efce8003b10 con 0x55ef40bca460 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 10 client.17199 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef40bc0500 con 0x55ef40bca460 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efccffff640 1 -- 172.21.15.103:0/1586953672 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efce8036040 con 0x55ef40bca460 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efccffff640 1 -- 172.21.15.103:0/1586953672 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7efce8004600 con 0x55ef40bca460 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 10 client.17199 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 20 client.17199 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40bd81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 4 client.17199 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.824+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef40b31100 con 0x55ef40bd5df0 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.825+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40bd81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.825+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40bd81e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc407537e6b4177 server_cookie=9191cc77afc8a1e7 in_seq=0 out_seq=0 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.825+0000 7efccffff640 10 client.17199 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efccffff640 1 -- 172.21.15.103:0/1586953672 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef40b31100 con 0x55ef40bd5df0 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efccffff640 10 client.17199 handle_command_reply: tid=0 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 client.17199 shutdown 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 msgr2=0x55ef40bc8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bc8f00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efce8009600 tx=0x7efce8031630 comp rx=0 tx=0).stop 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 shutdown_connections 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40bd81e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efccffff640 0 client.17199 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bc8f00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc53d0 0x55ef40bc7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/1586953672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bc79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 >> 172.21.15.103:0/1586953672 conn(0x55ef40bbbcf0 msgr2=0x55ef40bbd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 shutdown_connections 2024-08-19T01:42:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 1 -- 172.21.15.103:0/1586953672 wait complete. 2024-08-19T01:42:41.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.826+0000 7efcf9dfa740 20 client.17199 trim_cache size 0 max 16384 2024-08-19T01:42:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 Processor -- start 2024-08-19T01:42:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 -- start start 2024-08-19T01:42:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bbcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01a68a0 con 0x55ef40bca460 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01a83e0 con 0x55ef40bc4a90 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.845+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01a6f70 con 0x55ef40bc53d0 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bbcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bbcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39314/0 (socket says 172.21.15.103:39314) 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 -- 172.21.15.103:0/2154284860 learned_addr learned my addr 172.21.15.103:0/2154284860 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 -- 172.21.15.103:0/2154284860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 msgr2=0x55ef40bc6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 -- 172.21.15.103:0/2154284860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 msgr2=0x55ef40bc5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 -- 172.21.15.103:0/2154284860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef40bbbb10 con 0x55ef40bca460 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcf55d4640 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bbcba0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efce8031aa0 tx=0x7efce8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40252d6c90feef42 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcda7fc640 1 -- 172.21.15.103:0/2154284860 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8003970 con 0x55ef40bca460 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcda7fc640 1 -- 172.21.15.103:0/2154284860 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efce8036030 con 0x55ef40bca460 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.846+0000 7efcda7fc640 1 -- 172.21.15.103:0/2154284860 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce802f4c0 con 0x55ef40bca460 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2154284860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 msgr2=0x55ef40bbcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bbcba0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efce8031aa0 tx=0x7efce8005fd0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2154284860 shutdown_connections 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bca460 0x55ef40bbcba0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc53d0 0x55ef40bc6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/2154284860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bc4a90 0x55ef40bc5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2154284860 >> 172.21.15.103:0/2154284860 conn(0x55ef40bbbcf0 msgr2=0x55ef40bc7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2154284860 shutdown_connections 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 -- 172.21.15.103:0/2154284860 wait complete. 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 -- start start 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.847+0000 7efcf9dfa740 1 Processor -- start 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bcf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bc53d0 0x55ef40bcf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf9dfa740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bca460 0x55ef40c3ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01a7d10 con 0x55ef40bc53d0 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01a7640 con 0x55ef40bca460 2024-08-19T01:42:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf9dfa740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01a4880 con 0x55ef40bc4a90 2024-08-19T01:42:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf9dfa740 10 client.0 _ll_register_callbacks cb 0x7efce8031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bc53d0 0x55ef40bcf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bc53d0 0x55ef40bcf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39322/0 (socket says 172.21.15.103:39322) 2024-08-19T01:42:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 -- 172.21.15.103:0/3164029569 learned_addr learned my addr 172.21.15.103:0/3164029569 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf55d4640 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bcf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf55d4640 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bca460 0x55ef40c3ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 -- 172.21.15.103:0/3164029569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 msgr2=0x55ef40bcf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bcf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 -- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bca460 msgr2=0x55ef40c3ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bca460 0x55ef40c3ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.848+0000 7efcf4dd3640 1 -- 172.21.15.103:0/3164029569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef40bd2ac0 con 0x55ef40bc53d0 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf4dd3640 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bc53d0 0x55ef40bcf9d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efce4005fd0 tx=0x7efce400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5d8c43fdd2baf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efccffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efccffff640 1 -- 172.21.15.103:0/3164029569 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4005220 con 0x55ef40bc53d0 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efccffff640 1 -- 172.21.15.103:0/3164029569 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efce40053c0 con 0x55ef40bc53d0 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efce8036030 con 0x55ef40bc53d0 2024-08-19T01:42:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efccffff640 1 -- 172.21.15.103:0/3164029569 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efce40055c0 con 0x55ef40bc53d0 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 10 client.17205 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef40c3ff30 con 0x55ef40bc53d0 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efccffff640 1 -- 172.21.15.103:0/3164029569 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efce8036030 con 0x55ef40bc53d0 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efccffff640 1 -- 172.21.15.103:0/3164029569 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7efce401d790 con 0x55ef40bc53d0 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 10 client.17205 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 20 client.17205 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40c429a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 4 client.17205 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.849+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef40b2b450 con 0x55ef40bd5df0 2024-08-19T01:42:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.850+0000 7efcf55d4640 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40c429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.850+0000 7efccffff640 10 client.17205 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.850+0000 7efcf55d4640 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40c429a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f88b57072886a9 server_cookie=13e434a486e7bb1b in_seq=0 out_seq=0 2024-08-19T01:42:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efccffff640 1 -- 172.21.15.103:0/3164029569 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef40b2b450 con 0x55ef40bd5df0 2024-08-19T01:42:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efccffff640 10 client.17205 handle_command_reply: tid=0 2024-08-19T01:42:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 client.17205 shutdown 2024-08-19T01:42:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bc53d0 msgr2=0x55ef40bcf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bc53d0 0x55ef40bcf9d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efce4005fd0 tx=0x7efce400b690 comp rx=0 tx=0).stop 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 shutdown_connections 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ef40bd5df0 0x55ef40c429a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efccffff640 0 client.17205 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ef40bca460 0x55ef40c3ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef40bc53d0 0x55ef40bcf9d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 --2- 172.21.15.103:0/3164029569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef40bc4a90 0x55ef40bcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 >> 172.21.15.103:0/3164029569 conn(0x55ef40bbbcf0 msgr2=0x55ef40bc5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 shutdown_connections 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 1 -- 172.21.15.103:0/3164029569 wait complete. 2024-08-19T01:42:41.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.857+0000 7efcf9dfa740 20 client.17205 trim_cache size 0 max 16384 2024-08-19T01:42:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:42:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:42:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:42:41.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efce00a2c30 msgr2=0x7efce00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efce00a2c30 0x7efce00a50f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efcdc09b200 tx=0x7efcdc0bb040 comp rx=0 tx=0).stop 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0101960 msgr2=0x7efcf019c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0101960 0x7efcf019c720 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00d770 tx=0x7efcdc00dc30 comp rx=0 tx=0).stop 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 shutdown_connections 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efce00a2c30 0x7efce00a50f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efcf0102320 0x7efcf019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcf0101960 0x7efcf019c720 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 --2- 172.21.15.103:0/3635259777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf0100770 0x7efcf006a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 >> 172.21.15.103:0/3635259777 conn(0x7efcf00fbf10 msgr2=0x7efcf0104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 shutdown_connections 2024-08-19T01:42:41.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:41.858+0000 7efcf7037640 1 -- 172.21.15.103:0/3635259777 wait complete. 2024-08-19T01:42:43.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:43 smithi103 ceph-mon[18436]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 50 op/s 2024-08-19T01:42:43.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:43 smithi190 ceph-mon[22032]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 50 op/s 2024-08-19T01:42:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:43 smithi146 ceph-mon[22098]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 50 op/s 2024-08-19T01:42:45.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:45 smithi103 ceph-mon[18436]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 45 op/s 2024-08-19T01:42:45.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:45 smithi190 ceph-mon[22032]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 45 op/s 2024-08-19T01:42:45.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:45 smithi146 ceph-mon[22098]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 45 op/s 2024-08-19T01:42:46.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:46 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:42:46.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:46 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:42:46.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:46 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:42:47.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:47 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:42:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:47 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:42:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:47 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:42:47.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:47 smithi103 ceph-mon[18436]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:42:47.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:47 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:42:47.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:47 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:42:47.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:47 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:42:47.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:47 smithi190 ceph-mon[22032]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:42:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:47 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:42:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:47 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:42:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:47 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:42:47.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:47 smithi146 ceph-mon[22098]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 44 op/s 2024-08-19T01:42:49.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:49 smithi103 ceph-mon[18436]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 37 op/s 2024-08-19T01:42:49.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:49 smithi190 ceph-mon[22032]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 37 op/s 2024-08-19T01:42:49.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:49 smithi146 ceph-mon[22098]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 37 op/s 2024-08-19T01:42:51.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:51 smithi103 ceph-mon[18436]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:42:51.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:51 smithi190 ceph-mon[22032]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:42:51.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:51 smithi146 ceph-mon[22098]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-08-19T01:42:53.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:53 smithi103 ceph-mon[18436]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:42:53.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:53 smithi190 ceph-mon[22032]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:42:53.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:53 smithi146 ceph-mon[22098]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:42:55.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:55 smithi103 ceph-mon[18436]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-08-19T01:42:55.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:55 smithi190 ceph-mon[22032]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-08-19T01:42:55.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:55 smithi146 ceph-mon[22098]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-08-19T01:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 -- 172.21.15.103:0/3051115014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c104c80 msgr2=0x7ff72c107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 --2- 172.21.15.103:0/3051115014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c104c80 0x7ff72c107070 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009900 tx=0x7ff71c02a410 comp rx=0 tx=0).stop 2024-08-19T01:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 -- 172.21.15.103:0/3051115014 shutdown_connections 2024-08-19T01:42:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 --2- 172.21.15.103:0/3051115014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff72c1075b0 0x7ff72c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 --2- 172.21.15.103:0/3051115014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c104c80 0x7ff72c107070 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 --2- 172.21.15.103:0/3051115014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c102350 0x7ff72c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 -- 172.21.15.103:0/3051115014 >> 172.21.15.103:0/3051115014 conn(0x7ff72c0fbf10 msgr2=0x7ff72c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 -- 172.21.15.103:0/3051115014 shutdown_connections 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.667+0000 7ff73392e640 1 -- 172.21.15.103:0/3051115014 wait complete. 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 Processor -- start 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 -- start start 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c102350 0x7ff72c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c104c80 0x7ff72c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff72c1075b0 0x7ff72c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c10d3e0 con 0x7ff72c102350 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c10d260 con 0x7ff72c1075b0 2024-08-19T01:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff73392e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c10d560 con 0x7ff72c104c80 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c102350 0x7ff72c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c102350 0x7ff72c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37676/0 (socket says 172.21.15.103:37676) 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 -- 172.21.15.103:0/397610956 learned_addr learned my addr 172.21.15.103:0/397610956 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff730ec9640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c104c80 0x7ff72c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff731ecb640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff72c1075b0 0x7ff72c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 -- 172.21.15.103:0/397610956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c104c80 msgr2=0x7ff72c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c104c80 0x7ff72c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 -- 172.21.15.103:0/397610956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff72c1075b0 msgr2=0x7ff72c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff72c1075b0 0x7ff72c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 -- 172.21.15.103:0/397610956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c19f6e0 con 0x7ff72c102350 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff731ecb640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff72c1075b0 0x7ff72c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.668+0000 7ff7316ca640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c102350 0x7ff72c197ac0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e9f0 tx=0x7ff72800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.669+0000 7ff7227fc640 1 -- 172.21.15.103:0/397610956 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72800cd80 con 0x7ff72c102350 2024-08-19T01:42:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.669+0000 7ff7227fc640 1 -- 172.21.15.103:0/397610956 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7280054d0 con 0x7ff72c102350 2024-08-19T01:42:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.669+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c19f9d0 con 0x7ff72c102350 2024-08-19T01:42:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.669+0000 7ff7227fc640 1 -- 172.21.15.103:0/397610956 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728010490 con 0x7ff72c102350 2024-08-19T01:42:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.669+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c19fea0 con 0x7ff72c102350 2024-08-19T01:42:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.671+0000 7ff7227fc640 1 -- 172.21.15.103:0/397610956 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff728005020 con 0x7ff72c102350 2024-08-19T01:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.672+0000 7ff7227fc640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff7180a2c30 0x7ff7180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.672+0000 7ff730ec9640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff7180a2c30 0x7ff7180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.672+0000 7ff7227fc640 1 -- 172.21.15.103:0/397610956 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff7280cf420 con 0x7ff72c102350 2024-08-19T01:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.672+0000 7ff730ec9640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff7180a2c30 0x7ff7180a50f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff71c004770 tx=0x7ff71c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.837+0000 7ff7367bf740 1 Processor -- start 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.837+0000 7ff7367bf740 1 -- start start 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.837+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80354e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.837+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80359e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b8035c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7367bf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557b802b39d0 con 0x557b80354a90 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7367bf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557b8025b820 con 0x557b803553d0 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7367bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557b802af990 con 0x557b8035a460 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80354e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80354e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37678/0 (socket says 172.21.15.103:37678) 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 -- 172.21.15.103:0/2990035039 learned_addr learned my addr 172.21.15.103:0/2990035039 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff730ec9640 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80359e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff731ecb640 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b8035c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 -- 172.21.15.103:0/2990035039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 msgr2=0x557b8035c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b8035c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 -- 172.21.15.103:0/2990035039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 msgr2=0x557b80359e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80359e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 -- 172.21.15.103:0/2990035039 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b7ffc7210 con 0x557b80354a90 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7316ca640 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80354e90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff728098110 tx=0x7ff728005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc58951245e763e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7177fe640 1 -- 172.21.15.103:0/2990035039 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72809bcb0 con 0x557b80354a90 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7177fe640 1 -- 172.21.15.103:0/2990035039 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7280a0060 con 0x557b80354a90 2024-08-19T01:42:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.838+0000 7ff7177fe640 1 -- 172.21.15.103:0/2990035039 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728098320 con 0x557b80354a90 2024-08-19T01:42:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 -- 172.21.15.103:0/2990035039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 msgr2=0x557b80354e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80354e90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff728098110 tx=0x7ff728005310 comp rx=0 tx=0).stop 2024-08-19T01:42:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 -- 172.21.15.103:0/2990035039 shutdown_connections 2024-08-19T01:42:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b8035c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80359e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 --2- 172.21.15.103:0/2990035039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80354e90 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 -- 172.21.15.103:0/2990035039 >> 172.21.15.103:0/2990035039 conn(0x557b8034bcf0 msgr2=0x557b8034c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 -- 172.21.15.103:0/2990035039 shutdown_connections 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.839+0000 7ff7367bf740 1 -- 172.21.15.103:0/2990035039 wait complete. 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 -- start start 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 Processor -- start 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b80354a90 0x557b8035ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b803553d0 0x557b8035f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80366170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557b8035d430 con 0x557b803553d0 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557b8026f520 con 0x557b80354a90 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557b8035d5b0 con 0x557b8035a460 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.840+0000 7ff7367bf740 10 client.0 _ll_register_callbacks cb 0x557b7ffc6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff730ec9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b803553d0 0x557b8035f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7316ca640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b80354a90 0x557b8035ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff730ec9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b803553d0 0x557b8035f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37694/0 (socket says 172.21.15.103:37694) 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff730ec9640 1 -- 172.21.15.103:0/3091429920 learned_addr learned my addr 172.21.15.103:0/3091429920 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff731ecb640 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80366170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7316ca640 1 -- 172.21.15.103:0/3091429920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 msgr2=0x557b80366170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7316ca640 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80366170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7316ca640 1 -- 172.21.15.103:0/3091429920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b803553d0 msgr2=0x557b8035f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7316ca640 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b803553d0 0x557b8035f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7316ca640 1 -- 172.21.15.103:0/3091429920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b80093620 con 0x557b80354a90 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7316ca640 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b80354a90 0x557b8035ebf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff728098110 tx=0x7ff7280a4920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe036ec52eb0ec48 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff714ff9640 1 -- 172.21.15.103:0/3091429920 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7280a4490 con 0x557b80354a90 2024-08-19T01:42:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff714ff9640 1 -- 172.21.15.103:0/3091429920 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff7280a4630 con 0x557b80354a90 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.841+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7280a0060 con 0x557b80354a90 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff714ff9640 1 -- 172.21.15.103:0/3091429920 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff7280ab050 con 0x557b80354a90 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 10 client.26522 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557b80368f60 con 0x557b80354a90 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff714ff9640 1 -- 172.21.15.103:0/3091429920 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7280a0060 con 0x557b80354a90 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff714ff9640 1 -- 172.21.15.103:0/3091429920 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7ff72809ab70 con 0x557b80354a90 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 10 client.26522 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 10 client.26522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 20 client.26522 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b8036a7d0 0x557b8036cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 4 client.26522 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b802c1100 con 0x557b8036a7d0 2024-08-19T01:42:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.842+0000 7ff730ec9640 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b8036a7d0 0x557b8036cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.843+0000 7ff714ff9640 10 client.26522 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.843+0000 7ff730ec9640 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b8036a7d0 0x557b8036cc20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20290e40ce29c46e server_cookie=ca2e7830391926 in_seq=0 out_seq=0 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff714ff9640 1 -- 172.21.15.103:0/3091429920 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b802c1100 con 0x557b8036a7d0 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff714ff9640 10 client.26522 handle_command_reply: tid=0 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 client.26522 shutdown 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b80354a90 msgr2=0x557b8035ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b80354a90 0x557b8035ebf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff728098110 tx=0x7ff7280a4920 comp rx=0 tx=0).stop 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 shutdown_connections 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b8036a7d0 0x557b8036cc20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80366170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff714ff9640 0 client.26522 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b803553d0 0x557b8035f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3091429920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b80354a90 0x557b8035ebf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 >> 172.21.15.103:0/3091429920 conn(0x557b8034bcf0 msgr2=0x557b8034d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 shutdown_connections 2024-08-19T01:42:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 1 -- 172.21.15.103:0/3091429920 wait complete. 2024-08-19T01:42:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.844+0000 7ff7367bf740 20 client.26522 trim_cache size 0 max 16384 2024-08-19T01:42:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 Processor -- start 2024-08-19T01:42:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 -- start start 2024-08-19T01:42:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80354a90 0x557b8035f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80366100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b8035a460 0x557b8034ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c1a68f0 con 0x557b8035a460 2024-08-19T01:42:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c1a8430 con 0x557b803553d0 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff7367bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c1a6fc0 con 0x557b80354a90 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff731ecb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b8035a460 0x557b8034ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff731ecb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b8035a460 0x557b8034ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37708/0 (socket says 172.21.15.103:37708) 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff731ecb640 1 -- 172.21.15.103:0/1445217796 learned_addr learned my addr 172.21.15.103:0/1445217796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.861+0000 7ff730ec9640 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80366100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff7316ca640 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80354a90 0x557b8035f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff730ec9640 1 -- 172.21.15.103:0/1445217796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80354a90 msgr2=0x557b8035f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff730ec9640 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80354a90 0x557b8035f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff730ec9640 1 -- 172.21.15.103:0/1445217796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b8035a460 msgr2=0x557b8034ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff730ec9640 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b8035a460 0x557b8034ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff730ec9640 1 -- 172.21.15.103:0/1445217796 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b8035d560 con 0x557b803553d0 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff731ecb640 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b8035a460 0x557b8034ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff730ec9640 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80366100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x557b8036d4a0 tx=0x7ff71c03e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cca9e2a8183e428 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff716ffd640 1 -- 172.21.15.103:0/1445217796 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c035050 con 0x557b803553d0 2024-08-19T01:42:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff716ffd640 1 -- 172.21.15.103:0/1445217796 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff71c004310 con 0x557b803553d0 2024-08-19T01:42:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.862+0000 7ff716ffd640 1 -- 172.21.15.103:0/1445217796 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c052620 con 0x557b803553d0 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 -- 172.21.15.103:0/1445217796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 msgr2=0x557b80366100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80366100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x557b8036d4a0 tx=0x7ff71c03e070 comp rx=0 tx=0).stop 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 -- 172.21.15.103:0/1445217796 shutdown_connections 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b8035a460 0x557b8034ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80366100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 --2- 172.21.15.103:0/1445217796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b80354a90 0x557b8035f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 -- 172.21.15.103:0/1445217796 >> 172.21.15.103:0/1445217796 conn(0x557b8034bcf0 msgr2=0x557b8034a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 -- 172.21.15.103:0/1445217796 shutdown_connections 2024-08-19T01:42:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 -- 172.21.15.103:0/1445217796 wait complete. 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.863+0000 7ff7367bf740 1 -- start start 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 1 Processor -- start 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80363100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80363640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80364650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c1a7d60 con 0x557b80354a90 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c1a7690 con 0x557b803553d0 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c1a48d0 con 0x557b8035a460 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7367bf740 10 client.0 _ll_register_callbacks cb 0x7ff72809b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7316ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80363100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7316ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80363100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37712/0 (socket says 172.21.15.103:37712) 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff7316ca640 1 -- 172.21.15.103:0/3983461642 learned_addr learned my addr 172.21.15.103:0/3983461642 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff730ec9640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80363640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff731ecb640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80364650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff730ec9640 1 -- 172.21.15.103:0/3983461642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 msgr2=0x557b80364650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff730ec9640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80364650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff730ec9640 1 -- 172.21.15.103:0/3983461642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 msgr2=0x557b80363100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff730ec9640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80363100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.864+0000 7ff730ec9640 1 -- 172.21.15.103:0/3983461642 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b8036d2a0 con 0x557b803553d0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff7316ca640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80363100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff730ec9640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80363640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff71c031910 tx=0x7ff71c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28677806a054b21f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff714ff9640 1 -- 172.21.15.103:0/3983461642 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c035070 con 0x557b803553d0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff714ff9640 1 -- 172.21.15.103:0/3983461642 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff71c0035f0 con 0x557b803553d0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c004310 con 0x557b803553d0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff714ff9640 1 -- 172.21.15.103:0/3983461642 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff71c056040 con 0x557b803553d0 2024-08-19T01:42:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff7367bf740 10 client.26528 fetch_fsmap learned FSMap version 36 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557b80365da0 con 0x557b803553d0 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.865+0000 7ff714ff9640 1 -- 172.21.15.103:0/3983461642 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff71c004310 con 0x557b803553d0 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff714ff9640 1 -- 172.21.15.103:0/3983461642 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7ff71c031ce0 con 0x557b803553d0 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff7367bf740 10 client.26528 fetch_fsmap finished waiting for FSMap version 36 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff7367bf740 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff7367bf740 20 client.26528 populate_metadata read hostname 'smithi103' 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b803d5630 0x557b803d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff7367bf740 4 client.26528 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b802bb450 con 0x557b803d5630 2024-08-19T01:42:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.866+0000 7ff7316ca640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b803d5630 0x557b803d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:42:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.869+0000 7ff7316ca640 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b803d5630 0x557b803d7a20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a367c723d6e96f90 server_cookie=eb569f16da76852f in_seq=0 out_seq=0 2024-08-19T01:42:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.869+0000 7ff714ff9640 10 client.26528 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.876+0000 7ff714ff9640 1 -- 172.21.15.103:0/3983461642 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b802bb450 con 0x557b803d5630 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.876+0000 7ff714ff9640 10 client.26528 handle_command_reply: tid=0 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.876+0000 7ff7367bf740 1 client.26528 shutdown 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 msgr2=0x557b80363640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80363640 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff71c031910 tx=0x7ff71c005fd0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 shutdown_connections 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x557b803d5630 0x557b803d7a20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b8035a460 0x557b80364650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff714ff9640 0 client.26528 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557b803553d0 0x557b80363640 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 --2- 172.21.15.103:0/3983461642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b80354a90 0x557b80363100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 >> 172.21.15.103:0/3983461642 conn(0x557b8034bcf0 msgr2=0x557b8034a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 shutdown_connections 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 1 -- 172.21.15.103:0/3983461642 wait complete. 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.877+0000 7ff7367bf740 20 client.26528 trim_cache size 0 max 16384 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:42:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:42:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff7180a2c30 msgr2=0x7ff7180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff7180a2c30 0x7ff7180a50f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff71c004770 tx=0x7ff71c000bc0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c102350 msgr2=0x7ff72c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:42:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c102350 0x7ff72c197ac0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff72800e9f0 tx=0x7ff72800edf0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 shutdown_connections 2024-08-19T01:42:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff7180a2c30 0x7ff7180a50f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff72c1075b0 0x7ff72c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff72c104c80 0x7ff72c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 --2- 172.21.15.103:0/397610956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff72c102350 0x7ff72c197ac0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:42:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 >> 172.21.15.103:0/397610956 conn(0x7ff72c0fbf10 msgr2=0x7ff72c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:42:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 shutdown_connections 2024-08-19T01:42:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:42:55.878+0000 7ff73392e640 1 -- 172.21.15.103:0/397610956 wait complete. 2024-08-19T01:42:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:57 smithi103 ceph-mon[18436]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:42:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:57 smithi190 ceph-mon[22032]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:42:57.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:57 smithi146 ceph-mon[22098]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-08-19T01:42:59.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:42:59 smithi103 ceph-mon[18436]: pgmap v555: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:42:59.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:42:59 smithi190 ceph-mon[22032]: pgmap v555: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:42:59.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:42:59 smithi146 ceph-mon[22098]: pgmap v555: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:43:01.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:01 smithi103 ceph-mon[18436]: pgmap v556: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:43:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:01 smithi190 ceph-mon[22032]: pgmap v556: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:43:01.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:01 smithi146 ceph-mon[22098]: pgmap v556: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-08-19T01:43:03.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:03 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:43:03.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:03 smithi103 ceph-mon[18436]: pgmap v557: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:43:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:03 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:43:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:03 smithi190 ceph-mon[22032]: pgmap v557: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:43:03.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:03 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:43:03.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:03 smithi146 ceph-mon[22098]: pgmap v557: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-08-19T01:43:05.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:05 smithi103 ceph-mon[18436]: pgmap v558: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:43:05.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:05 smithi190 ceph-mon[22032]: pgmap v558: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:43:05.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:05 smithi146 ceph-mon[22098]: pgmap v558: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-08-19T01:43:07.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:43:07.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:07 smithi103 ceph-mon[18436]: pgmap v559: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:43:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:07 smithi190 ceph-mon[22032]: pgmap v559: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:43:07.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:07 smithi146 ceph-mon[22098]: pgmap v559: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:43:07.699 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:08.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.647+0000 7fe256adb640 1 -- 172.21.15.103:0/2510539426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2501006e0 msgr2=0x7fe250100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.647+0000 7fe256adb640 1 --2- 172.21.15.103:0/2510539426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2501006e0 0x7fe250100ac0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe244009900 tx=0x7fe24402a410 comp rx=0 tx=0).stop 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 -- 172.21.15.103:0/2510539426 shutdown_connections 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 --2- 172.21.15.103:0/2510539426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25010e200 0x7fe250110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 --2- 172.21.15.103:0/2510539426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe250101090 0x7fe25010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 --2- 172.21.15.103:0/2510539426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2501006e0 0x7fe250100ac0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 -- 172.21.15.103:0/2510539426 >> 172.21.15.103:0/2510539426 conn(0x7fe2500fc500 msgr2=0x7fe2500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 -- 172.21.15.103:0/2510539426 shutdown_connections 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 -- 172.21.15.103:0/2510539426 wait complete. 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.648+0000 7fe256adb640 1 Processor -- start 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe256adb640 1 -- start start 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe256adb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2501006e0 0x7fe25019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe256adb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe250101090 0x7fe25019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe256adb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25010e200 0x7fe25019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe256adb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe250113e70 con 0x7fe250101090 2024-08-19T01:43:08.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe256adb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe250113cf0 con 0x7fe2501006e0 2024-08-19T01:43:08.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe256adb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe250113ff0 con 0x7fe25010e200 2024-08-19T01:43:08.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe250101090 0x7fe25019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe250101090 0x7fe25019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50442/0 (socket says 172.21.15.103:50442) 2024-08-19T01:43:08.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 -- 172.21.15.103:0/2861375609 learned_addr learned my addr 172.21.15.103:0/2861375609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:08.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe254877640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2501006e0 0x7fe25019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe255078640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25010e200 0x7fe25019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 -- 172.21.15.103:0/2861375609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25010e200 msgr2=0x7fe25019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25010e200 0x7fe25019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 -- 172.21.15.103:0/2861375609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2501006e0 msgr2=0x7fe25019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2501006e0 0x7fe25019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.649+0000 7fe24ffff640 1 -- 172.21.15.103:0/2861375609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25019f330 con 0x7fe250101090 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.650+0000 7fe24ffff640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe250101090 0x7fe25019a8c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00d950 tx=0x7fe23c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.650+0000 7fe24dffb640 1 -- 172.21.15.103:0/2861375609 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c017030 con 0x7fe250101090 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.650+0000 7fe24dffb640 1 -- 172.21.15.103:0/2861375609 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe23c004020 con 0x7fe250101090 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.650+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25019f620 con 0x7fe250101090 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.650+0000 7fe24dffb640 1 -- 172.21.15.103:0/2861375609 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0047a0 con 0x7fe250101090 2024-08-19T01:43:08.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.650+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2501a6f00 con 0x7fe250101090 2024-08-19T01:43:08.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe24dffb640 1 -- 172.21.15.103:0/2861375609 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe23c0041d0 con 0x7fe250101090 2024-08-19T01:43:08.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 Processor -- start 2024-08-19T01:43:08.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 -- start start 2024-08-19T01:43:08.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bfb30 0x5600c55c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600c5524980 con 0x5600c55ca3f0 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5600c54bca70 con 0x5600c55bfb30 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe258ed9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5600c5520940 con 0x5600c55bf210 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50444/0 (socket says 172.21.15.103:50444) 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe24ffff640 1 -- 172.21.15.103:0/1618792147 learned_addr learned my addr 172.21.15.103:0/1618792147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe255078640 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.653+0000 7fe254877640 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bfb30 0x5600c55c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe255078640 1 -- 172.21.15.103:0/1618792147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bfb30 msgr2=0x5600c55c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe255078640 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bfb30 0x5600c55c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe255078640 1 -- 172.21.15.103:0/1618792147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 msgr2=0x5600c55cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe255078640 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe24dffb640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2400a2b80 0x7fe2400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe255078640 1 -- 172.21.15.103:0/1618792147 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c551f6e0 con 0x5600c55bf210 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe254877640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2400a2b80 0x7fe2400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe24dffb640 1 -- 172.21.15.103:0/2861375609 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe23c0ce9b0 con 0x7fe250101090 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe255078640 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55bf5f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe248009eb0 tx=0x7fe248010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=294a9c4e9d8e07d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe23affd640 1 -- 172.21.15.103:0/1618792147 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248016070 con 0x5600c55bf210 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe23affd640 1 -- 172.21.15.103:0/1618792147 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe248004d20 con 0x5600c55bf210 2024-08-19T01:43:08.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.654+0000 7fe23affd640 1 -- 172.21.15.103:0/1618792147 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2480126b0 con 0x5600c55bf210 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 -- 172.21.15.103:0/1618792147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 msgr2=0x5600c55bf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe254877640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2400a2b80 0x7fe2400a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe244005ab0 tx=0x7fe244005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55bf5f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe248009eb0 tx=0x7fe248010740 comp rx=0 tx=0).stop 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 -- 172.21.15.103:0/1618792147 shutdown_connections 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bfb30 0x5600c55c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 --2- 172.21.15.103:0/1618792147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55bf5f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 -- 172.21.15.103:0/1618792147 >> 172.21.15.103:0/1618792147 conn(0x5600c55bb7a0 msgr2=0x5600c55bbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 -- 172.21.15.103:0/1618792147 shutdown_connections 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.655+0000 7fe258ed9740 1 -- 172.21.15.103:0/1618792147 wait complete. 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 -- start start 2024-08-19T01:43:08.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 Processor -- start 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600c55ccfc0 con 0x5600c55ca3f0 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5600c54dfaa0 con 0x5600c55bf210 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5600c55cd140 con 0x5600c55bfb30 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe258ed9740 10 client.0 _ll_register_callbacks cb 0x5600c54cee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe254877640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe254877640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39086/0 (socket says 172.21.15.103:39086) 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe254877640 1 -- 172.21.15.103:0/525427772 learned_addr learned my addr 172.21.15.103:0/525427772 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe255078640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.656+0000 7fe24ffff640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe254877640 1 -- 172.21.15.103:0/525427772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 msgr2=0x5600c55c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe254877640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe254877640 1 -- 172.21.15.103:0/525427772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 msgr2=0x5600c55c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe254877640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe254877640 1 -- 172.21.15.103:0/525427772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c54c3650 con 0x5600c55bf210 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe24ffff640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe255078640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe254877640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55c72c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe2440041b0 tx=0x7fe244009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a05c6787a3b64ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe22ffff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe22ffff640 1 -- 172.21.15.103:0/525427772 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24404e070 con 0x5600c55bf210 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe248004d20 con 0x5600c55bf210 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe22ffff640 1 -- 172.21.15.103:0/525427772 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe244004b80 con 0x5600c55bf210 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe22ffff640 1 -- 172.21.15.103:0/525427772 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe24405a050 con 0x5600c55bf210 2024-08-19T01:43:08.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe258ed9740 10 client.26540 fetch_fsmap learned FSMap version 37 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.657+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5600c52963c0 con 0x5600c55bf210 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe22ffff640 1 -- 172.21.15.103:0/525427772 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe248004d20 con 0x5600c55bf210 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe22ffff640 1 -- 172.21.15.103:0/525427772 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fe2440324b0 con 0x5600c55bf210 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe258ed9740 10 client.26540 fetch_fsmap finished waiting for FSMap version 37 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe258ed9740 10 client.26540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe258ed9740 20 client.26540 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe258ed9740 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c55d9e50 0x5600c55dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe258ed9740 4 client.26540 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600c5531ef0 con 0x5600c55d9e50 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.658+0000 7fe24ffff640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c55d9e50 0x5600c55dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe24ffff640 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c55d9e50 0x5600c55dc240 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550efa16c04f6d7f server_cookie=4e86301f5d6b5ec0 in_seq=0 out_seq=0 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe22ffff640 10 client.26540 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe22ffff640 1 -- 172.21.15.103:0/525427772 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600c5531ef0 con 0x5600c55d9e50 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe22ffff640 10 client.26540 handle_command_reply: tid=0 2024-08-19T01:43:08.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 client.26540 shutdown 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 msgr2=0x5600c55c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55c72c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe2440041b0 tx=0x7fe244009260 comp rx=0 tx=0).stop 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 shutdown_connections 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c55d9e50 0x5600c55dc240 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe22ffff640 0 client.26540 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 --2- 172.21.15.103:0/525427772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55c72c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.659+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 >> 172.21.15.103:0/525427772 conn(0x5600c55bb7a0 msgr2=0x5600c55c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.660+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 shutdown_connections 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.660+0000 7fe258ed9740 1 -- 172.21.15.103:0/525427772 wait complete. 2024-08-19T01:43:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.660+0000 7fe258ed9740 20 client.26540 trim_cache size 0 max 16384 2024-08-19T01:43:08.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 Processor -- start 2024-08-19T01:43:08.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 -- start start 2024-08-19T01:43:08.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55bfb30 0x5600c55e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55ca3f0 0x5600c55c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501af2d0 con 0x5600c55bfb30 2024-08-19T01:43:08.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501b0e40 con 0x5600c55ca3f0 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.682+0000 7fe258ed9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501af9a0 con 0x5600c55bf210 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55bfb30 0x5600c55e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55bfb30 0x5600c55e2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50464/0 (socket says 172.21.15.103:50464) 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 -- 172.21.15.103:0/2473537302 learned_addr learned my addr 172.21.15.103:0/2473537302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe254877640 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 -- 172.21.15.103:0/2473537302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 msgr2=0x5600c55e1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55e1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 -- 172.21.15.103:0/2473537302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55ca3f0 msgr2=0x5600c55c0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55ca3f0 0x5600c55c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 -- 172.21.15.103:0/2473537302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600c55bb5d0 con 0x5600c55bfb30 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe24ffff640 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55bfb30 0x5600c55e2030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009610 tx=0x7fe23c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe28250a02a875e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe23a7fc640 1 -- 172.21.15.103:0/2473537302 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0cee60 con 0x5600c55bfb30 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe23a7fc640 1 -- 172.21.15.103:0/2473537302 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe23c09b940 con 0x5600c55bfb30 2024-08-19T01:43:08.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.683+0000 7fe23a7fc640 1 -- 172.21.15.103:0/2473537302 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0a3eb0 con 0x5600c55bfb30 2024-08-19T01:43:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 -- 172.21.15.103:0/2473537302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55bfb30 msgr2=0x5600c55e2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55bfb30 0x5600c55e2030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009610 tx=0x7fe23c0bb040 comp rx=0 tx=0).stop 2024-08-19T01:43:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 -- 172.21.15.103:0/2473537302 shutdown_connections 2024-08-19T01:43:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55ca3f0 0x5600c55c0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55bfb30 0x5600c55e2030 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 --2- 172.21.15.103:0/2473537302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bf210 0x5600c55e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 -- 172.21.15.103:0/2473537302 >> 172.21.15.103:0/2473537302 conn(0x5600c55bb7a0 msgr2=0x5600c55c5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 -- 172.21.15.103:0/2473537302 shutdown_connections 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.684+0000 7fe258ed9740 1 -- 172.21.15.103:0/2473537302 wait complete. 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 -- start start 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 Processor -- start 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501b0770 con 0x5600c55ca3f0 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501b0070 con 0x5600c55bf210 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501ad2b0 con 0x5600c55bfb30 2024-08-19T01:43:08.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe258ed9740 10 client.0 _ll_register_callbacks cb 0x7fe2440040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe255078640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe255078640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50478/0 (socket says 172.21.15.103:50478) 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe255078640 1 -- 172.21.15.103:0/3136762144 learned_addr learned my addr 172.21.15.103:0/3136762144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe24ffff640 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.685+0000 7fe254877640 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe255078640 1 -- 172.21.15.103:0/3136762144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 msgr2=0x5600c55cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe255078640 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe255078640 1 -- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 msgr2=0x5600c55d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe255078640 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe255078640 1 -- 172.21.15.103:0/3136762144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600c55ce5a0 con 0x5600c55ca3f0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe255078640 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55ce060 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe248009eb0 tx=0x7fe248012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4746dd14dd9eb522 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe22ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe22ffff640 1 -- 172.21.15.103:0/3136762144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24802a060 con 0x5600c55ca3f0 2024-08-19T01:43:08.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe22ffff640 1 -- 172.21.15.103:0/3136762144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe24801a410 con 0x5600c55ca3f0 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23c09b940 con 0x5600c55ca3f0 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe22ffff640 1 -- 172.21.15.103:0/3136762144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe248016070 con 0x5600c55ca3f0 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe258ed9740 10 client.17241 fetch_fsmap learned FSMap version 37 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.686+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5600c5644840 con 0x5600c55ca3f0 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe22ffff640 1 -- 172.21.15.103:0/3136762144 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe24801aac0 con 0x5600c55ca3f0 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe22ffff640 1 -- 172.21.15.103:0/3136762144 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fe248024030 con 0x5600c55ca3f0 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe258ed9740 10 client.17241 fetch_fsmap finished waiting for FSMap version 37 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe258ed9740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe258ed9740 20 client.17241 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe258ed9740 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c5644c90 0x5600c56470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe258ed9740 4 client.17241 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe23c09b940 con 0x5600c5644c90 2024-08-19T01:43:08.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe254877640 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c5644c90 0x5600c56470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe22ffff640 10 client.17241 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.687+0000 7fe254877640 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c5644c90 0x5600c56470b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=280ec0b120def28d server_cookie=eb5d205a307d2b93 in_seq=0 out_seq=0 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe22ffff640 1 -- 172.21.15.103:0/3136762144 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe23c09b940 con 0x5600c5644c90 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe22ffff640 10 client.17241 handle_command_reply: tid=0 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 client.17241 shutdown 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 msgr2=0x5600c55ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55ce060 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe248009eb0 tx=0x7fe248012b60 comp rx=0 tx=0).stop 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 shutdown_connections 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5600c5644c90 0x5600c56470b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600c55ca3f0 0x5600c55ce060 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600c55bfb30 0x5600c55cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 --2- 172.21.15.103:0/3136762144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5600c55bf210 0x5600c55d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 >> 172.21.15.103:0/3136762144 conn(0x5600c55bb7a0 msgr2=0x5600c55c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:08.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe22ffff640 0 client.17241 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:08.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 shutdown_connections 2024-08-19T01:43:08.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 1 -- 172.21.15.103:0/3136762144 wait complete. 2024-08-19T01:43:08.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.688+0000 7fe258ed9740 20 client.17241 trim_cache size 0 max 16384 2024-08-19T01:43:08.694 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:43:08.694 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:43:08.694 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:43:08.694 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2400a2b80 msgr2=0x7fe2400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2400a2b80 0x7fe2400a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe244005ab0 tx=0x7fe244005a20 comp rx=0 tx=0).stop 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe250101090 msgr2=0x7fe25019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe250101090 0x7fe25019a8c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00d950 tx=0x7fe23c00de10 comp rx=0 tx=0).stop 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 shutdown_connections 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2400a2b80 0x7fe2400a5040 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe25010e200 0x7fe25019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe250101090 0x7fe25019a8c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 --2- 172.21.15.103:0/2861375609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2501006e0 0x7fe25019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:08.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 >> 172.21.15.103:0/2861375609 conn(0x7fe2500fc500 msgr2=0x7fe2500fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:08.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 shutdown_connections 2024-08-19T01:43:08.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:08.689+0000 7fe256adb640 1 -- 172.21.15.103:0/2861375609 wait complete. 2024-08-19T01:43:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 -- 172.21.15.103:0/2910574267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14103440 msgr2=0x7fad14109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 --2- 172.21.15.103:0/2910574267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14103440 0x7fad14109cd0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009ca0 tx=0x7fad0c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 -- 172.21.15.103:0/2910574267 shutdown_connections 2024-08-19T01:43:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 --2- 172.21.15.103:0/2910574267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14103440 0x7fad14109cd0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 --2- 172.21.15.103:0/2910574267 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14102a80 0x7fad14102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 --2- 172.21.15.103:0/2910574267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad14101890 0x7fad14101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 -- 172.21.15.103:0/2910574267 >> 172.21.15.103:0/2910574267 conn(0x7fad140fd010 msgr2=0x7fad140ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 -- 172.21.15.103:0/2910574267 shutdown_connections 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.023+0000 7fad1bc84640 1 -- 172.21.15.103:0/2910574267 wait complete. 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 Processor -- start 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 -- start start 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14101890 0x7fad14197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14102a80 0x7fad14197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad14103440 0x7fad1419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410d440 con 0x7fad14101890 2024-08-19T01:43:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410d2c0 con 0x7fad14102a80 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1bc84640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410d5c0 con 0x7fad14103440 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad19a20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14101890 0x7fad14197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad19a20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14101890 0x7fad14197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50486/0 (socket says 172.21.15.103:50486) 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad19a20640 1 -- 172.21.15.103:0/1400373660 learned_addr learned my addr 172.21.15.103:0/1400373660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.024+0000 7fad1a221640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad14103440 0x7fad1419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad1921f640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14102a80 0x7fad14197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad19a20640 1 -- 172.21.15.103:0/1400373660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad14103440 msgr2=0x7fad1419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad19a20640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad14103440 0x7fad1419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad19a20640 1 -- 172.21.15.103:0/1400373660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14102a80 msgr2=0x7fad14197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad19a20640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14102a80 0x7fad14197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad19a20640 1 -- 172.21.15.103:0/1400373660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1419f690 con 0x7fad14101890 2024-08-19T01:43:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad1921f640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14102a80 0x7fad14197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad19a20640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14101890 0x7fad14197a70 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d770 tx=0x7fad1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad0affd640 1 -- 172.21.15.103:0/1400373660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10017070 con 0x7fad14101890 2024-08-19T01:43:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad0affd640 1 -- 172.21.15.103:0/1400373660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad10009c30 con 0x7fad14101890 2024-08-19T01:43:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad0affd640 1 -- 172.21.15.103:0/1400373660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10003d00 con 0x7fad14101890 2024-08-19T01:43:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1419f980 con 0x7fad14101890 2024-08-19T01:43:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.025+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1419ff00 con 0x7fad14101890 2024-08-19T01:43:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad0affd640 1 -- 172.21.15.103:0/1400373660 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fad10003ea0 con 0x7fad14101890 2024-08-19T01:43:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 Processor -- start 2024-08-19T01:43:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 -- start start 2024-08-19T01:43:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5dae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5dfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 0x558d5b5e2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d5b5399d0 con 0x558d5b5daa90 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558d5b4e1820 con 0x558d5b5e0460 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1ea44740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558d5b535990 con 0x558d5b5db3d0 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5dae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5dae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50492/0 (socket says 172.21.15.103:50492) 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad1a221640 1 -- 172.21.15.103:0/905098305 learned_addr learned my addr 172.21.15.103:0/905098305 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.028+0000 7fad19a20640 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5dfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad1a221640 1 -- 172.21.15.103:0/905098305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 msgr2=0x558d5b5dfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad1a221640 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5dfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad1a221640 1 -- 172.21.15.103:0/905098305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 msgr2=0x558d5b5e2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad1a221640 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 0x558d5b5e2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad1a221640 1 -- 172.21.15.103:0/905098305 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5b24d210 con 0x558d5b5daa90 2024-08-19T01:43:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad0affd640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fad000a2c30 0x7fad000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad1a221640 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5dae90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0315d0 tx=0x7fad0c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adf4f100a0267d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad19a20640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fad000a2c30 0x7fad000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7fad0affd640 1 -- 172.21.15.103:0/1400373660 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fad100cee30 con 0x7fad14101890 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7facfffff640 1 -- 172.21.15.103:0/905098305 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c031c30 con 0x558d5b5daa90 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7facfffff640 1 -- 172.21.15.103:0/905098305 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0c036040 con 0x558d5b5daa90 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.029+0000 7facfffff640 1 -- 172.21.15.103:0/905098305 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c004780 con 0x558d5b5daa90 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 -- 172.21.15.103:0/905098305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 msgr2=0x558d5b5dae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad19a20640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fad000a2c30 0x7fad000a50f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad100cccd0 tx=0x7fad100d2910 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5dae90 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0315d0 tx=0x7fad0c009a20 comp rx=0 tx=0).stop 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 -- 172.21.15.103:0/905098305 shutdown_connections 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 0x558d5b5e2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5dfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 --2- 172.21.15.103:0/905098305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5dae90 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 -- 172.21.15.103:0/905098305 >> 172.21.15.103:0/905098305 conn(0x558d5b5d1cf0 msgr2=0x558d5b5d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 -- 172.21.15.103:0/905098305 shutdown_connections 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 -- 172.21.15.103:0/905098305 wait complete. 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 -- start start 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.030+0000 7fad1ea44740 1 Processor -- start 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 0x558d5b5e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1ea44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d5b5e33d0 con 0x558d5b5e0460 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1ea44740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558d5b4f5520 con 0x558d5b5db3d0 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1ea44740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558d5b5e3550 con 0x558d5b5daa90 2024-08-19T01:43:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1ea44740 10 client.0 _ll_register_callbacks cb 0x558d5b24ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5ec170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50506/0 (socket says 172.21.15.103:50506) 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 -- 172.21.15.103:0/1694856250 learned_addr learned my addr 172.21.15.103:0/1694856250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 0x558d5b5e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1921f640 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 -- 172.21.15.103:0/1694856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 msgr2=0x558d5b5e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 0x558d5b5e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 -- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 msgr2=0x558d5b5e5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5e5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 -- 172.21.15.103:0/1694856250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5b319620 con 0x558d5b5e0460 2024-08-19T01:43:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7fad1a221640 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5ec170 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fad0c006010 tx=0x7fad0c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e04fc7b3ace9d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.031+0000 7facfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7facfd7fa640 1 -- 172.21.15.103:0/1694856250 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c058060 con 0x558d5b5e0460 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7facfd7fa640 1 -- 172.21.15.103:0/1694856250 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fad0c003970 con 0x558d5b5e0460 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0c036040 con 0x558d5b5e0460 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7facfd7fa640 1 -- 172.21.15.103:0/1694856250 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fad0c003b70 con 0x558d5b5e0460 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 10 client.17253 fetch_fsmap learned FSMap version 37 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558d5b5eef60 con 0x558d5b5e0460 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7facfd7fa640 1 -- 172.21.15.103:0/1694856250 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0c036040 con 0x558d5b5e0460 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7facfd7fa640 1 -- 172.21.15.103:0/1694856250 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fad0c05f480 con 0x558d5b5e0460 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 10 client.17253 fetch_fsmap finished waiting for FSMap version 37 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 20 client.17253 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b5f02b0 0x558d5b5f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 4 client.17253 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.032+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d5b547100 con 0x558d5b5f02b0 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.033+0000 7fad1921f640 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b5f02b0 0x558d5b5f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.033+0000 7fad1921f640 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b5f02b0 0x558d5b5f26a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb388592aada2a7 server_cookie=79ccb031735607fd in_seq=0 out_seq=0 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.033+0000 7facfd7fa640 10 client.17253 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7facfd7fa640 1 -- 172.21.15.103:0/1694856250 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d5b547100 con 0x558d5b5f02b0 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7facfd7fa640 10 client.17253 handle_command_reply: tid=0 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 client.17253 shutdown 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 msgr2=0x558d5b5ec170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5ec170 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fad0c006010 tx=0x7fad0c03d040 comp rx=0 tx=0).stop 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 shutdown_connections 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b5f02b0 0x558d5b5f26a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5ec170 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5e5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 --2- 172.21.15.103:0/1694856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 0x558d5b5e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 >> 172.21.15.103:0/1694856250 conn(0x558d5b5d1cf0 msgr2=0x558d5b5d3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7facfd7fa640 0 client.17253 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 shutdown_connections 2024-08-19T01:43:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 1 -- 172.21.15.103:0/1694856250 wait complete. 2024-08-19T01:43:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.034+0000 7fad1ea44740 20 client.17253 trim_cache size 0 max 16384 2024-08-19T01:43:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.049+0000 7fad1ea44740 1 Processor -- start 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.049+0000 7fad1ea44740 1 -- start start 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 0x558d5b5ecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1ea44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a69d0 con 0x558d5b5e0460 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1ea44740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a8540 con 0x558d5b5db3d0 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1ea44740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a70a0 con 0x558d5b5daa90 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5d2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50514/0 (socket says 172.21.15.103:50514) 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 -- 172.21.15.103:0/739389663 learned_addr learned my addr 172.21.15.103:0/739389663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1921f640 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 -- 172.21.15.103:0/739389663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 msgr2=0x558d5b5ecf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 0x558d5b5ecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 -- 172.21.15.103:0/739389663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 msgr2=0x558d5b5ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 -- 172.21.15.103:0/739389663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5b5db8b0 con 0x558d5b5e0460 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7fad1a221640 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5d2ba0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fad0c02ae00 tx=0x7fad0c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e939f83016fb83 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7facff7fe640 1 -- 172.21.15.103:0/739389663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c031c40 con 0x558d5b5e0460 2024-08-19T01:43:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7facff7fe640 1 -- 172.21.15.103:0/739389663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0c036070 con 0x558d5b5e0460 2024-08-19T01:43:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.050+0000 7facff7fe640 1 -- 172.21.15.103:0/739389663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c05f650 con 0x558d5b5e0460 2024-08-19T01:43:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 -- 172.21.15.103:0/739389663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 msgr2=0x558d5b5d2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5d2ba0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fad0c02ae00 tx=0x7fad0c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 -- 172.21.15.103:0/739389663 shutdown_connections 2024-08-19T01:43:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5e0460 0x558d5b5d2ba0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5db3d0 0x558d5b5ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 --2- 172.21.15.103:0/739389663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5daa90 0x558d5b5ecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 -- 172.21.15.103:0/739389663 >> 172.21.15.103:0/739389663 conn(0x558d5b5d1cf0 msgr2=0x558d5b5ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 -- 172.21.15.103:0/739389663 shutdown_connections 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.051+0000 7fad1ea44740 1 -- 172.21.15.103:0/739389663 wait complete. 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 -- start start 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 Processor -- start 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5ea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 0x558d5b5eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a7e70 con 0x558d5b5daa90 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a7770 con 0x558d5b5e0460 2024-08-19T01:43:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad141a49b0 con 0x558d5b5db3d0 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1ea44740 10 client.0 _ll_register_callbacks cb 0x7fad100d27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1a221640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5e9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50518/0 (socket says 172.21.15.103:50518) 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1a221640 1 -- 172.21.15.103:0/11081242 learned_addr learned my addr 172.21.15.103:0/11081242 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.052+0000 7fad1921f640 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1a221640 1 -- 172.21.15.103:0/11081242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 msgr2=0x558d5b5ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1a221640 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1a221640 1 -- 172.21.15.103:0/11081242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 msgr2=0x558d5b5eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1a221640 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 0x558d5b5eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1a221640 1 -- 172.21.15.103:0/11081242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5b5eb7f0 con 0x558d5b5daa90 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1a221640 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5e9ce0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0317f0 tx=0x7fad0c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c247ed6cf771adb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7facfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7facfd7fa640 1 -- 172.21.15.103:0/11081242 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c058040 con 0x558d5b5daa90 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7facfd7fa640 1 -- 172.21.15.103:0/11081242 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fad0c05fa70 con 0x558d5b5daa90 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0c036070 con 0x558d5b5daa90 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7facfd7fa640 1 -- 172.21.15.103:0/11081242 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fad0c05fc70 con 0x558d5b5daa90 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1ea44740 10 client.17259 fetch_fsmap learned FSMap version 37 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558d5b65b2a0 con 0x558d5b5daa90 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.053+0000 7facfd7fa640 1 -- 172.21.15.103:0/11081242 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0c036070 con 0x558d5b5daa90 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7facfd7fa640 1 -- 172.21.15.103:0/11081242 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 37) ==== 8366+0+0 (secure 0 0 0) 0x7fad0c068460 con 0x558d5b5daa90 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1ea44740 10 client.17259 fetch_fsmap finished waiting for FSMap version 37 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1ea44740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1ea44740 20 client.17259 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1ea44740 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b65bb30 0x558d5b65df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1ea44740 4 client.17259 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:43:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d5b5ccfc0 con 0x558d5b65bb30 2024-08-19T01:43:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1921f640 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b65bb30 0x558d5b65df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7facfd7fa640 10 client.17259 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.054+0000 7fad1921f640 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b65bb30 0x558d5b65df50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf64bd1d20268a server_cookie=96189754788e616e in_seq=0 out_seq=0 2024-08-19T01:43:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7facfd7fa640 1 -- 172.21.15.103:0/11081242 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d5b5ccfc0 con 0x558d5b65bb30 2024-08-19T01:43:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7facfd7fa640 10 client.17259 handle_command_reply: tid=0 2024-08-19T01:43:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 client.17259 shutdown 2024-08-19T01:43:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 msgr2=0x558d5b5e9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5e9ce0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0317f0 tx=0x7fad0c009540 comp rx=0 tx=0).stop 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 shutdown_connections 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x558d5b65bb30 0x558d5b65df50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d5b5e0460 0x558d5b5eb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558d5b5db3d0 0x558d5b5ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7facfd7fa640 0 client.17259 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 --2- 172.21.15.103:0/11081242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d5b5daa90 0x558d5b5e9ce0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 >> 172.21.15.103:0/11081242 conn(0x558d5b5d1cf0 msgr2=0x558d5b5d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 shutdown_connections 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 1 -- 172.21.15.103:0/11081242 wait complete. 2024-08-19T01:43:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.061+0000 7fad1ea44740 20 client.17259 trim_cache size 0 max 16384 2024-08-19T01:43:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:43:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:43:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:43:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:43:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fad000a2c30 msgr2=0x7fad000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fad000a2c30 0x7fad000a50f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad100cccd0 tx=0x7fad100d2910 comp rx=0 tx=0).stop 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14101890 msgr2=0x7fad14197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14101890 0x7fad14197a70 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d770 tx=0x7fad1000dc30 comp rx=0 tx=0).stop 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 shutdown_connections 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fad000a2c30 0x7fad000a50f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad14103440 0x7fad1419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14102a80 0x7fad14197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 --2- 172.21.15.103:0/1400373660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad14101890 0x7fad14197a70 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 >> 172.21.15.103:0/1400373660 conn(0x7fad140fd010 msgr2=0x7fad14105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 shutdown_connections 2024-08-19T01:43:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:09.062+0000 7fad1bc84640 1 -- 172.21.15.103:0/1400373660 wait complete. 2024-08-19T01:43:09.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:09 smithi103 ceph-mon[18436]: pgmap v560: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:43:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:09 smithi190 ceph-mon[22032]: pgmap v560: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:43:09.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:09 smithi146 ceph-mon[22098]: pgmap v560: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 24 op/s 2024-08-19T01:43:09.602 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:43:09.602 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:43:09.893 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:11.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:10 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1922676168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:11.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:10 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1922676168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:11.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 -- 172.21.15.103:0/3501792186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14107fd0 msgr2=0x7f1d1410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:11.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 --2- 172.21.15.103:0/3501792186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14107fd0 0x7f1d1410e970 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009cd0 tx=0x7f1d0c02a8e0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 -- 172.21.15.103:0/3501792186 shutdown_connections 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 --2- 172.21.15.103:0/3501792186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14107fd0 0x7f1d1410e970 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 --2- 172.21.15.103:0/3501792186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d14103410 0x7f1d141078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 --2- 172.21.15.103:0/3501792186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d14102a60 0x7f1d14102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 -- 172.21.15.103:0/3501792186 >> 172.21.15.103:0/3501792186 conn(0x7f1d140fc500 msgr2=0x7f1d140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 -- 172.21.15.103:0/3501792186 shutdown_connections 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.534+0000 7f1d18c47640 1 -- 172.21.15.103:0/3501792186 wait complete. 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 Processor -- start 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 -- start start 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14102a60 0x7f1d141981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d14103410 0x7f1d14198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d14107fd0 0x7f1d1419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14111c80 con 0x7f1d14102a60 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14111b00 con 0x7f1d14103410 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.535+0000 7f1d18c47640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14111e00 con 0x7f1d14107fd0 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14102a60 0x7f1d141981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:11.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14102a60 0x7f1d141981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50540/0 (socket says 172.21.15.103:50540) 2024-08-19T01:43:11.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:10 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1922676168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:43:11.293 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:43:11.293 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":37,"btime":"2024-08-19T01:43:02:012811+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:43:01.573672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 -- 172.21.15.103:0/1922676168 learned_addr learned my addr 172.21.15.103:0/1922676168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d0bfff640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d14103410 0x7f1d14198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 -- 172.21.15.103:0/1922676168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d14107fd0 msgr2=0x7f1d1419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d14107fd0 0x7f1d1419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 -- 172.21.15.103:0/1922676168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d14103410 msgr2=0x7f1d14198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d14103410 0x7f1d14198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 -- 172.21.15.103:0/1922676168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1419d180 con 0x7f1d14102a60 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d1259c640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14102a60 0x7f1d141981d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400c970 tx=0x7f1d0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d0b7fe640 1 -- 172.21.15.103:0/1922676168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04007bb0 con 0x7f1d14102a60 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1419d470 con 0x7f1d14102a60 2024-08-19T01:43:11.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d0b7fe640 1 -- 172.21.15.103:0/1922676168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d04007d50 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d0b7fe640 1 -- 172.21.15.103:0/1922676168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400f450 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.536+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d141a4d50 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.538+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d14102e40 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.539+0000 7f1d0b7fe640 1 -- 172.21.15.103:0/1922676168 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1d04004f60 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.540+0000 7f1d0b7fe640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1d000a2b80 0x7f1d000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.540+0000 7f1d0bfff640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1d000a2b80 0x7f1d000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.540+0000 7f1d0b7fe640 1 -- 172.21.15.103:0/1922676168 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1d040cedb0 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.540+0000 7f1d0bfff640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1d000a2b80 0x7f1d000a5040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d140feb50 tx=0x7f1cfc00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.544+0000 7f1d0b7fe640 1 -- 172.21.15.103:0/1922676168 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d04014030 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.809+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d1410a010 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.810+0000 7f1d0b7fe640 1 -- 172.21.15.103:0/1922676168 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11990 (secure 0 0 0) 0x7f1d04097020 con 0x7f1d14102a60 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 37 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1d000a2b80 msgr2=0x7f1d000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1d000a2b80 0x7f1d000a5040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d140feb50 tx=0x7f1cfc00b390 comp rx=0 tx=0).stop 2024-08-19T01:43:11.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14102a60 msgr2=0x7f1d141981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14102a60 0x7f1d141981d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400c970 tx=0x7f1d0400ce30 comp rx=0 tx=0).stop 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 shutdown_connections 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1d000a2b80 0x7f1d000a5040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1d14107fd0 0x7f1d1419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d14103410 0x7f1d14198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 --2- 172.21.15.103:0/1922676168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d14102a60 0x7f1d141981d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 >> 172.21.15.103:0/1922676168 conn(0x7f1d140fc500 msgr2=0x7f1d14105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 shutdown_connections 2024-08-19T01:43:11.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:10.812+0000 7f1d18c47640 1 -- 172.21.15.103:0/1922676168 wait complete. 2024-08-19T01:43:12.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:11 smithi190 ceph-mon[22032]: pgmap v561: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:43:12.201 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:11 smithi146 ceph-mon[22098]: pgmap v561: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:43:12.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:11 smithi103 ceph-mon[18436]: pgmap v561: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-08-19T01:43:13.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:12 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:43:13.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:12 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:43:13.281 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:12 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:43:13.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:43:13.666 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:14.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:13 smithi103 ceph-mon[18436]: pgmap v562: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-08-19T01:43:14.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:13 smithi190 ceph-mon[22032]: pgmap v562: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-08-19T01:43:14.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:13 smithi146 ceph-mon[22098]: pgmap v562: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 -- 172.21.15.103:0/1107885849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8108030 msgr2=0x7f66b810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 --2- 172.21.15.103:0/1107885849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8108030 0x7f66b810e9a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009cd0 tx=0x7f66b002a8d0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 -- 172.21.15.103:0/1107885849 shutdown_connections 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 --2- 172.21.15.103:0/1107885849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8108030 0x7f66b810e9a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 --2- 172.21.15.103:0/1107885849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66b81034b0 0x7f66b81078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 --2- 172.21.15.103:0/1107885849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b8102b00 0x7f66b8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 -- 172.21.15.103:0/1107885849 >> 172.21.15.103:0/1107885849 conn(0x7f66b80fc500 msgr2=0x7f66b80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 -- 172.21.15.103:0/1107885849 shutdown_connections 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.197+0000 7f66bda53640 1 -- 172.21.15.103:0/1107885849 wait complete. 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 Processor -- start 2024-08-19T01:43:14.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 -- start start 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8102b00 0x7f66b8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b81034b0 0x7f66b8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66b8108030 0x7f66b819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8111cb0 con 0x7f66b8102b00 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8111b30 con 0x7f66b8108030 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66bda53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8111e30 con 0x7f66b81034b0 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8102b00 0x7f66b8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8102b00 0x7f66b8198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59648/0 (socket says 172.21.15.103:59648) 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.198+0000 7f66b6ffd640 1 -- 172.21.15.103:0/2910551168 learned_addr learned my addr 172.21.15.103:0/2910551168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b81034b0 0x7f66b8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b77fe640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66b8108030 0x7f66b819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b6ffd640 1 -- 172.21.15.103:0/2910551168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b81034b0 msgr2=0x7f66b8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b6ffd640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b81034b0 0x7f66b8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b6ffd640 1 -- 172.21.15.103:0/2910551168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66b8108030 msgr2=0x7f66b819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b6ffd640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66b8108030 0x7f66b819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b6ffd640 1 -- 172.21.15.103:0/2910551168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b819d0c0 con 0x7f66b8102b00 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b77fe640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66b8108030 0x7f66b819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b81034b0 0x7f66b8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66b6ffd640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8102b00 0x7f66b8198110 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c920 tx=0x7f66a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66bca51640 1 -- 172.21.15.103:0/2910551168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8007c70 con 0x7f66b8102b00 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b819d3b0 con 0x7f66b8102b00 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66bca51640 1 -- 172.21.15.103:0/2910551168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66a8007e10 con 0x7f66b8102b00 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66bca51640 1 -- 172.21.15.103:0/2910551168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8018820 con 0x7f66b8102b00 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.199+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b81a4c90 con 0x7f66b8102b00 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 Processor -- start 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 -- start start 2024-08-19T01:43:14.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bca51640 1 -- 172.21.15.103:0/2910551168 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f66a80189c0 con 0x7f66b8102b00 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa330270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa295a40 con 0x55e0aa3307b0 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa22d9d0 con 0x55e0aa32fe90 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66bfe51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa291a00 con 0x55e0aa33f5e0 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.202+0000 7f66b6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa330270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa330270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:60164/0 (socket says 172.21.15.103:60164) 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 -- 172.21.15.103:0/4006612741 learned_addr learned my addr 172.21.15.103:0/4006612741 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b67fc640 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 -- 172.21.15.103:0/4006612741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 msgr2=0x55e0aa3419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 -- 172.21.15.103:0/4006612741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 msgr2=0x55e0aa33f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 -- 172.21.15.103:0/4006612741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0aa234650 con 0x55e0aa32fe90 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b6ffd640 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66bca51640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f66a00a2b80 0x7f66a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b6ffd640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f66a00a2b80 0x7f66a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66b77fe640 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa330270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009a20 tx=0x7f66b0005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd45c567ea5b60d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f66bca51640 1 -- 172.21.15.103:0/2910551168 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f66a80cf230 con 0x7f66b8102b00 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.203+0000 7f669dffb640 1 -- 172.21.15.103:0/4006612741 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0031420 con 0x55e0aa32fe90 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f669dffb640 1 -- 172.21.15.103:0/4006612741 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0040070 con 0x55e0aa32fe90 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f669dffb640 1 -- 172.21.15.103:0/4006612741 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0004450 con 0x55e0aa32fe90 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 -- 172.21.15.103:0/4006612741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 msgr2=0x55e0aa330270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa330270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009a20 tx=0x7f66b0005fa0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66b6ffd640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f66a00a2b80 0x7f66a00a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f66a80cd0d0 tx=0x7f66a80d2d10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:14.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 -- 172.21.15.103:0/4006612741 shutdown_connections 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 --2- 172.21.15.103:0/4006612741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa330270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 -- 172.21.15.103:0/4006612741 >> 172.21.15.103:0/4006612741 conn(0x55e0aa32c580 msgr2=0x55e0aa32c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.204+0000 7f66bfe51740 1 -- 172.21.15.103:0/4006612741 shutdown_connections 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 -- 172.21.15.103:0/4006612741 wait complete. 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 -- start start 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 Processor -- start 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa32fe90 0x55e0aa33c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 0x55e0aa33cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa3421b0 con 0x55e0aa3307b0 2024-08-19T01:43:14.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa250b20 con 0x55e0aa33f5e0 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.205+0000 7f66bfe51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e0aa342330 con 0x55e0aa32fe90 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66bfe51740 10 client.0 _ll_register_callbacks cb 0x55e0aa24ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59658/0 (socket says 172.21.15.103:59658) 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 -- 172.21.15.103:0/110829334 learned_addr learned my addr 172.21.15.103:0/110829334 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b77fe640 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 0x55e0aa33cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b77fe640 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa32fe90 0x55e0aa33c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 -- 172.21.15.103:0/110829334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa32fe90 msgr2=0x55e0aa33c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa32fe90 0x55e0aa33c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 -- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 msgr2=0x55e0aa33cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 0x55e0aa33cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 -- 172.21.15.103:0/110829334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0aa0073c0 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66b67fc640 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33ca60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00bde0 tx=0x7f66ac012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4531b8fa7208f57c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f6696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f6696ffd640 1 -- 172.21.15.103:0/110829334 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac004230 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f6696ffd640 1 -- 172.21.15.103:0/110829334 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f66ac0043d0 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.206+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b0040070 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f6696ffd640 1 -- 172.21.15.103:0/110829334 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f66ac0106a0 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 10 client.17277 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e0aa34deb0 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f6696ffd640 1 -- 172.21.15.103:0/110829334 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0040070 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f6696ffd640 1 -- 172.21.15.103:0/110829334 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f66ac003ab0 con 0x55e0aa3307b0 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 10 client.17277 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 10 client.17277 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 20 client.17277 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:14.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 4 client.17277 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0aa2a2fb0 con 0x55e0aa34f3c0 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.207+0000 7f66b77fe640 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.208+0000 7f6696ffd640 10 client.17277 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.208+0000 7f66b77fe640 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3517e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=729c2b038b28f751 server_cookie=ae1da55980315829 in_seq=0 out_seq=0 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f6696ffd640 1 -- 172.21.15.103:0/110829334 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0aa2a2fb0 con 0x55e0aa34f3c0 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f6696ffd640 10 client.17277 handle_command_reply: tid=0 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 client.17277 shutdown 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 msgr2=0x55e0aa33ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33ca60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00bde0 tx=0x7f66ac012820 comp rx=0 tx=0).stop 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 shutdown_connections 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3517e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 0x55e0aa33cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f6696ffd640 0 client.17277 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:43:14.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa33ca60 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 --2- 172.21.15.103:0/110829334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa32fe90 0x55e0aa33c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 >> 172.21.15.103:0/110829334 conn(0x55e0aa32c580 msgr2=0x55e0aa3322e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:14.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 shutdown_connections 2024-08-19T01:43:14.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 1 -- 172.21.15.103:0/110829334 wait complete. 2024-08-19T01:43:14.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.209+0000 7f66bfe51740 20 client.17277 trim_cache size 0 max 16384 2024-08-19T01:43:14.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.229+0000 7f66bfe51740 1 Processor -- start 2024-08-19T01:43:14.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66bfe51740 1 -- start start 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa33cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa3570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66bfe51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81aaf50 con 0x55e0aa3307b0 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66bfe51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81acb20 con 0x55e0aa32fe90 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66bfe51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81ab620 con 0x55e0aa33f5e0 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa3570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa3570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59662/0 (socket says 172.21.15.103:59662) 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 -- 172.21.15.103:0/2801404160 learned_addr learned my addr 172.21.15.103:0/2801404160 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b77fe640 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b77fe640 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa33cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 -- 172.21.15.103:0/2801404160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 msgr2=0x55e0aa3575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 -- 172.21.15.103:0/2801404160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 msgr2=0x55e0aa33cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:14.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa33cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.230+0000 7f66b67fc640 1 -- 172.21.15.103:0/2801404160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0aa351b50 con 0x55e0aa3307b0 2024-08-19T01:43:14.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.231+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa3570b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c010 tx=0x7f66ac005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69711f515dbb7676 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:14.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.231+0000 7f669d7fa640 1 -- 172.21.15.103:0/2801404160 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac01b990 con 0x55e0aa3307b0 2024-08-19T01:43:14.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.231+0000 7f669d7fa640 1 -- 172.21.15.103:0/2801404160 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66ac003fd0 con 0x55e0aa3307b0 2024-08-19T01:43:14.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.231+0000 7f669d7fa640 1 -- 172.21.15.103:0/2801404160 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac010bc0 con 0x55e0aa3307b0 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 -- 172.21.15.103:0/2801404160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 msgr2=0x55e0aa3570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa3570b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c010 tx=0x7f66ac005a70 comp rx=0 tx=0).stop 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 -- 172.21.15.103:0/2801404160 shutdown_connections 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa33f5e0 0x55e0aa3575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa3307b0 0x55e0aa3570b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2801404160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa32fe90 0x55e0aa33cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 -- 172.21.15.103:0/2801404160 >> 172.21.15.103:0/2801404160 conn(0x55e0aa32c580 msgr2=0x55e0aa33adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 -- 172.21.15.103:0/2801404160 shutdown_connections 2024-08-19T01:43:14.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 -- 172.21.15.103:0/2801404160 wait complete. 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.232+0000 7f66bfe51740 1 -- start start 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 1 Processor -- start 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa32fe90 0x55e0aa342580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa3307b0 0x55e0aa342ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 0x55e0aa343000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81ac450 con 0x55e0aa32fe90 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81abcf0 con 0x55e0aa33f5e0 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66b81a8f30 con 0x55e0aa3307b0 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66bfe51740 10 client.0 _ll_register_callbacks cb 0x7f66ac00ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa32fe90 0x55e0aa342580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa32fe90 0x55e0aa342580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59666/0 (socket says 172.21.15.103:59666) 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 -- 172.21.15.103:0/2870824661 learned_addr learned my addr 172.21.15.103:0/2870824661 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa3307b0 0x55e0aa342ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 -- 172.21.15.103:0/2870824661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa3307b0 msgr2=0x55e0aa342ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa3307b0 0x55e0aa342ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 -- 172.21.15.103:0/2870824661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 msgr2=0x55e0aa343000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 0x55e0aa343000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.233+0000 7f66b77fe640 1 -- 172.21.15.103:0/2870824661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0aa3b8140 con 0x55e0aa32fe90 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f66b77fe640 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa32fe90 0x55e0aa342580 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009620 tx=0x7f66b0031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f747c30336808 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f6696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f6696ffd640 1 -- 172.21.15.103:0/2870824661 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0059060 con 0x55e0aa32fe90 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f6696ffd640 1 -- 172.21.15.103:0/2870824661 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f66b0003970 con 0x55e0aa32fe90 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ac003fd0 con 0x55e0aa32fe90 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f6696ffd640 1 -- 172.21.15.103:0/2870824661 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f66b0003b70 con 0x55e0aa32fe90 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f66bfe51740 10 client.17283 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e0aa3b9420 con 0x55e0aa32fe90 2024-08-19T01:43:14.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f6696ffd640 1 -- 172.21.15.103:0/2870824661 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66ac003fd0 con 0x55e0aa32fe90 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f6696ffd640 1 -- 172.21.15.103:0/2870824661 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f66b0030590 con 0x55e0aa32fe90 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f66bfe51740 10 client.17283 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.234+0000 7f66bfe51740 10 client.17283 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f66bfe51740 20 client.17283 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f66bfe51740 4 client.17283 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0aa29d4c0 con 0x55e0aa34f3c0 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f66b67fc640 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3bb770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f81c65f9c06838e7 server_cookie=ba293c98e9f2255 in_seq=0 out_seq=0 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f6696ffd640 10 client.17283 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f6696ffd640 1 -- 172.21.15.103:0/2870824661 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0aa29d4c0 con 0x55e0aa34f3c0 2024-08-19T01:43:14.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f6696ffd640 10 client.17283 handle_command_reply: tid=0 2024-08-19T01:43:14.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.235+0000 7f66bfe51740 1 client.17283 shutdown 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa32fe90 msgr2=0x55e0aa342580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa32fe90 0x55e0aa342580 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009620 tx=0x7f66b0031ca0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 shutdown_connections 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55e0aa34f3c0 0x55e0aa3bb770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f6696ffd640 0 client.17283 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e0aa33f5e0 0x55e0aa343000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e0aa3307b0 0x55e0aa342ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 --2- 172.21.15.103:0/2870824661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0aa32fe90 0x55e0aa342580 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 >> 172.21.15.103:0/2870824661 conn(0x55e0aa32c580 msgr2=0x55e0aa331a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 shutdown_connections 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 1 -- 172.21.15.103:0/2870824661 wait complete. 2024-08-19T01:43:14.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.236+0000 7f66bfe51740 20 client.17283 trim_cache size 0 max 16384 2024-08-19T01:43:14.242 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:43:14.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f66a00a2b80 msgr2=0x7f66a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f66a00a2b80 0x7f66a00a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f66a80cd0d0 tx=0x7f66a80d2d10 comp rx=0 tx=0).stop 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8102b00 msgr2=0x7f66b8198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8102b00 0x7f66b8198110 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c920 tx=0x7f66a800cde0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 shutdown_connections 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f66a00a2b80 0x7f66a00a5040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f66b8108030 0x7f66b819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b81034b0 0x7f66b8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 --2- 172.21.15.103:0/2910551168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b8102b00 0x7f66b8198110 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 >> 172.21.15.103:0/2910551168 conn(0x7f66b80fc500 msgr2=0x7f66b81060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 shutdown_connections 2024-08-19T01:43:14.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:14.237+0000 7f66bda53640 1 -- 172.21.15.103:0/2910551168 wait complete. 2024-08-19T01:43:15.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:43:15.830 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:16.248 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:15 smithi103 ceph-mon[18436]: pgmap v563: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:43:16.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 -- 172.21.15.103:0/1313653503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c101150 msgr2=0x7f672c10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 --2- 172.21.15.103:0/1313653503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c101150 0x7f672c10d680 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6724009cd0 tx=0x7f672402a8e0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 -- 172.21.15.103:0/1313653503 shutdown_connections 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 --2- 172.21.15.103:0/1313653503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c101150 0x7f672c10d680 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 --2- 172.21.15.103:0/1313653503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f672c1007b0 0x7f672c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 --2- 172.21.15.103:0/1313653503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1067b0 0x7f672c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 -- 172.21.15.103:0/1313653503 >> 172.21.15.103:0/1313653503 conn(0x7f672c0fc4f0 msgr2=0x7f672c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 -- 172.21.15.103:0/1313653503 shutdown_connections 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.340+0000 7f673160d640 1 -- 172.21.15.103:0/1313653503 wait complete. 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.341+0000 7f673160d640 1 Processor -- start 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.341+0000 7f673160d640 1 -- start start 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f673160d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1007b0 0x7f672c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f673160d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f672c101150 0x7f672c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f673160d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c1067b0 0x7f672c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f673160d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f672c110a60 con 0x7f672c1067b0 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f673160d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1108e0 con 0x7f672c101150 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f673160d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f672c110be0 con 0x7f672c1007b0 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c1067b0 0x7f672c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c1067b0 0x7f672c19b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59696/0 (socket says 172.21.15.103:59696) 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 -- 172.21.15.103:0/1736185855 learned_addr learned my addr 172.21.15.103:0/1736185855 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672a7fc640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f672c101150 0x7f672c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672affd640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1007b0 0x7f672c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 -- 172.21.15.103:0/1736185855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1007b0 msgr2=0x7f672c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1007b0 0x7f672c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 -- 172.21.15.103:0/1736185855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f672c101150 msgr2=0x7f672c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f672c101150 0x7f672c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 -- 172.21.15.103:0/1736185855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c19bf30 con 0x7f672c1067b0 2024-08-19T01:43:16.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f672b7fe640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c1067b0 0x7f672c19b850 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6724005cc0 tx=0x7f6724003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.342+0000 7f6713fff640 1 -- 172.21.15.103:0/1736185855 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724004450 con 0x7f672c1067b0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.343+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c19c1c0 con 0x7f672c1067b0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.343+0000 7f6713fff640 1 -- 172.21.15.103:0/1736185855 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6724031c90 con 0x7f672c1067b0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.343+0000 7f6713fff640 1 -- 172.21.15.103:0/1736185855 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672403b400 con 0x7f672c1067b0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.343+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c1a3b00 con 0x7f672c1067b0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.345+0000 7f6713fff640 1 -- 172.21.15.103:0/1736185855 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6724004680 con 0x7f672c1067b0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6713fff640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67140a2b80 0x7f67140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 Processor -- start 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 -- start start 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6713fff640 1 -- 172.21.15.103:0/1736185855 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f672402e030 con 0x7f672c1067b0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f672affd640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67140a2b80 0x7f67140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4260c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c42586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c426ba70 0x55f3c426de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3c41bda40 con 0x55f3c4258250 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f3c41559d0 con 0x55f3c426ba70 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f6733a0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f3c41b9a00 con 0x55f3c4257e70 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f672a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c42586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f672a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c42586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59706/0 (socket says 172.21.15.103:59706) 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.346+0000 7f672a7fc640 1 -- 172.21.15.103:0/2388517927 learned_addr learned my addr 172.21.15.103:0/2388517927 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672affd640 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4260c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672b7fe640 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c426ba70 0x55f3c426de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672a7fc640 1 -- 172.21.15.103:0/2388517927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 msgr2=0x55f3c4260c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672a7fc640 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4260c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672a7fc640 1 -- 172.21.15.103:0/2388517927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c426ba70 msgr2=0x55f3c426de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672affd640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67140a2b80 0x7f67140a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f671c005e00 tx=0x7f671c005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:16.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672a7fc640 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c426ba70 0x55f3c426de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672a7fc640 1 -- 172.21.15.103:0/2388517927 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c415c650 con 0x55f3c4258250 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672b7fe640 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c426ba70 0x55f3c426de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f672a7fc640 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c42586b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f672000bcb0 tx=0x7f67200107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae7e387b07a30d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f67117fa640 1 -- 172.21.15.103:0/2388517927 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720004030 con 0x55f3c4258250 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f67117fa640 1 -- 172.21.15.103:0/2388517927 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67200041d0 con 0x55f3c4258250 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.347+0000 7f67117fa640 1 -- 172.21.15.103:0/2388517927 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200116a0 con 0x55f3c4258250 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 -- 172.21.15.103:0/2388517927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 msgr2=0x55f3c42586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c42586b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f672000bcb0 tx=0x7f67200107a0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 -- 172.21.15.103:0/2388517927 shutdown_connections 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c426ba70 0x55f3c426de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c42586b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2388517927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4260c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 -- 172.21.15.103:0/2388517927 >> 172.21.15.103:0/2388517927 conn(0x55f3c4254580 msgr2=0x55f3c4254960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 -- 172.21.15.103:0/2388517927 shutdown_connections 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 -- 172.21.15.103:0/2388517927 wait complete. 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 -- start start 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.348+0000 7f6733a0b740 1 Processor -- start 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c425e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c425e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c426ba70 0x55f3c425ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f6733a0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3c426e5e0 con 0x55f3c4258250 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f6733a0b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f3c4178b20 con 0x55f3c4257e70 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c425e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f6733a0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f3c426e760 con 0x55f3c426ba70 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c425e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59722/0 (socket says 172.21.15.103:59722) 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 -- 172.21.15.103:0/2718318999 learned_addr learned my addr 172.21.15.103:0/2718318999 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f6733a0b740 10 client.0 _ll_register_callbacks cb 0x55f3c4176bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672affd640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c425e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672b7fe640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c426ba70 0x55f3c425ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 -- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c426ba70 msgr2=0x55f3c425ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c426ba70 0x55f3c425ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 -- 172.21.15.103:0/2718318999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 msgr2=0x55f3c425e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c425e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672a7fc640 1 -- 172.21.15.103:0/2718318999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c3f2f3c0 con 0x55f3c4258250 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.349+0000 7f672b7fe640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c426ba70 0x55f3c425ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f672a7fc640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c425e680 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f67200041a0 tx=0x7f6720003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43316f0ab81dc40e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:16.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f670eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f670eff5640 1 -- 172.21.15.103:0/2718318999 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672002a060 con 0x55f3c4258250 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f670eff5640 1 -- 172.21.15.103:0/2718318999 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6720004380 con 0x55f3c4258250 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67200041d0 con 0x55f3c4258250 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f670eff5640 1 -- 172.21.15.103:0/2718318999 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f67200035f0 con 0x55f3c4258250 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 10 client.17295 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f3c427a1e0 con 0x55f3c4258250 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f670eff5640 1 -- 172.21.15.103:0/2718318999 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67200041d0 con 0x55f3c4258250 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f670eff5640 1 -- 172.21.15.103:0/2718318999 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f6720004a40 con 0x55f3c4258250 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 10 client.17295 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 10 client.17295 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 20 client.17295 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c427b640 0x55f3c427da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 4 client.17295 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.350+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c41cafb0 con 0x55f3c427b640 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.351+0000 7f672affd640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c427b640 0x55f3c427da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.351+0000 7f672affd640 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c427b640 0x55f3c427da30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff3e92024cd1542d server_cookie=175fae5337dd7844 in_seq=0 out_seq=0 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.351+0000 7f670eff5640 10 client.17295 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f670eff5640 1 -- 172.21.15.103:0/2718318999 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3c41cafb0 con 0x55f3c427b640 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f670eff5640 10 client.17295 handle_command_reply: tid=0 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 client.17295 shutdown 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 msgr2=0x55f3c425e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c425e680 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f67200041a0 tx=0x7f6720003cc0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 shutdown_connections 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c427b640 0x55f3c427da30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c426ba70 0x55f3c425ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c4258250 0x55f3c425e680 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 --2- 172.21.15.103:0/2718318999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c425e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f670eff5640 0 client.17295 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:43:16.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 >> 172.21.15.103:0/2718318999 conn(0x55f3c4254580 msgr2=0x55f3c425a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 shutdown_connections 2024-08-19T01:43:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 1 -- 172.21.15.103:0/2718318999 wait complete. 2024-08-19T01:43:16.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.352+0000 7f6733a0b740 20 client.17295 trim_cache size 0 max 16384 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 Processor -- start 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 -- start start 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4253aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4258250 0x55f3c425f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c425fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1aaea0 con 0x55f3c426ba70 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1aca70 con 0x55f3c4258250 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f6733a0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1ab570 con 0x55f3c4257e70 2024-08-19T01:43:16.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f672b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c425fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f672b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c425fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59730/0 (socket says 172.21.15.103:59730) 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.367+0000 7f672b7fe640 1 -- 172.21.15.103:0/3247471990 learned_addr learned my addr 172.21.15.103:0/3247471990 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672affd640 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4253aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672a7fc640 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4258250 0x55f3c425f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672b7fe640 1 -- 172.21.15.103:0/3247471990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 msgr2=0x55f3c4253aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672b7fe640 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4253aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672b7fe640 1 -- 172.21.15.103:0/3247471990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4258250 msgr2=0x55f3c425f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672b7fe640 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4258250 0x55f3c425f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672b7fe640 1 -- 172.21.15.103:0/3247471990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c4277840 con 0x55f3c426ba70 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672a7fc640 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4258250 0x55f3c425f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f672b7fe640 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c425fdd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f67240b9480 tx=0x7f67240a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ca759590b3d385 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f6710ff9640 1 -- 172.21.15.103:0/3247471990 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240d5020 con 0x55f3c426ba70 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f6710ff9640 1 -- 172.21.15.103:0/3247471990 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6724036040 con 0x55f3c426ba70 2024-08-19T01:43:16.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.368+0000 7f6710ff9640 1 -- 172.21.15.103:0/3247471990 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240bbf00 con 0x55f3c426ba70 2024-08-19T01:43:16.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 -- 172.21.15.103:0/3247471990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 msgr2=0x55f3c425fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c425fdd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f67240b9480 tx=0x7f67240a7040 comp rx=0 tx=0).stop 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 -- 172.21.15.103:0/3247471990 shutdown_connections 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c425fdd0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4258250 0x55f3c425f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 --2- 172.21.15.103:0/3247471990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4257e70 0x55f3c4253aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 -- 172.21.15.103:0/3247471990 >> 172.21.15.103:0/3247471990 conn(0x55f3c4254580 msgr2=0x55f3c426db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 -- 172.21.15.103:0/3247471990 shutdown_connections 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.369+0000 7f6733a0b740 1 -- 172.21.15.103:0/3247471990 wait complete. 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 -- start start 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 Processor -- start 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c426ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4258250 0x55f3c426f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c42e4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1ac3a0 con 0x55f3c426ba70 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1abc40 con 0x55f3c4257e70 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1a8e80 con 0x55f3c4258250 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f6733a0b740 10 client.0 _ll_register_callbacks cb 0x7f672000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c42e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c42e4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59746/0 (socket says 172.21.15.103:59746) 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672b7fe640 1 -- 172.21.15.103:0/24990656 learned_addr learned my addr 172.21.15.103:0/24990656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672affd640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c426ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672a7fc640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4258250 0x55f3c426f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672b7fe640 1 -- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4258250 msgr2=0x55f3c426f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672b7fe640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4258250 0x55f3c426f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672b7fe640 1 -- 172.21.15.103:0/24990656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 msgr2=0x55f3c426ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.370+0000 7f672b7fe640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c426ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f672b7fe640 1 -- 172.21.15.103:0/24990656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c42e4560 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f672affd640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c426ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f672b7fe640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c42e4020 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6724004630 tx=0x7f6724009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9169ab1e1b1e315f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f670eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f670eff5640 1 -- 172.21.15.103:0/24990656 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724086060 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f670eff5640 1 -- 172.21.15.103:0/24990656 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f67240305a0 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6724036040 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f670eff5640 1 -- 172.21.15.103:0/24990656 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f67240d5040 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f6733a0b740 10 client.17301 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f3c42e5720 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.371+0000 7f670eff5640 1 -- 172.21.15.103:0/24990656 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6724036040 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f670eff5640 1 -- 172.21.15.103:0/24990656 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f672404d020 con 0x55f3c426ba70 2024-08-19T01:43:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f6733a0b740 10 client.17301 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f6733a0b740 10 client.17301 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f6733a0b740 20 client.17301 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f6733a0b740 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c42e5c50 0x55f3c42e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f6733a0b740 4 client.17301 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3c41c54c0 con 0x55f3c42e5c50 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f672affd640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c42e5c50 0x55f3c42e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f670eff5640 10 client.17301 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.372+0000 7f672affd640 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c42e5c50 0x55f3c42e8040 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac0c0da8e67c2b85 server_cookie=c8f082482833df6a in_seq=0 out_seq=0 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f670eff5640 1 -- 172.21.15.103:0/24990656 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3c41c54c0 con 0x55f3c42e5c50 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f670eff5640 10 client.17301 handle_command_reply: tid=0 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 client.17301 shutdown 2024-08-19T01:43:16.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 msgr2=0x55f3c42e4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c42e4020 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6724004630 tx=0x7f6724009970 comp rx=0 tx=0).stop 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 shutdown_connections 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x55f3c42e5c50 0x55f3c42e8040 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3c426ba70 0x55f3c42e4020 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f670eff5640 0 client.17301 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f3c4258250 0x55f3c426f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 --2- 172.21.15.103:0/24990656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f3c4257e70 0x55f3c426ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 >> 172.21.15.103:0/24990656 conn(0x55f3c4254580 msgr2=0x55f3c425b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 shutdown_connections 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 1 -- 172.21.15.103:0/24990656 wait complete. 2024-08-19T01:43:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.373+0000 7f6733a0b740 20 client.17301 trim_cache size 0 max 16384 2024-08-19T01:43:16.380 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:43:16.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67140a2b80 msgr2=0x7f67140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67140a2b80 0x7f67140a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f671c005e00 tx=0x7f671c005c50 comp rx=0 tx=0).stop 2024-08-19T01:43:16.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c1067b0 msgr2=0x7f672c19b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:16.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c1067b0 0x7f672c19b850 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6724005cc0 tx=0x7f6724003c20 comp rx=0 tx=0).stop 2024-08-19T01:43:16.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 shutdown_connections 2024-08-19T01:43:16.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f67140a2b80 0x7f67140a5040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f672c1067b0 0x7f672c19b850 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f672c101150 0x7f672c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 --2- 172.21.15.103:0/1736185855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f672c1007b0 0x7f672c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:16.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 >> 172.21.15.103:0/1736185855 conn(0x7f672c0fc4f0 msgr2=0x7f672c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:16.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.374+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 shutdown_connections 2024-08-19T01:43:16.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:16.375+0000 7f673160d640 1 -- 172.21.15.103:0/1736185855 wait complete. 2024-08-19T01:43:16.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:15 smithi190 ceph-mon[22032]: pgmap v563: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:43:16.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:15 smithi146 ceph-mon[22098]: pgmap v563: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:43:17.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:43:17.487 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:18.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 -- 172.21.15.103:0/346525055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e810bed0 msgr2=0x7f69e810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/346525055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e810bed0 0x7f69e810e340 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009ca0 tx=0x7f69e002a950 comp rx=0 tx=0).stop 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 -- 172.21.15.103:0/346525055 shutdown_connections 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/346525055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e810bed0 0x7f69e810e340 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/346525055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e8103450 0x7f69e810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/346525055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e8102b30 0x7f69e8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 -- 172.21.15.103:0/346525055 >> 172.21.15.103:0/346525055 conn(0x7f69e80fc4d0 msgr2=0x7f69e80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 -- 172.21.15.103:0/346525055 shutdown_connections 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.064+0000 7f69ecc1a640 1 -- 172.21.15.103:0/346525055 wait complete. 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 Processor -- start 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 -- start start 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e8102b30 0x7f69e8100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e8103450 0x7f69e8100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e810bed0 0x7f69e8101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8111b80 con 0x7f69e8102b30 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8111a00 con 0x7f69e810bed0 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69ecc1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8111d00 con 0x7f69e8103450 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e8102b30 0x7f69e8100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e8102b30 0x7f69e8100790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59754/0 (socket says 172.21.15.103:59754) 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69e659c640 1 -- 172.21.15.103:0/544284240 learned_addr learned my addr 172.21.15.103:0/544284240 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.065+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e8103450 0x7f69e8100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e810bed0 0x7f69e8101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e659c640 1 -- 172.21.15.103:0/544284240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e8103450 msgr2=0x7f69e8100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e659c640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e8103450 0x7f69e8100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e659c640 1 -- 172.21.15.103:0/544284240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e810bed0 msgr2=0x7f69e8101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e659c640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e810bed0 0x7f69e8101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e659c640 1 -- 172.21.15.103:0/544284240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e81a6520 con 0x7f69e8102b30 2024-08-19T01:43:18.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e8103450 0x7f69e8100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:18.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69e659c640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e8102b30 0x7f69e8100790 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f69d800d950 tx=0x7f69d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:18.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69d77fe640 1 -- 172.21.15.103:0/544284240 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8017070 con 0x7f69e8102b30 2024-08-19T01:43:18.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69d77fe640 1 -- 172.21.15.103:0/544284240 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69d8009c30 con 0x7f69e8102b30 2024-08-19T01:43:18.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69d77fe640 1 -- 172.21.15.103:0/544284240 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d800f620 con 0x7f69e8102b30 2024-08-19T01:43:18.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e81a6810 con 0x7f69e8102b30 2024-08-19T01:43:18.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.066+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e81a6d00 con 0x7f69e8102b30 2024-08-19T01:43:18.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69d77fe640 1 -- 172.21.15.103:0/544284240 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f69d80109a0 con 0x7f69e8102b30 2024-08-19T01:43:18.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 Processor -- start 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 -- start start 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c480c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c4835c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55817c3dba40 con 0x55817c4811d0 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55817c3739d0 con 0x55817c475ff0 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69ef018740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55817c3d7a00 con 0x55817c476910 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c4835c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69e6d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c4835c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59766/0 (socket says 172.21.15.103:59766) 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2705992314 learned_addr learned my addr 172.21.15.103:0/2705992314 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.070+0000 7f69e659c640 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c480c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69d77fe640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f69d00a2c00 0x7f69d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2705992314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 msgr2=0x55817c480c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c480c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2705992314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 msgr2=0x55817c4763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4763d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2705992314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817c37a650 con 0x55817c4811d0 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f69d00a2c00 0x7f69d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69d77fe640 1 -- 172.21.15.103:0/544284240 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f69d80cec90 con 0x7f69e8102b30 2024-08-19T01:43:18.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c4835c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00bcb0 tx=0x7f69dc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dc3dc73149b7da server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:18.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.071+0000 7f69d4ff9640 1 -- 172.21.15.103:0/2705992314 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc015070 con 0x55817c4811d0 2024-08-19T01:43:18.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69d4ff9640 1 -- 172.21.15.103:0/2705992314 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69dc0054d0 con 0x55817c4811d0 2024-08-19T01:43:18.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69d4ff9640 1 -- 172.21.15.103:0/2705992314 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc011440 con 0x55817c4811d0 2024-08-19T01:43:18.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69ef018740 1 -- 172.21.15.103:0/2705992314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 msgr2=0x55817c4835c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69ef018740 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c4835c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00bcb0 tx=0x7f69dc00f730 comp rx=0 tx=0).stop 2024-08-19T01:43:18.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f69d00a2c00 0x7f69d00a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005fa0 tx=0x7f69e00095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:18.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69ef018740 1 -- 172.21.15.103:0/2705992314 shutdown_connections 2024-08-19T01:43:18.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69ef018740 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c4835c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69ef018740 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c480c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69ef018740 1 --2- 172.21.15.103:0/2705992314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.072+0000 7f69ef018740 1 -- 172.21.15.103:0/2705992314 >> 172.21.15.103:0/2705992314 conn(0x55817c472580 msgr2=0x55817c472960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 -- 172.21.15.103:0/2705992314 shutdown_connections 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 -- 172.21.15.103:0/2705992314 wait complete. 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 -- start start 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 Processor -- start 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c475ff0 0x55817c47dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c47e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c4811d0 0x55817c47ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55817c483da0 con 0x55817c476910 2024-08-19T01:43:18.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55817c396b20 con 0x55817c4811d0 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.073+0000 7f69ef018740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55817c483f20 con 0x55817c475ff0 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69ef018740 10 client.0 _ll_register_callbacks cb 0x55817c394bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c47e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c47e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59776/0 (socket says 172.21.15.103:59776) 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c475ff0 0x55817c47dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2694731710 learned_addr learned my addr 172.21.15.103:0/2694731710 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c4811d0 0x55817c47ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c475ff0 msgr2=0x55817c47dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c475ff0 0x55817c47dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2694731710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c4811d0 msgr2=0x55817c47ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c4811d0 0x55817c47ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 -- 172.21.15.103:0/2694731710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817c14d3c0 con 0x55817c476910 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c4811d0 0x55817c47ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e659c640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c475ff0 0x55817c47dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c47e530 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00bc80 tx=0x7f69dc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd6613ea9bbc8db server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69cdffb640 1 -- 172.21.15.103:0/2694731710 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc028060 con 0x55817c476910 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69cdffb640 1 -- 172.21.15.103:0/2694731710 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f69dc0046c0 con 0x55817c476910 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.074+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69dc0054d0 con 0x55817c476910 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69cdffb640 1 -- 172.21.15.103:0/2694731710 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f69dc015070 con 0x55817c476910 2024-08-19T01:43:18.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 10 client.17313 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55817c48f8f0 con 0x55817c476910 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69cdffb640 1 -- 172.21.15.103:0/2694731710 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69dc0054d0 con 0x55817c476910 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69cdffb640 1 -- 172.21.15.103:0/2694731710 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f69dc011a20 con 0x55817c476910 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 10 client.17313 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 10 client.17313 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 20 client.17313 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c490db0 0x55817c4931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 4 client.17313 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55817c3e8fb0 con 0x55817c490db0 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.075+0000 7f69e659c640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c490db0 0x55817c4931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.076+0000 7f69e659c640 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c490db0 0x55817c4931a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b227c364d87dee21 server_cookie=6b49d04bc8ee8e9e in_seq=0 out_seq=0 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.076+0000 7f69cdffb640 10 client.17313 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.076+0000 7f69cdffb640 1 -- 172.21.15.103:0/2694731710 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55817c3e8fb0 con 0x55817c490db0 2024-08-19T01:43:18.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.076+0000 7f69cdffb640 10 client.17313 handle_command_reply: tid=0 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.076+0000 7f69ef018740 1 client.17313 shutdown 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 msgr2=0x55817c47e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c47e530 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00bc80 tx=0x7f69dc005710 comp rx=0 tx=0).stop 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 shutdown_connections 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c490db0 0x55817c4931a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c4811d0 0x55817c47ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c47e530 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69cdffb640 0 client.17313 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 --2- 172.21.15.103:0/2694731710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c475ff0 0x55817c47dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 >> 172.21.15.103:0/2694731710 conn(0x55817c472580 msgr2=0x55817c478260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 shutdown_connections 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 1 -- 172.21.15.103:0/2694731710 wait complete. 2024-08-19T01:43:18.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.077+0000 7f69ef018740 20 client.17313 trim_cache size 0 max 16384 2024-08-19T01:43:18.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 Processor -- start 2024-08-19T01:43:18.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 -- start start 2024-08-19T01:43:18.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c498e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c470d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81af350 con 0x55817c4811d0 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81b0f20 con 0x55817c475ff0 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69ef018740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81afa20 con 0x55817c476910 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c470d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c470d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59786/0 (socket says 172.21.15.103:59786) 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 -- 172.21.15.103:0/3022947492 learned_addr learned my addr 172.21.15.103:0/3022947492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c498e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e659c640 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 -- 172.21.15.103:0/3022947492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 msgr2=0x55817c498e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c498e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 -- 172.21.15.103:0/3022947492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 msgr2=0x55817c4988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e6d9d640 1 -- 172.21.15.103:0/3022947492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817c476700 con 0x55817c4811d0 2024-08-19T01:43:18.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.091+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c498e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:18.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.092+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c470d90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f69e0035000 tx=0x7f69e0042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd667530ada0b5c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:18.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.092+0000 7f69cffff640 1 -- 172.21.15.103:0/3022947492 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0034040 con 0x55817c4811d0 2024-08-19T01:43:18.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.092+0000 7f69cffff640 1 -- 172.21.15.103:0/3022947492 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69e0004050 con 0x55817c4811d0 2024-08-19T01:43:18.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.092+0000 7f69cffff640 1 -- 172.21.15.103:0/3022947492 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0030680 con 0x55817c4811d0 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 -- 172.21.15.103:0/3022947492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 msgr2=0x55817c470d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c470d90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f69e0035000 tx=0x7f69e0042000 comp rx=0 tx=0).stop 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 -- 172.21.15.103:0/3022947492 shutdown_connections 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c4811d0 0x55817c470d90 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c476910 0x55817c498e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 --2- 172.21.15.103:0/3022947492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c4988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 -- 172.21.15.103:0/3022947492 >> 172.21.15.103:0/3022947492 conn(0x55817c472580 msgr2=0x55817c47c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 -- 172.21.15.103:0/3022947492 shutdown_connections 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 -- 172.21.15.103:0/3022947492 wait complete. 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 -- start start 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.093+0000 7f69ef018740 1 Processor -- start 2024-08-19T01:43:18.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69ef018740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c489410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69ef018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c489950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69ef018740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c4811d0 0x55817c47a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69ef018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81b0850 con 0x55817c476910 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69ef018740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81b00f0 con 0x55817c475ff0 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69ef018740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69e81ad330 con 0x55817c4811d0 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69ef018740 10 client.0 _ll_register_callbacks cb 0x7f69e0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c489950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c489950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59800/0 (socket says 172.21.15.103:59800) 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 -- 172.21.15.103:0/3016165830 learned_addr learned my addr 172.21.15.103:0/3016165830 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c4811d0 0x55817c47a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e659c640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c489410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 -- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c4811d0 msgr2=0x55817c47a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c4811d0 0x55817c47a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 -- 172.21.15.103:0/3016165830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 msgr2=0x55817c489410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c489410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 -- 172.21.15.103:0/3016165830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817c47b0b0 con 0x55817c476910 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e6d9d640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c4811d0 0x55817c47a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e659c640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c489410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69e5d9b640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c489950 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x55817c47f630 tx=0x7f69dc023420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599a588ee1f028d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.094+0000 7f69cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69cdffb640 1 -- 172.21.15.103:0/3016165830 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc033030 con 0x55817c476910 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69cdffb640 1 -- 172.21.15.103:0/3016165830 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f69dc020050 con 0x55817c476910 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e0004050 con 0x55817c476910 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69cdffb640 1 -- 172.21.15.103:0/3016165830 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f69dc004bc0 con 0x55817c476910 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 10 client.17319 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55817c488cb0 con 0x55817c476910 2024-08-19T01:43:18.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69cdffb640 1 -- 172.21.15.103:0/3016165830 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69e0004050 con 0x55817c476910 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69cdffb640 1 -- 172.21.15.103:0/3016165830 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f69dc023850 con 0x55817c476910 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 10 client.17319 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 10 client.17319 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 20 client.17319 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c4fbaf0 0x55817c4fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 4 client.17319 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.095+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55817c3e34c0 con 0x55817c4fbaf0 2024-08-19T01:43:18.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.096+0000 7f69e659c640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c4fbaf0 0x55817c4fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:18.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.096+0000 7f69e659c640 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c4fbaf0 0x55817c4fdf10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ee49cc04a4e2f43 server_cookie=2594ac1df10e47e in_seq=0 out_seq=0 2024-08-19T01:43:18.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.096+0000 7f69cdffb640 10 client.17319 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:43:18.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69cdffb640 1 -- 172.21.15.103:0/3016165830 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55817c3e34c0 con 0x55817c4fbaf0 2024-08-19T01:43:18.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69cdffb640 10 client.17319 handle_command_reply: tid=0 2024-08-19T01:43:18.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 client.17319 shutdown 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 msgr2=0x55817c489950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c489950 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x55817c47f630 tx=0x7f69dc023420 comp rx=0 tx=0).stop 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 shutdown_connections 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x55817c4fbaf0 0x55817c4fdf10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817c4811d0 0x55817c47a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817c476910 0x55817c489950 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69cdffb640 0 client.17319 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 --2- 172.21.15.103:0/3016165830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817c475ff0 0x55817c489410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 >> 172.21.15.103:0/3016165830 conn(0x55817c472580 msgr2=0x55817c470d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 shutdown_connections 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 1 -- 172.21.15.103:0/3016165830 wait complete. 2024-08-19T01:43:18.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.097+0000 7f69ef018740 20 client.17319 trim_cache size 0 max 16384 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f69d00a2c00 msgr2=0x7f69d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f69d00a2c00 0x7f69d00a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005fa0 tx=0x7f69e00095c0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e8102b30 msgr2=0x7f69e8100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e8102b30 0x7f69e8100790 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f69d800d950 tx=0x7f69d800de10 comp rx=0 tx=0).stop 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 shutdown_connections 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f69d00a2c00 0x7f69d00a50c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69e810bed0 0x7f69e8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e8103450 0x7f69e8100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 --2- 172.21.15.103:0/544284240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e8102b30 0x7f69e8100790 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:18.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 >> 172.21.15.103:0/544284240 conn(0x7f69e80fc4d0 msgr2=0x7f69e80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:18.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 shutdown_connections 2024-08-19T01:43:18.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:18.098+0000 7f69ecc1a640 1 -- 172.21.15.103:0/544284240 wait complete. 2024-08-19T01:43:18.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:17 smithi103 ceph-mon[18436]: pgmap v564: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:43:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:17 smithi190 ceph-mon[22032]: pgmap v564: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:43:18.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:17 smithi146 ceph-mon[22098]: pgmap v564: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:43:19.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:19 smithi103 ceph-mon[18436]: pgmap v565: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:43:19.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:19 smithi190 ceph-mon[22032]: pgmap v565: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:43:19.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:19 smithi146 ceph-mon[22098]: pgmap v565: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-08-19T01:43:20.363 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:43:20.656 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:21.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 -- 172.21.15.103:0/1953633799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c1022a0 msgr2=0x7f354c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 --2- 172.21.15.103:0/1953633799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c1022a0 0x7f354c1026e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3540009900 tx=0x7f354002a480 comp rx=0 tx=0).stop 2024-08-19T01:43:21.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 -- 172.21.15.103:0/1953633799 shutdown_connections 2024-08-19T01:43:21.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 --2- 172.21.15.103:0/1953633799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f354c102c20 0x7f354c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 --2- 172.21.15.103:0/1953633799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c1022a0 0x7f354c1026e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 --2- 172.21.15.103:0/1953633799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f354c100b50 0x7f354c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 -- 172.21.15.103:0/1953633799 >> 172.21.15.103:0/1953633799 conn(0x7f354c0fc500 msgr2=0x7f354c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 -- 172.21.15.103:0/1953633799 shutdown_connections 2024-08-19T01:43:21.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.308+0000 7f35510f9640 1 -- 172.21.15.103:0/1953633799 wait complete. 2024-08-19T01:43:21.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 Processor -- start 2024-08-19T01:43:21.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 -- start start 2024-08-19T01:43:21.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f354c100b50 0x7f354c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f354c1022a0 0x7f354c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c102c20 0x7f354c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f354c110b20 con 0x7f354c102c20 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1109a0 con 0x7f354c100b50 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f35510f9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f354c110ca0 con 0x7f354c1022a0 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f354b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c102c20 0x7f354c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f354ad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f354c100b50 0x7f354c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f354b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c102c20 0x7f354c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59816/0 (socket says 172.21.15.103:59816) 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f354a59c640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f354c1022a0 0x7f354c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.309+0000 7f354b59e640 1 -- 172.21.15.103:0/3411167323 learned_addr learned my addr 172.21.15.103:0/3411167323 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f354b59e640 1 -- 172.21.15.103:0/3411167323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f354c1022a0 msgr2=0x7f354c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f354b59e640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f354c1022a0 0x7f354c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f354b59e640 1 -- 172.21.15.103:0/3411167323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f354c100b50 msgr2=0x7f354c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f354b59e640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f354c100b50 0x7f354c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f354b59e640 1 -- 172.21.15.103:0/3411167323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c19d130 con 0x7f354c102c20 2024-08-19T01:43:21.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f354a59c640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f354c1022a0 0x7f354c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:21.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f354b59e640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c102c20 0x7f354c19ca50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3544009fd0 tx=0x7f354400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f3533fff640 1 -- 172.21.15.103:0/3411167323 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544017070 con 0x7f354c102c20 2024-08-19T01:43:21.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f3533fff640 1 -- 172.21.15.103:0/3411167323 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3544003e20 con 0x7f354c102c20 2024-08-19T01:43:21.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c19d420 con 0x7f354c102c20 2024-08-19T01:43:21.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f3533fff640 1 -- 172.21.15.103:0/3411167323 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544010460 con 0x7f354c102c20 2024-08-19T01:43:21.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.310+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c0fc8e0 con 0x7f354c102c20 2024-08-19T01:43:21.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.313+0000 7f3533fff640 1 -- 172.21.15.103:0/3411167323 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3544003970 con 0x7f354c102c20 2024-08-19T01:43:21.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.313+0000 7f3533fff640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f35340a2b80 0x7f35340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.314+0000 7f3533fff640 1 -- 172.21.15.103:0/3411167323 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f35440ced80 con 0x7f354c102c20 2024-08-19T01:43:21.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.314+0000 7f354ad9d640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f35340a2b80 0x7f35340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.314+0000 7f354ad9d640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f35340a2b80 0x7f35340a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f353c005d00 tx=0x7f353c00f820 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.317+0000 7f35534f7740 1 Processor -- start 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.317+0000 7f35534f7740 1 -- start start 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f35534f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abb73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f35534f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abc1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f35534f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abc45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f35534f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a69ab1ca40 con 0x55a69abc21d0 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f35534f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a69aab49d0 con 0x55a69abb7910 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f35534f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a69ab18a00 con 0x55a69abb6ff0 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abc45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abc45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59832/0 (socket says 172.21.15.103:59832) 2024-08-19T01:43:21.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 -- 172.21.15.103:0/2084470035 learned_addr learned my addr 172.21.15.103:0/2084470035 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354a59c640 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abc1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354ad9d640 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abb73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 -- 172.21.15.103:0/2084470035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 msgr2=0x55a69abb73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abb73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 -- 172.21.15.103:0/2084470035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 msgr2=0x55a69abc1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abc1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354b59e640 1 -- 172.21.15.103:0/2084470035 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69aabb650 con 0x55a69abc21d0 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.318+0000 7f354a59c640 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abc1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.319+0000 7f354b59e640 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abc45c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3544098070 tx=0x7f354409b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315b9d9d26239c4f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.319+0000 7f35317fa640 1 -- 172.21.15.103:0/2084470035 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544020040 con 0x55a69abc21d0 2024-08-19T01:43:21.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.319+0000 7f35317fa640 1 -- 172.21.15.103:0/2084470035 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f354409b6a0 con 0x55a69abc21d0 2024-08-19T01:43:21.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.319+0000 7f35317fa640 1 -- 172.21.15.103:0/2084470035 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544097d10 con 0x55a69abc21d0 2024-08-19T01:43:21.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.319+0000 7f35534f7740 1 -- 172.21.15.103:0/2084470035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 msgr2=0x55a69abc45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.319+0000 7f35534f7740 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abc45c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3544098070 tx=0x7f354409b120 comp rx=0 tx=0).stop 2024-08-19T01:43:21.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 -- 172.21.15.103:0/2084470035 shutdown_connections 2024-08-19T01:43:21.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abc45c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abc1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 --2- 172.21.15.103:0/2084470035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abb73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 -- 172.21.15.103:0/2084470035 >> 172.21.15.103:0/2084470035 conn(0x55a69abb3580 msgr2=0x55a69abb3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 -- 172.21.15.103:0/2084470035 shutdown_connections 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 -- 172.21.15.103:0/2084470035 wait complete. 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 -- start start 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 Processor -- start 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb6ff0 0x55a69abbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abbf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.320+0000 7f35534f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abc21d0 0x55a69abbfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f35534f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a69abc4da0 con 0x55a69abb6ff0 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f35534f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a69aad7b20 con 0x55a69abb7910 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f35534f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a69abc4f20 con 0x55a69abc21d0 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f35534f7740 10 client.0 _ll_register_callbacks cb 0x55a69aad5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb6ff0 0x55a69abbf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb6ff0 0x55a69abbf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59838/0 (socket says 172.21.15.103:59838) 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 -- 172.21.15.103:0/146001736 learned_addr learned my addr 172.21.15.103:0/146001736 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354a59c640 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abbf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354b59e640 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abc21d0 0x55a69abbfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 -- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abc21d0 msgr2=0x55a69abbfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abc21d0 0x55a69abbfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 -- 172.21.15.103:0/146001736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 msgr2=0x55a69abbf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abbf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 -- 172.21.15.103:0/146001736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69a88e3c0 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f354ad9d640 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb6ff0 0x55a69abbf090 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f353c01bd40 tx=0x7f353c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80636c0ffd3f474 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.321+0000 7f352eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f352eff5640 1 -- 172.21.15.103:0/146001736 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0273f0 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f352eff5640 1 -- 172.21.15.103:0/146001736 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f353c027590 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f354409b6a0 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f352eff5640 1 -- 172.21.15.103:0/146001736 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f353c0277d0 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 10 client.17331 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a69abd09e0 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f352eff5640 1 -- 172.21.15.103:0/146001736 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f354409b6a0 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f352eff5640 1 -- 172.21.15.103:0/146001736 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f353c030a50 con 0x55a69abb6ff0 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 10 client.17331 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:21.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 10 client.17331 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 20 client.17331 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69abd1f40 0x55a69abd4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 4 client.17331 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.322+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a69ab29fb0 con 0x55a69abd1f40 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.323+0000 7f354a59c640 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69abd1f40 0x55a69abd4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.323+0000 7f352eff5640 10 client.17331 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.323+0000 7f354a59c640 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69abd1f40 0x55a69abd4380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5eec1b2142236426 server_cookie=e3f096f580ceff69 in_seq=0 out_seq=0 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f352eff5640 1 -- 172.21.15.103:0/146001736 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a69ab29fb0 con 0x55a69abd1f40 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f352eff5640 10 client.17331 handle_command_reply: tid=0 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 client.17331 shutdown 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb6ff0 msgr2=0x55a69abbf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb6ff0 0x55a69abbf090 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f353c01bd40 tx=0x7f353c004730 comp rx=0 tx=0).stop 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 shutdown_connections 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69abd1f40 0x55a69abd4380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abc21d0 0x55a69abbfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abbf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 --2- 172.21.15.103:0/146001736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb6ff0 0x55a69abbf090 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f352eff5640 0 client.17331 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 >> 172.21.15.103:0/146001736 conn(0x55a69abb3580 msgr2=0x55a69abb9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 shutdown_connections 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 1 -- 172.21.15.103:0/146001736 wait complete. 2024-08-19T01:43:21.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.324+0000 7f35534f7740 20 client.17331 trim_cache size 0 max 16384 2024-08-19T01:43:21.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 Processor -- start 2024-08-19T01:43:21.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 -- start start 2024-08-19T01:43:21.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abd9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb7910 0x55a69abda190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abc21d0 0x55a69abb1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1aaf30 con 0x55a69abb7910 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1acb00 con 0x55a69abc21d0 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f35534f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1ab600 con 0x55a69abb6ff0 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.337+0000 7f354a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb7910 0x55a69abda190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb7910 0x55a69abda190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59846/0 (socket says 172.21.15.103:59846) 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 -- 172.21.15.103:0/3230885329 learned_addr learned my addr 172.21.15.103:0/3230885329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354b59e640 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abc21d0 0x55a69abb1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354ad9d640 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abd9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 -- 172.21.15.103:0/3230885329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 msgr2=0x55a69abd9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abd9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 -- 172.21.15.103:0/3230885329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abc21d0 msgr2=0x55a69abb1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abc21d0 0x55a69abb1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 -- 172.21.15.103:0/3230885329 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69abb4710 con 0x55a69abb7910 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354b59e640 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abc21d0 0x55a69abb1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f354a59c640 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb7910 0x55a69abda190 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f354002a9a0 tx=0x7f35400328f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512f29f91a1de6d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f3530ff9640 1 -- 172.21.15.103:0/3230885329 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540035040 con 0x55a69abb7910 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f3530ff9640 1 -- 172.21.15.103:0/3230885329 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35400054d0 con 0x55a69abb7910 2024-08-19T01:43:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.338+0000 7f3530ff9640 1 -- 172.21.15.103:0/3230885329 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540004c30 con 0x55a69abb7910 2024-08-19T01:43:21.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 -- 172.21.15.103:0/3230885329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb7910 msgr2=0x55a69abda190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb7910 0x55a69abda190 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f354002a9a0 tx=0x7f35400328f0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 -- 172.21.15.103:0/3230885329 shutdown_connections 2024-08-19T01:43:21.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abc21d0 0x55a69abb1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abb7910 0x55a69abda190 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 --2- 172.21.15.103:0/3230885329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abd9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 -- 172.21.15.103:0/3230885329 >> 172.21.15.103:0/3230885329 conn(0x55a69abb3580 msgr2=0x55a69abbd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 -- 172.21.15.103:0/3230885329 shutdown_connections 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.339+0000 7f35534f7740 1 -- 172.21.15.103:0/3230885329 wait complete. 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 -- start start 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 Processor -- start 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abbb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1ac430 con 0x55a69abc21d0 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1abcd0 con 0x55a69abb7910 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1a8f10 con 0x55a69abb6ff0 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f35534f7740 10 client.0 _ll_register_callbacks cb 0x7f353c01b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abbb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354b59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abbb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59850/0 (socket says 172.21.15.103:59850) 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354b59e640 1 -- 172.21.15.103:0/1353235746 learned_addr learned my addr 172.21.15.103:0/1353235746 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354ad9d640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354a59c640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354b59e640 1 -- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 msgr2=0x55a69abca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354b59e640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354b59e640 1 -- 172.21.15.103:0/1353235746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 msgr2=0x55a69abca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.340+0000 7f354b59e640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f354b59e640 1 -- 172.21.15.103:0/1353235746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69abbbd00 con 0x55a69abc21d0 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f354a59c640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f354b59e640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abbb6c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f354400dfd0 tx=0x7f354400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6b966283103ea1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f352eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f352eff5640 1 -- 172.21.15.103:0/1353235746 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354407f060 con 0x55a69abc21d0 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f352eff5640 1 -- 172.21.15.103:0/1353235746 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3544097d10 con 0x55a69abc21d0 2024-08-19T01:43:21.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35400054d0 con 0x55a69abc21d0 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f352eff5640 1 -- 172.21.15.103:0/1353235746 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3544020040 con 0x55a69abc21d0 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f35534f7740 10 client.17337 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.341+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a69ac3c560 con 0x55a69abc21d0 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f352eff5640 1 -- 172.21.15.103:0/1353235746 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35400054d0 con 0x55a69abc21d0 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f352eff5640 1 -- 172.21.15.103:0/1353235746 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f35440afb10 con 0x55a69abc21d0 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f35534f7740 10 client.17337 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f35534f7740 10 client.17337 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f35534f7740 20 client.17337 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f35534f7740 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69ac3c890 0x55a69ac3ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f35534f7740 4 client.17337 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a69ab244c0 con 0x55a69ac3c890 2024-08-19T01:43:21.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f354ad9d640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69ac3c890 0x55a69ac3ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f354ad9d640 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69ac3c890 0x55a69ac3ecb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e87a7293dc0ef77 server_cookie=960925c18cf953e0 in_seq=0 out_seq=0 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.342+0000 7f352eff5640 10 client.17337 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f352eff5640 1 -- 172.21.15.103:0/1353235746 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a69ab244c0 con 0x55a69ac3c890 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f352eff5640 10 client.17337 handle_command_reply: tid=0 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 client.17337 shutdown 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 msgr2=0x55a69abbb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abbb6c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f354400dfd0 tx=0x7f354400b900 comp rx=0 tx=0).stop 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 shutdown_connections 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x55a69ac3c890 0x55a69ac3ecb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a69abc21d0 0x55a69abbb6c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a69abb7910 0x55a69abca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 --2- 172.21.15.103:0/1353235746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a69abb6ff0 0x55a69abca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 >> 172.21.15.103:0/1353235746 conn(0x55a69abb3580 msgr2=0x55a69abb8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f352eff5640 0 client.17337 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:43:21.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 shutdown_connections 2024-08-19T01:43:21.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 1 -- 172.21.15.103:0/1353235746 wait complete. 2024-08-19T01:43:21.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.343+0000 7f35534f7740 20 client.17337 trim_cache size 0 max 16384 2024-08-19T01:43:21.352 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f35340a2b80 msgr2=0x7f35340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f35340a2b80 0x7f35340a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f353c005d00 tx=0x7f353c00f820 comp rx=0 tx=0).stop 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c102c20 msgr2=0x7f354c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c102c20 0x7f354c19ca50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3544009fd0 tx=0x7f354400b260 comp rx=0 tx=0).stop 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 shutdown_connections 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f35340a2b80 0x7f35340a5040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f354c102c20 0x7f354c19ca50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f354c1022a0 0x7f354c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 --2- 172.21.15.103:0/3411167323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f354c100b50 0x7f354c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 >> 172.21.15.103:0/3411167323 conn(0x7f354c0fc500 msgr2=0x7f354c06a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:21.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 shutdown_connections 2024-08-19T01:43:21.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:21.344+0000 7f35510f9640 1 -- 172.21.15.103:0/3411167323 wait complete. 2024-08-19T01:43:22.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:43:22.099 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:21 smithi103 ceph-mon[18436]: pgmap v566: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:43:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 -- 172.21.15.103:0/3226046234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101e00 msgr2=0x7f5b3c102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 --2- 172.21.15.103:0/3226046234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101e00 0x7f5b3c102200 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c02a400 comp rx=0 tx=0).stop 2024-08-19T01:43:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 -- 172.21.15.103:0/3226046234 shutdown_connections 2024-08-19T01:43:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 --2- 172.21.15.103:0/3226046234 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c101380 0x7f5b3c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 --2- 172.21.15.103:0/3226046234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b3c1009c0 0x7f5b3c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 --2- 172.21.15.103:0/3226046234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101e00 0x7f5b3c102200 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 -- 172.21.15.103:0/3226046234 >> 172.21.15.103:0/3226046234 conn(0x7f5b3c0fb660 msgr2=0x7f5b3c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 -- 172.21.15.103:0/3226046234 shutdown_connections 2024-08-19T01:43:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 -- 172.21.15.103:0/3226046234 wait complete. 2024-08-19T01:43:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.173+0000 7f5b42010640 1 Processor -- start 2024-08-19T01:43:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b42010640 1 -- start start 2024-08-19T01:43:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b42010640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b3c1009c0 0x7f5b3c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b42010640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101380 0x7f5b3c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b42010640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c101e00 0x7f5b3c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b42010640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c10c310 con 0x7f5b3c101380 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b42010640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c10c190 con 0x7f5b3c101e00 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b42010640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c10c490 con 0x7f5b3c1009c0 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101380 0x7f5b3c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101380 0x7f5b3c197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46988/0 (socket says 172.21.15.103:46988) 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 -- 172.21.15.103:0/3273710080 learned_addr learned my addr 172.21.15.103:0/3273710080 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b3c1009c0 0x7f5b3c1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c101e00 0x7f5b3c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 -- 172.21.15.103:0/3273710080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b3c1009c0 msgr2=0x7f5b3c1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b3c1009c0 0x7f5b3c1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 -- 172.21.15.103:0/3273710080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c101e00 msgr2=0x7f5b3c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c101e00 0x7f5b3c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3affd640 1 -- 172.21.15.103:0/3273710080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c19f5c0 con 0x7f5b3c101380 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.174+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c101e00 0x7f5b3c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.175+0000 7f5b3affd640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101380 0x7f5b3c197ee0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c970 tx=0x7f5b3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.175+0000 7f5b38ff9640 1 -- 172.21.15.103:0/3273710080 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30007c10 con 0x7f5b3c101380 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.175+0000 7f5b38ff9640 1 -- 172.21.15.103:0/3273710080 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b30007db0 con 0x7f5b3c101380 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.175+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c19f8b0 con 0x7f5b3c101380 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.175+0000 7f5b38ff9640 1 -- 172.21.15.103:0/3273710080 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000f870 con 0x7f5b3c101380 2024-08-19T01:43:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.175+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c19fe00 con 0x7f5b3c101380 2024-08-19T01:43:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.177+0000 7f5b38ff9640 1 -- 172.21.15.103:0/3273710080 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5b3000fa10 con 0x7f5b3c101380 2024-08-19T01:43:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 Processor -- start 2024-08-19T01:43:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 -- start start 2024-08-19T01:43:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d819a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d81ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d81f050 0x561b9d821510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b9d77c9d0 con 0x561b9d81f050 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561b9d724820 con 0x561b9d819680 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561b9d778990 con 0x561b9d819fc0 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d81f050 0x561b9d821510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b3bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d819a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b38ff9640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b240a2c30 0x7f5b240a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b3bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d819a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:44770/0 (socket says 172.21.15.103:44770) 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b3bfff640 1 -- 172.21.15.103:0/2805499292 learned_addr learned my addr 172.21.15.103:0/2805499292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.178+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d81ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b240a2c30 0x7f5b240a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 -- 172.21.15.103:0/2805499292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 msgr2=0x561b9d81ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d81ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 -- 172.21.15.103:0/2805499292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d81f050 msgr2=0x561b9d821510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d81f050 0x561b9d821510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 -- 172.21.15.103:0/2805499292 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b9d490210 con 0x561b9d819680 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b38ff9640 1 -- 172.21.15.103:0/3273710080 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5b300cf300 con 0x7f5b3c101380 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d819a80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009fd0 tx=0x7f5b34013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b272975d38975dc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b240a2c30 0x7f5b240a50f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b3401d000 tx=0x7f5b3401f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b21ffb640 1 -- 172.21.15.103:0/2805499292 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400f8b0 con 0x561b9d819680 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b21ffb640 1 -- 172.21.15.103:0/2805499292 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b34022020 con 0x561b9d819680 2024-08-19T01:43:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.179+0000 7f5b21ffb640 1 -- 172.21.15.103:0/2805499292 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34010980 con 0x561b9d819680 2024-08-19T01:43:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 -- 172.21.15.103:0/2805499292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 msgr2=0x561b9d819a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d819a80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009fd0 tx=0x7f5b34013830 comp rx=0 tx=0).stop 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 -- 172.21.15.103:0/2805499292 shutdown_connections 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d81f050 0x561b9d821510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d81ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/2805499292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d819a80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 -- 172.21.15.103:0/2805499292 >> 172.21.15.103:0/2805499292 conn(0x561b9d814cf0 msgr2=0x561b9d8150f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 -- 172.21.15.103:0/2805499292 shutdown_connections 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.180+0000 7f5b44dd3740 1 -- 172.21.15.103:0/2805499292 wait complete. 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 -- start start 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 Processor -- start 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819680 0x561b9d823840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d823d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d81f050 0x561b9d82ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b9d821fc0 con 0x561b9d819680 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561b9d738520 con 0x561b9d81f050 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561b9d822140 con 0x561b9d819fc0 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b44dd3740 10 client.0 _ll_register_callbacks cb 0x561b9d48fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819680 0x561b9d823840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.181+0000 7f5b3b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819680 0x561b9d823840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47004/0 (socket says 172.21.15.103:47004) 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3b7fe640 1 -- 172.21.15.103:0/344371241 learned_addr learned my addr 172.21.15.103:0/344371241 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3affd640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d823d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d81f050 0x561b9d82ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3b7fe640 1 -- 172.21.15.103:0/344371241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 msgr2=0x561b9d823d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d823d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3b7fe640 1 -- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d81f050 msgr2=0x561b9d82ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d81f050 0x561b9d82ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3b7fe640 1 -- 172.21.15.103:0/344371241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9d55c620 con 0x561b9d819680 2024-08-19T01:43:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d81f050 0x561b9d82ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819680 0x561b9d823840 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c02a9f0 tx=0x7f5b2c004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddbae028b8f49ee server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b1affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b1affd640 1 -- 172.21.15.103:0/344371241 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c035040 con 0x561b9d819680 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b1affd640 1 -- 172.21.15.103:0/344371241 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5b2c003a80 con 0x561b9d819680 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.182+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b34022020 con 0x561b9d819680 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b1affd640 1 -- 172.21.15.103:0/344371241 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c003c80 con 0x561b9d819680 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 10 client.17349 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561b9d82dbb0 con 0x561b9d819680 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b1affd640 1 -- 172.21.15.103:0/344371241 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b34022020 con 0x561b9d819680 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b1affd640 1 -- 172.21.15.103:0/344371241 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f5b2c03c5e0 con 0x561b9d819680 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 10 client.17349 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 20 client.17349 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d82f280 0x561b9d831710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 4 client.17349 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b9d78a100 con 0x561b9d82f280 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.183+0000 7f5b3affd640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d82f280 0x561b9d831710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.184+0000 7f5b3affd640 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d82f280 0x561b9d831710 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95132e6d684fd89c server_cookie=147a2dcdf0c1fe34 in_seq=0 out_seq=0 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.184+0000 7f5b1affd640 10 client.17349 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b1affd640 1 -- 172.21.15.103:0/344371241 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b9d78a100 con 0x561b9d82f280 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b1affd640 10 client.17349 handle_command_reply: tid=0 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 client.17349 shutdown 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819680 msgr2=0x561b9d823840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819680 0x561b9d823840 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c02a9f0 tx=0x7f5b2c004580 comp rx=0 tx=0).stop 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 shutdown_connections 2024-08-19T01:43:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d82f280 0x561b9d831710 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d81f050 0x561b9d82ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d819fc0 0x561b9d823d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/344371241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819680 0x561b9d823840 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b1affd640 0 client.17349 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 >> 172.21.15.103:0/344371241 conn(0x561b9d814cf0 msgr2=0x561b9d816500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 shutdown_connections 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 1 -- 172.21.15.103:0/344371241 wait complete. 2024-08-19T01:43:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.185+0000 7f5b44dd3740 20 client.17349 trim_cache size 0 max 16384 2024-08-19T01:43:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 Processor -- start 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 -- start start 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d823e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d82ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d813d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1a8a00 con 0x561b9d819fc0 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1aa5d0 con 0x561b9d819680 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1a90d0 con 0x561b9d81f050 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d82ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d823e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d82ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47008/0 (socket says 172.21.15.103:47008) 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 -- 172.21.15.103:0/1476694043 learned_addr learned my addr 172.21.15.103:0/1476694043 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d813d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 -- 172.21.15.103:0/1476694043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 msgr2=0x561b9d813d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d813d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 -- 172.21.15.103:0/1476694043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 msgr2=0x561b9d823e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d823e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3affd640 1 -- 172.21.15.103:0/1476694043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b9d81a4a0 con 0x561b9d819fc0 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.205+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d823e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.206+0000 7f5b3affd640 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d82ac70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c940 tx=0x7f5b300d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd2b01f8309b8bc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.206+0000 7f5b217fa640 1 -- 172.21.15.103:0/1476694043 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3009c320 con 0x561b9d819fc0 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.206+0000 7f5b217fa640 1 -- 172.21.15.103:0/1476694043 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3009bc20 con 0x561b9d819fc0 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.206+0000 7f5b217fa640 1 -- 172.21.15.103:0/1476694043 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30098290 con 0x561b9d819fc0 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1476694043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 msgr2=0x561b9d82ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d82ac70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c940 tx=0x7f5b300d1040 comp rx=0 tx=0).stop 2024-08-19T01:43:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1476694043 shutdown_connections 2024-08-19T01:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d813d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d82ac70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1476694043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d823e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1476694043 >> 172.21.15.103:0/1476694043 conn(0x561b9d814cf0 msgr2=0x561b9d813680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1476694043 shutdown_connections 2024-08-19T01:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1476694043 wait complete. 2024-08-19T01:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 -- start start 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.207+0000 7f5b44dd3740 1 Processor -- start 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d8287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d828d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b44dd3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d829d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1a9f00 con 0x561b9d819fc0 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1a97a0 con 0x561b9d819680 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b44dd3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1a69e0 con 0x561b9d81f050 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b44dd3740 10 client.0 _ll_register_callbacks cb 0x7f5b2c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d828d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d828d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47022/0 (socket says 172.21.15.103:47022) 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 -- 172.21.15.103:0/1884814801 learned_addr learned my addr 172.21.15.103:0/1884814801 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3bfff640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d829d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d8287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 -- 172.21.15.103:0/1884814801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 msgr2=0x561b9d829d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d829d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 -- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 msgr2=0x561b9d8287e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d8287e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 -- 172.21.15.103:0/1884814801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b9d82a340 con 0x561b9d819fc0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d8287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b3affd640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d828d20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5b3009b920 tx=0x7f5b3009bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c033bdac0a3f86 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b1affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.208+0000 7f5b1affd640 1 -- 172.21.15.103:0/1884814801 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3007e030 con 0x561b9d819fc0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b1affd640 1 -- 172.21.15.103:0/1884814801 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f5b3000fd00 con 0x561b9d819fc0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b3009bc20 con 0x561b9d819fc0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b1affd640 1 -- 172.21.15.103:0/1884814801 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5b3009c320 con 0x561b9d819fc0 2024-08-19T01:43:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 10 client.17355 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561b9d89a4e0 con 0x561b9d819fc0 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b1affd640 1 -- 172.21.15.103:0/1884814801 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3009bc20 con 0x561b9d819fc0 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b1affd640 1 -- 172.21.15.103:0/1884814801 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f5b300ce040 con 0x561b9d819fc0 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 10 client.17355 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 20 client.17355 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d89ae10 0x561b9d89d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 4 client.17355 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b9d784450 con 0x561b9d89ae10 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.209+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d89ae10 0x561b9d89d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.210+0000 7f5b1affd640 10 client.17355 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.210+0000 7f5b3b7fe640 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d89ae10 0x561b9d89d230 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae49fd1b3deabac5 server_cookie=f06e16b25573afdb in_seq=0 out_seq=0 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b1affd640 1 -- 172.21.15.103:0/1884814801 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b9d784450 con 0x561b9d89ae10 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b1affd640 10 client.17355 handle_command_reply: tid=0 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 client.17355 shutdown 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 msgr2=0x561b9d828d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d828d20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5b3009b920 tx=0x7f5b3009bac0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 shutdown_connections 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x561b9d89ae10 0x561b9d89d230 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561b9d81f050 0x561b9d829d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b9d819fc0 0x561b9d828d20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 --2- 172.21.15.103:0/1884814801 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561b9d819680 0x561b9d8287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b1affd640 0 client.17355 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 >> 172.21.15.103:0/1884814801 conn(0x561b9d814cf0 msgr2=0x561b9d823c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 shutdown_connections 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 1 -- 172.21.15.103:0/1884814801 wait complete. 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.217+0000 7f5b44dd3740 20 client.17355 trim_cache size 0 max 16384 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:43:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:43:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b240a2c30 msgr2=0x7f5b240a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b240a2c30 0x7f5b240a50f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b3401d000 tx=0x7f5b3401f000 comp rx=0 tx=0).stop 2024-08-19T01:43:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101380 msgr2=0x7f5b3c197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101380 0x7f5b3c197ee0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c970 tx=0x7f5b3000ce30 comp rx=0 tx=0).stop 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 shutdown_connections 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f5b240a2c30 0x7f5b240a50f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b3c101e00 0x7f5b3c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c101380 0x7f5b3c197ee0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 --2- 172.21.15.103:0/3273710080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b3c1009c0 0x7f5b3c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 >> 172.21.15.103:0/3273710080 conn(0x7f5b3c0fb660 msgr2=0x7f5b3c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 shutdown_connections 2024-08-19T01:43:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:22.218+0000 7f5b42010640 1 -- 172.21.15.103:0/3273710080 wait complete. 2024-08-19T01:43:22.356 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:21 smithi190 ceph-mon[22032]: pgmap v566: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:43:22.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:21 smithi146 ceph-mon[22098]: pgmap v566: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 -- 172.21.15.103:0/2921949304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec103380 msgr2=0x7f0fec10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/2921949304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec103380 0x7f0fec10b870 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009900 tx=0x7f0fdc02a480 comp rx=0 tx=0).stop 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 -- 172.21.15.103:0/2921949304 shutdown_connections 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/2921949304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fec10be90 0x7f0fec10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/2921949304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec103380 0x7f0fec10b870 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/2921949304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fec102a60 0x7f0fec102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 -- 172.21.15.103:0/2921949304 >> 172.21.15.103:0/2921949304 conn(0x7f0fec0fc4c0 msgr2=0x7f0fec0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 -- 172.21.15.103:0/2921949304 shutdown_connections 2024-08-19T01:43:22.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.968+0000 7f0ff3d82640 1 -- 172.21.15.103:0/2921949304 wait complete. 2024-08-19T01:43:22.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 Processor -- start 2024-08-19T01:43:22.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 -- start start 2024-08-19T01:43:22.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec102a60 0x7f0fec198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fec103380 0x7f0fec1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fec10be90 0x7f0fec19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec111b40 con 0x7f0fec102a60 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1119c0 con 0x7f0fec10be90 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff3d82640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec111cc0 con 0x7f0fec103380 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec102a60 0x7f0fec198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec102a60 0x7f0fec198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47046/0 (socket says 172.21.15.103:47046) 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 -- 172.21.15.103:0/3347888477 learned_addr learned my addr 172.21.15.103:0/3347888477 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff131d640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fec103380 0x7f0fec1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff231f640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fec10be90 0x7f0fec19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 -- 172.21.15.103:0/3347888477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fec103380 msgr2=0x7f0fec1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fec103380 0x7f0fec1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 -- 172.21.15.103:0/3347888477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fec10be90 msgr2=0x7f0fec19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fec10be90 0x7f0fec19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff1b1e640 1 -- 172.21.15.103:0/3347888477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fec19d030 con 0x7f0fec102a60 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.969+0000 7f0ff231f640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fec10be90 0x7f0fec19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.970+0000 7f0ff131d640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fec103380 0x7f0fec1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.970+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec102a60 0x7f0fec198080 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c970 tx=0x7f0fe800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.970+0000 7f0fe2ffd640 1 -- 172.21.15.103:0/3347888477 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8007bb0 con 0x7f0fec102a60 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.970+0000 7f0fe2ffd640 1 -- 172.21.15.103:0/3347888477 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe8007d50 con 0x7f0fec102a60 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.970+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fec19d320 con 0x7f0fec102a60 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.970+0000 7f0fe2ffd640 1 -- 172.21.15.103:0/3347888477 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800f450 con 0x7f0fec102a60 2024-08-19T01:43:22.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.970+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fec1005f0 con 0x7f0fec102a60 2024-08-19T01:43:22.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.973+0000 7f0fe2ffd640 1 -- 172.21.15.103:0/3347888477 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0fe8004f60 con 0x7f0fec102a60 2024-08-19T01:43:22.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.973+0000 7f0fe2ffd640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0fd80a2b80 0x7f0fd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.973+0000 7f0ff131d640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0fd80a2b80 0x7f0fd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.973+0000 7f0fe2ffd640 1 -- 172.21.15.103:0/3347888477 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0fe80cee60 con 0x7f0fec102a60 2024-08-19T01:43:22.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.974+0000 7f0ff131d640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0fd80a2b80 0x7f0fd80a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc031440 tx=0x7f0fdc005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.977+0000 7f0ff6180740 1 Processor -- start 2024-08-19T01:43:22.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.977+0000 7f0ff6180740 1 -- start start 2024-08-19T01:43:22.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b2083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b208910 0x55b22b212c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b2155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff6180740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b22b16da40 con 0x55b22b2131d0 2024-08-19T01:43:22.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff6180740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b22b1059d0 con 0x55b22b207ff0 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff6180740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b22b169a00 con 0x55b22b208910 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b2155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b2155c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47054/0 (socket says 172.21.15.103:47054) 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 -- 172.21.15.103:0/3914878720 learned_addr learned my addr 172.21.15.103:0/3914878720 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff131d640 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b208910 0x55b22b212c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b2083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 -- 172.21.15.103:0/3914878720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b208910 msgr2=0x55b22b212c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b208910 0x55b22b212c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 -- 172.21.15.103:0/3914878720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 msgr2=0x55b22b2083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b2083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 -- 172.21.15.103:0/3914878720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22b10c650 con 0x55b22b2131d0 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.978+0000 7f0ff231f640 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b2155c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e40 tx=0x7f0fe400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983d0fa940acab33 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0fd7fff640 1 -- 172.21.15.103:0/3914878720 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4004d20 con 0x55b22b2131d0 2024-08-19T01:43:22.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0fd7fff640 1 -- 172.21.15.103:0/3914878720 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe40054d0 con 0x55b22b2131d0 2024-08-19T01:43:22.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0fd7fff640 1 -- 172.21.15.103:0/3914878720 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4011800 con 0x55b22b2131d0 2024-08-19T01:43:22.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0ff6180740 1 -- 172.21.15.103:0/3914878720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 msgr2=0x55b22b2155c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0ff6180740 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b2155c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e40 tx=0x7f0fe400f730 comp rx=0 tx=0).stop 2024-08-19T01:43:22.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0ff6180740 1 -- 172.21.15.103:0/3914878720 shutdown_connections 2024-08-19T01:43:22.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0ff6180740 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b2155c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0ff6180740 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b208910 0x55b22b212c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0ff6180740 1 --2- 172.21.15.103:0/3914878720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b2083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.979+0000 7f0ff6180740 1 -- 172.21.15.103:0/3914878720 >> 172.21.15.103:0/3914878720 conn(0x55b22b204580 msgr2=0x55b22b204960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.980+0000 7f0ff6180740 1 -- 172.21.15.103:0/3914878720 shutdown_connections 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.980+0000 7f0ff6180740 1 -- 172.21.15.103:0/3914878720 wait complete. 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.980+0000 7f0ff6180740 1 -- start start 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.980+0000 7f0ff6180740 1 Processor -- start 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.980+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b207ff0 0x55b22b210060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.980+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b208910 0x55b22b2105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b210ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff6180740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b22b215da0 con 0x55b22b2131d0 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff6180740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b22b128b20 con 0x55b22b208910 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff6180740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b22b215f20 con 0x55b22b207ff0 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff6180740 10 client.0 _ll_register_callbacks cb 0x55b22b126bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff131d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b208910 0x55b22b2105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff1b1e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b207ff0 0x55b22b210060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff131d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b208910 0x55b22b2105a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:44852/0 (socket says 172.21.15.103:44852) 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff1b1e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b207ff0 0x55b22b210060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:37140/0 (socket says 172.21.15.103:37140) 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff131d640 1 -- 172.21.15.103:0/576429135 learned_addr learned my addr 172.21.15.103:0/576429135 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff231f640 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b210ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff1b1e640 1 -- 172.21.15.103:0/576429135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b208910 msgr2=0x55b22b2105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b208910 0x55b22b2105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff1b1e640 1 -- 172.21.15.103:0/576429135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 msgr2=0x55b22b210ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b210ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.981+0000 7f0ff1b1e640 1 -- 172.21.15.103:0/576429135 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22aedf3c0 con 0x55b22b207ff0 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b207ff0 0x55b22b210060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c940 tx=0x7f0fe809bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a85910693d273339 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0fd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0fd57fa640 1 -- 172.21.15.103:0/576429135 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe801d040 con 0x55b22b207ff0 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0fd57fa640 1 -- 172.21.15.103:0/576429135 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0fe80999f0 con 0x55b22b207ff0 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe40054d0 con 0x55b22b207ff0 2024-08-19T01:43:22.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0fd57fa640 1 -- 172.21.15.103:0/576429135 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0fe80a4360 con 0x55b22b207ff0 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0ff6180740 10 client.26575 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b22b2219b0 con 0x55b22b207ff0 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0fd57fa640 1 -- 172.21.15.103:0/576429135 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe40054d0 con 0x55b22b207ff0 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0fd57fa640 1 -- 172.21.15.103:0/576429135 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f0fe809a640 con 0x55b22b207ff0 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0ff6180740 10 client.26575 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.982+0000 7f0ff6180740 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.983+0000 7f0ff6180740 20 client.26575 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.983+0000 7f0ff6180740 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b222e10 0x55b22b225240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.983+0000 7f0ff6180740 4 client.26575 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:22.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.983+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22b17afb0 con 0x55b22b222e10 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.983+0000 7f0ff131d640 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b222e10 0x55b22b225240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.983+0000 7f0fd57fa640 10 client.26575 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.983+0000 7f0ff131d640 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b222e10 0x55b22b225240 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e48ceefe2f612b server_cookie=b7c8750db3dac9c1 in_seq=0 out_seq=0 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.984+0000 7f0fd57fa640 1 -- 172.21.15.103:0/576429135 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b22b17afb0 con 0x55b22b222e10 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.984+0000 7f0fd57fa640 10 client.26575 handle_command_reply: tid=0 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.984+0000 7f0ff6180740 1 client.26575 shutdown 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.984+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b207ff0 msgr2=0x55b22b210060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.984+0000 7f0ff6180740 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b207ff0 0x55b22b210060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c940 tx=0x7f0fe809bf20 comp rx=0 tx=0).stop 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.984+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 shutdown_connections 2024-08-19T01:43:22.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b222e10 0x55b22b225240 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b2131d0 0x55b22b210ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b208910 0x55b22b2105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 1 --2- 172.21.15.103:0/576429135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b207ff0 0x55b22b210060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 >> 172.21.15.103:0/576429135 conn(0x55b22b204580 msgr2=0x55b22b20a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0fd57fa640 0 client.26575 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 shutdown_connections 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 1 -- 172.21.15.103:0/576429135 wait complete. 2024-08-19T01:43:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:22.985+0000 7f0ff6180740 20 client.26575 trim_cache size 0 max 16384 2024-08-19T01:43:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 Processor -- start 2024-08-19T01:43:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 -- start start 2024-08-19T01:43:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b20aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b211ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b22ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1aaff0 con 0x55b22b208910 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1acbc0 con 0x55b22b207ff0 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff6180740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1ab6c0 con 0x55b22b2131d0 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b211ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b211ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47062/0 (socket says 172.21.15.103:47062) 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 -- 172.21.15.103:0/771150685 learned_addr learned my addr 172.21.15.103:0/771150685 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b20aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff231f640 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b22ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 -- 172.21.15.103:0/771150685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 msgr2=0x55b22b22ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b22ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 -- 172.21.15.103:0/771150685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 msgr2=0x55b22b20aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b20aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.001+0000 7f0ff131d640 1 -- 172.21.15.103:0/771150685 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22b208740 con 0x55b22b208910 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.002+0000 7f0ff131d640 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b211ec0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc000c00 tx=0x7f0fdc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1924b2570bde6d97 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.002+0000 7f0fd77fe640 1 -- 172.21.15.103:0/771150685 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc035050 con 0x55b22b208910 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.002+0000 7f0fd77fe640 1 -- 172.21.15.103:0/771150685 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fdc056070 con 0x55b22b208910 2024-08-19T01:43:23.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.002+0000 7f0fd77fe640 1 -- 172.21.15.103:0/771150685 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc051520 con 0x55b22b208910 2024-08-19T01:43:23.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 -- 172.21.15.103:0/771150685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 msgr2=0x55b22b211ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b211ec0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc000c00 tx=0x7f0fdc03f000 comp rx=0 tx=0).stop 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 -- 172.21.15.103:0/771150685 shutdown_connections 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b22ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b211ec0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 --2- 172.21.15.103:0/771150685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b20aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 -- 172.21.15.103:0/771150685 >> 172.21.15.103:0/771150685 conn(0x55b22b204580 msgr2=0x55b22b2117b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 -- 172.21.15.103:0/771150685 shutdown_connections 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 -- 172.21.15.103:0/771150685 wait complete. 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 -- start start 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.003+0000 7f0ff6180740 1 Processor -- start 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b21b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b21b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff6180740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b20c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff6180740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1ac4f0 con 0x55b22b208910 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff6180740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1abd90 con 0x55b22b207ff0 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff6180740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec1a8fd0 con 0x55b22b2131d0 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff6180740 10 client.0 _ll_register_callbacks cb 0x7f0fe809b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b21b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b21b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47068/0 (socket says 172.21.15.103:47068) 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 -- 172.21.15.103:0/2195413272 learned_addr learned my addr 172.21.15.103:0/2195413272 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b21b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff231f640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b20c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 -- 172.21.15.103:0/2195413272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 msgr2=0x55b22b20c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b20c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 -- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 msgr2=0x55b22b21b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b21b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff131d640 1 -- 172.21.15.103:0/2195413272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22b20cf00 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.004+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b21b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff131d640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b21b890 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc032fd0 tx=0x7f0fdc031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea18370a0f2dc926 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0fd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0fd57fa640 1 -- 172.21.15.103:0/2195413272 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc05f060 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0fd57fa640 1 -- 172.21.15.103:0/2195413272 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0fdc003bc0 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fdc056070 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0fd57fa640 1 -- 172.21.15.103:0/2195413272 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0fdc035040 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff6180740 10 client.17373 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b22b28d3e0 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0fd57fa640 1 -- 172.21.15.103:0/2195413272 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fdc056070 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0fd57fa640 1 -- 172.21.15.103:0/2195413272 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f0fdc05a450 con 0x55b22b208910 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff6180740 10 client.17373 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff6180740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:23.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff6180740 20 client.17373 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.005+0000 7f0ff6180740 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b28da90 0x55b22b28feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0ff6180740 4 client.17373 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b22b1754c0 con 0x55b22b28da90 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b28da90 0x55b22b28feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0ff1b1e640 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b28da90 0x55b22b28feb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d125fdc865dc5c7b server_cookie=324f64d3ab8c168a in_seq=0 out_seq=0 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0fd57fa640 10 client.17373 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0fd57fa640 1 -- 172.21.15.103:0/2195413272 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b22b1754c0 con 0x55b22b28da90 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0fd57fa640 10 client.17373 handle_command_reply: tid=0 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0ff6180740 1 client.17373 shutdown 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 msgr2=0x55b22b21b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.006+0000 7f0ff6180740 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b21b890 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc032fd0 tx=0x7f0fdc031980 comp rx=0 tx=0).stop 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 shutdown_connections 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b22b28da90 0x55b22b28feb0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b22b2131d0 0x55b22b20c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b22b208910 0x55b22b21b890 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0fd57fa640 0 client.17373 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 --2- 172.21.15.103:0/2195413272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b22b207ff0 0x55b22b21b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 >> 172.21.15.103:0/2195413272 conn(0x55b22b204580 msgr2=0x55b22b2035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 shutdown_connections 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 1 -- 172.21.15.103:0/2195413272 wait complete. 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff6180740 20 client.17373 trim_cache size 0 max 16384 2024-08-19T01:43:23.013 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0fd80a2b80 msgr2=0x7f0fd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0fd80a2b80 0x7f0fd80a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc031440 tx=0x7f0fdc005a20 comp rx=0 tx=0).stop 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec102a60 msgr2=0x7f0fec198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.007+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec102a60 0x7f0fec198080 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c970 tx=0x7f0fe800ce30 comp rx=0 tx=0).stop 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 shutdown_connections 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0fd80a2b80 0x7f0fd80a5040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fec10be90 0x7f0fec19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fec103380 0x7f0fec1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 --2- 172.21.15.103:0/3347888477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fec102a60 0x7f0fec198080 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:23.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 >> 172.21.15.103:0/3347888477 conn(0x7f0fec0fc4c0 msgr2=0x7f0fec0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:23.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 shutdown_connections 2024-08-19T01:43:23.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:23.008+0000 7f0ff3d82640 1 -- 172.21.15.103:0/3347888477 wait complete. 2024-08-19T01:43:24.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:23 smithi103 ceph-mon[18436]: pgmap v567: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-08-19T01:43:24.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:23 smithi190 ceph-mon[22032]: pgmap v567: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-08-19T01:43:24.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:23 smithi146 ceph-mon[22098]: pgmap v567: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-08-19T01:43:24.830 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:43:24.830 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:43:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:25.252 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:25 smithi103 ceph-mon[18436]: pgmap v568: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 MiB/s wr, 22 op/s 2024-08-19T01:43:25.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:25 smithi190 ceph-mon[22032]: pgmap v568: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 MiB/s wr, 22 op/s 2024-08-19T01:43:25.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:25 smithi146 ceph-mon[22098]: pgmap v568: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 MiB/s wr, 22 op/s 2024-08-19T01:43:27.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 -- 172.21.15.103:0/2236640890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 msgr2=0x7fefc4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 --2- 172.21.15.103:0/2236640890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4102ea0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009930 tx=0x7fefb402a420 comp rx=0 tx=0).stop 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 -- 172.21.15.103:0/2236640890 shutdown_connections 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 --2- 172.21.15.103:0/2236640890 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefc4107fc0 0x7fefc410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 --2- 172.21.15.103:0/2236640890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4103470 0x7fefc41078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 --2- 172.21.15.103:0/2236640890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4102ea0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 -- 172.21.15.103:0/2236640890 >> 172.21.15.103:0/2236640890 conn(0x7fefc40fc500 msgr2=0x7fefc40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 -- 172.21.15.103:0/2236640890 shutdown_connections 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.410+0000 7fefc892e640 1 -- 172.21.15.103:0/2236640890 wait complete. 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.411+0000 7fefc892e640 1 Processor -- start 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.411+0000 7fefc892e640 1 -- start start 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc892e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc892e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefc4103470 0x7fefc4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc892e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4107fc0 0x7fefc419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc892e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4111cb0 con 0x7fefc4102ac0 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc892e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4111b30 con 0x7fefc4103470 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc892e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4111e30 con 0x7fefc4107fc0 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47098/0 (socket says 172.21.15.103:47098) 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 -- 172.21.15.103:0/2220065627 learned_addr learned my addr 172.21.15.103:0/2220065627 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc2d9d640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4107fc0 0x7fefc419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefc4103470 0x7fefc4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 -- 172.21.15.103:0/2220065627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4107fc0 msgr2=0x7fefc419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4107fc0 0x7fefc419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 -- 172.21.15.103:0/2220065627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefc4103470 msgr2=0x7fefc4198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefc4103470 0x7fefc4198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 -- 172.21.15.103:0/2220065627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc419d2b0 con 0x7fefc4102ac0 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.412+0000 7fefc259c640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4198300 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fefb4005940 tx=0x7fefb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.413+0000 7fefb37fe640 1 -- 172.21.15.103:0/2220065627 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb403f030 con 0x7fefc4102ac0 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.413+0000 7fefb37fe640 1 -- 172.21.15.103:0/2220065627 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefb4003cb0 con 0x7fefc4102ac0 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.413+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc419d540 con 0x7fefc4102ac0 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.413+0000 7fefb37fe640 1 -- 172.21.15.103:0/2220065627 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4035070 con 0x7fefc4102ac0 2024-08-19T01:43:27.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.413+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc4100650 con 0x7fefc4102ac0 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.415+0000 7fefb37fe640 1 -- 172.21.15.103:0/2220065627 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fefb4042030 con 0x7fefc4102ac0 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 Processor -- start 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 -- start start 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78b20 0x56361ac82ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac857d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56361ac3be40 con 0x56361ac78200 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56361abdd7f0 con 0x56361ac78b20 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefcad2c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56361ab84fb0 con 0x56361ac833e0 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefc2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefc2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac785e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47102/0 (socket says 172.21.15.103:47102) 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefc2d9d640 1 -- 172.21.15.103:0/1137456613 learned_addr learned my addr 172.21.15.103:0/1137456613 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefc259c640 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78b20 0x56361ac82ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.416+0000 7fefb37fe640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fefac0a2c00 0x7fefac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac857d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc2d9d640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fefac0a2c00 0x7fefac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefb37fe640 1 -- 172.21.15.103:0/2220065627 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fefb40ef440 con 0x7fefc4102ac0 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc259c640 1 -- 172.21.15.103:0/1137456613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 msgr2=0x56361ac857d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc259c640 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac857d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc259c640 1 -- 172.21.15.103:0/1137456613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 msgr2=0x56361ac785e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc259c640 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac785e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc259c640 1 -- 172.21.15.103:0/1137456613 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56361a94f6a0 con 0x56361ac78b20 2024-08-19T01:43:27.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc259c640 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78b20 0x56361ac82ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefb40ecfc0 tx=0x7fefb40b7970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6b64a78990863a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefc2d9d640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fefac0a2c00 0x7fefac0a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fefbc013c60 tx=0x7fefbc004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefb0ff9640 1 -- 172.21.15.103:0/1137456613 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4049070 con 0x56361ac78b20 2024-08-19T01:43:27.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefb0ff9640 1 -- 172.21.15.103:0/1137456613 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefb40bc3d0 con 0x56361ac78b20 2024-08-19T01:43:27.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.417+0000 7fefb0ff9640 1 -- 172.21.15.103:0/1137456613 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb40b8340 con 0x56361ac78b20 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 -- 172.21.15.103:0/1137456613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78b20 msgr2=0x56361ac82ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78b20 0x56361ac82ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefb40ecfc0 tx=0x7fefb40b7970 comp rx=0 tx=0).stop 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 -- 172.21.15.103:0/1137456613 shutdown_connections 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac857d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78b20 0x56361ac82ea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1137456613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 -- 172.21.15.103:0/1137456613 >> 172.21.15.103:0/1137456613 conn(0x56361ac74790 msgr2=0x56361ac74b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 -- 172.21.15.103:0/1137456613 shutdown_connections 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.418+0000 7fefcad2c740 1 -- 172.21.15.103:0/1137456613 wait complete. 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 -- start start 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 Processor -- start 2024-08-19T01:43:27.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac80270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac80cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56361ab98cb0 con 0x56361ac78200 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56361abd97b0 con 0x56361ac833e0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56361ac85d70 con 0x56361ac78b20 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefcad2c740 10 client.0 _ll_register_callbacks cb 0x56361a8ecba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefc259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac80270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefc259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac80270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47114/0 (socket says 172.21.15.103:47114) 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefc259c640 1 -- 172.21.15.103:0/3754420590 learned_addr learned my addr 172.21.15.103:0/3754420590 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.419+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc2d9d640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac80cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc259c640 1 -- 172.21.15.103:0/3754420590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 msgr2=0x56361ac807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc259c640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac807b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc259c640 1 -- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 msgr2=0x56361ac80cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc259c640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac80cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc259c640 1 -- 172.21.15.103:0/3754420590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56361ab97bc0 con 0x56361ac78200 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc2d9d640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac80cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefc259c640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac80270 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fefb40bb9d0 tx=0x7fefb40044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a43d41dad5e195 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefa9ffb640 1 -- 172.21.15.103:0/3754420590 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4049070 con 0x56361ac78200 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefa9ffb640 1 -- 172.21.15.103:0/3754420590 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fefb40ed080 con 0x56361ac78200 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb40bc3d0 con 0x56361ac78200 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefa9ffb640 1 -- 172.21.15.103:0/3754420590 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fefb40c4940 con 0x56361ac78200 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefcad2c740 10 client.17385 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56361ac91b30 con 0x56361ac78200 2024-08-19T01:43:27.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.420+0000 7fefa9ffb640 1 -- 172.21.15.103:0/3754420590 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefb40bc3d0 con 0x56361ac78200 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefa9ffb640 1 -- 172.21.15.103:0/3754420590 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fefb403bb80 con 0x56361ac78200 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefcad2c740 10 client.17385 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefcad2c740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefcad2c740 20 client.17385 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ac95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefcad2c740 4 client.17385 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56361abead60 con 0x56361ac92e80 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.421+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ac95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ac95270 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0e4617360060cd server_cookie=e6f67b2ae44a5a7a in_seq=0 out_seq=0 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefa9ffb640 10 client.17385 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefa9ffb640 1 -- 172.21.15.103:0/3754420590 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56361abead60 con 0x56361ac92e80 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefa9ffb640 10 client.17385 handle_command_reply: tid=0 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 client.17385 shutdown 2024-08-19T01:43:27.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 msgr2=0x56361ac80270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac80270 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fefb40bb9d0 tx=0x7fefb40044e0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 shutdown_connections 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ac95270 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac80cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefa9ffb640 0 client.17385 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3754420590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac80270 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.422+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 >> 172.21.15.103:0/3754420590 conn(0x56361ac74790 msgr2=0x56361ac7a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.423+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 shutdown_connections 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.423+0000 7fefcad2c740 1 -- 172.21.15.103:0/3754420590 wait complete. 2024-08-19T01:43:27.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.423+0000 7fefcad2c740 20 client.17385 trim_cache size 0 max 16384 2024-08-19T01:43:27.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.446+0000 7fefcad2c740 1 Processor -- start 2024-08-19T01:43:27.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.446+0000 7fefcad2c740 1 -- start start 2024-08-19T01:43:27.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac9aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefcad2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41aafb0 con 0x56361ac78200 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefcad2c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41acb80 con 0x56361ac833e0 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefcad2c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ab680 con 0x56361ac78b20 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac821e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47124/0 (socket says 172.21.15.103:47124) 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 -- 172.21.15.103:0/3211245862 learned_addr learned my addr 172.21.15.103:0/3211245862 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac9aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc2d9d640 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 -- 172.21.15.103:0/3211245862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 msgr2=0x56361ac9aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac9aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 -- 172.21.15.103:0/3211245862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 msgr2=0x56361ac9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 -- 172.21.15.103:0/3211245862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56361ac79710 con 0x56361ac78200 2024-08-19T01:43:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac9aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.447+0000 7fefc259c640 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac821e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fefb403bf40 tx=0x7fefb40ef770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd76f2a9e2bf35a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.448+0000 7fefabfff640 1 -- 172.21.15.103:0/3211245862 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4049070 con 0x56361ac78200 2024-08-19T01:43:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.448+0000 7fefabfff640 1 -- 172.21.15.103:0/3211245862 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefb40bc3d0 con 0x56361ac78200 2024-08-19T01:43:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.448+0000 7fefabfff640 1 -- 172.21.15.103:0/3211245862 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb40ef110 con 0x56361ac78200 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 -- 172.21.15.103:0/3211245862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 msgr2=0x56361ac821e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac821e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fefb403bf40 tx=0x7fefb40ef770 comp rx=0 tx=0).stop 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 -- 172.21.15.103:0/3211245862 shutdown_connections 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac833e0 0x56361ac9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac78b20 0x56361ac9aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 --2- 172.21.15.103:0/3211245862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78200 0x56361ac821e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 -- 172.21.15.103:0/3211245862 >> 172.21.15.103:0/3211245862 conn(0x56361ac74790 msgr2=0x56361ac81ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 -- 172.21.15.103:0/3211245862 shutdown_connections 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 -- 172.21.15.103:0/3211245862 wait complete. 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 -- start start 2024-08-19T01:43:27.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.449+0000 7fefcad2c740 1 Processor -- start 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78200 0x56361ac8b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78b20 0x56361ac869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefcad2c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac86ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefcad2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ac4b0 con 0x56361ac78b20 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefcad2c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41abd50 con 0x56361ac78200 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefcad2c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41a8f90 con 0x56361ac833e0 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefcad2c740 10 client.0 _ll_register_callbacks cb 0x7fefbc00a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78b20 0x56361ac869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78b20 0x56361ac869a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47140/0 (socket says 172.21.15.103:47140) 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 -- 172.21.15.103:0/1916239744 learned_addr learned my addr 172.21.15.103:0/1916239744 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc2d9d640 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac86ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc259c640 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78200 0x56361ac8b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 -- 172.21.15.103:0/1916239744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 msgr2=0x56361ac86ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac86ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 -- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78200 msgr2=0x56361ac8b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78200 0x56361ac8b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.450+0000 7fefc1d9b640 1 -- 172.21.15.103:0/1916239744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56361ac87710 con 0x56361ac78b20 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefc1d9b640 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78b20 0x56361ac869a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fefc40fec90 tx=0x7fefb800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb2d9aa27169fdb server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefa9ffb640 1 -- 172.21.15.103:0/1916239744 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb801a810 con 0x56361ac78b20 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefa9ffb640 1 -- 172.21.15.103:0/1916239744 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fefb801a9b0 con 0x56361ac78b20 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb40bc3d0 con 0x56361ac78b20 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefa9ffb640 1 -- 172.21.15.103:0/1916239744 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fefb8031050 con 0x56361ac78b20 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefcad2c740 10 client.17391 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56361acfdad0 con 0x56361ac78b20 2024-08-19T01:43:27.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefa9ffb640 1 -- 172.21.15.103:0/1916239744 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefb8025070 con 0x56361ac78b20 2024-08-19T01:43:27.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.451+0000 7fefa9ffb640 1 -- 172.21.15.103:0/1916239744 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fefb802ac70 con 0x56361ac78b20 2024-08-19T01:43:27.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefcad2c740 10 client.17391 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:27.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefcad2c740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:27.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefcad2c740 20 client.17391 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:27.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ad002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:27.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefcad2c740 4 client.17391 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fefb40bc3d0 con 0x56361ac92e80 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefc259c640 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ad002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefc259c640 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ad002c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da82d9c41f5bbd1 server_cookie=cfe2cac2a5538973 in_seq=0 out_seq=0 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.452+0000 7fefa9ffb640 10 client.17391 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.453+0000 7fefa9ffb640 1 -- 172.21.15.103:0/1916239744 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fefb40bc3d0 con 0x56361ac92e80 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.453+0000 7fefa9ffb640 10 client.17391 handle_command_reply: tid=0 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.453+0000 7fefcad2c740 1 client.17391 shutdown 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78b20 msgr2=0x56361ac869a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78b20 0x56361ac869a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fefc40fec90 tx=0x7fefb800d520 comp rx=0 tx=0).stop 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 shutdown_connections 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56361ac92e80 0x56361ad002c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361ac833e0 0x56361ac86ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56361ac78b20 0x56361ac869a0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 --2- 172.21.15.103:0/1916239744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56361ac78200 0x56361ac8b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 >> 172.21.15.103:0/1916239744 conn(0x56361ac74790 msgr2=0x56361ac738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefa9ffb640 0 client.17391 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 shutdown_connections 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 1 -- 172.21.15.103:0/1916239744 wait complete. 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.454+0000 7fefcad2c740 20 client.17391 trim_cache size 0 max 16384 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "bb16fdb1-952a-4873-99b9-b05d6d9c6076", 2024-08-19T01:43:27.459 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:43:27.460 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:43:27.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fefac0a2c00 msgr2=0x7fefac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fefac0a2c00 0x7fefac0a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fefbc013c60 tx=0x7fefbc004ab0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 msgr2=0x7fefc4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:27.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4198300 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fefb4005940 tx=0x7fefb40042f0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 shutdown_connections 2024-08-19T01:43:27.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fefac0a2c00 0x7fefac0a50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4107fc0 0x7fefc419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fefc4103470 0x7fefc4198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 --2- 172.21.15.103:0/2220065627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc4102ac0 0x7fefc4198300 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:27.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 >> 172.21.15.103:0/2220065627 conn(0x7fefc40fc500 msgr2=0x7fefc4105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:27.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 shutdown_connections 2024-08-19T01:43:27.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:27.455+0000 7fefc892e640 1 -- 172.21.15.103:0/2220065627 wait complete. 2024-08-19T01:43:28.243 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:27 smithi103 ceph-mon[18436]: pgmap v569: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:43:28.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:27 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:43:28.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:27 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:43:28.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:27 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:43:28.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:27 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:27 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.244 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:27 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:27 smithi190 ceph-mon[22032]: pgmap v569: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:43:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:27 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:43:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:27 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:43:28.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:27 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:43:28.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:27 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:27 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:27 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:27 smithi146 ceph-mon[22098]: pgmap v569: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-08-19T01:43:28.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:27 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:43:28.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:27 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:43:28.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:27 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:43:28.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:27 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:27 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:27 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:28.615 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:43:28.896 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:43:29.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:43:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:43:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:43:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:43:29.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:28 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:43:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:43:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:43:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:43:29.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:28 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:43:29.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:43:29.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:43:29.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:43:29.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:28 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 -- 172.21.15.103:0/610163872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 msgr2=0x7f084c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 --2- 172.21.15.103:0/610163872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c100c10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0838009930 tx=0x7f083802a420 comp rx=0 tx=0).stop 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 -- 172.21.15.103:0/610163872 shutdown_connections 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 --2- 172.21.15.103:0/610163872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f084c101150 0x7f084c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 --2- 172.21.15.103:0/610163872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c100c10 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 --2- 172.21.15.103:0/610163872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c1067b0 0x7f084c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 -- 172.21.15.103:0/610163872 >> 172.21.15.103:0/610163872 conn(0x7f084c0fc4d0 msgr2=0x7f084c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 -- 172.21.15.103:0/610163872 shutdown_connections 2024-08-19T01:43:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.434+0000 7f0852da6640 1 -- 172.21.15.103:0/610163872 wait complete. 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 Processor -- start 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 -- start start 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c101150 0x7f084c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f084c1067b0 0x7f084c19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f084c110a20 con 0x7f084c1007b0 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1108a0 con 0x7f084c1067b0 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0852da6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f084c110ba0 con 0x7f084c101150 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0850b42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0850b42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47158/0 (socket says 172.21.15.103:47158) 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.435+0000 7f0850b42640 1 -- 172.21.15.103:0/1777070746 learned_addr learned my addr 172.21.15.103:0/1777070746 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f084bfff640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c101150 0x7f084c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0851343640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f084c1067b0 0x7f084c19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0850b42640 1 -- 172.21.15.103:0/1777070746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c101150 msgr2=0x7f084c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0850b42640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c101150 0x7f084c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0850b42640 1 -- 172.21.15.103:0/1777070746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f084c1067b0 msgr2=0x7f084c19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0850b42640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f084c1067b0 0x7f084c19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0850b42640 1 -- 172.21.15.103:0/1777070746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c19bf60 con 0x7f084c1007b0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f084bfff640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c101150 0x7f084c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0850b42640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c196fb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0840009900 tx=0x7f0840009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0849ffb640 1 -- 172.21.15.103:0/1777070746 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084000d030 con 0x7f084c1007b0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0849ffb640 1 -- 172.21.15.103:0/1777070746 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f084000d1d0 con 0x7f084c1007b0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0849ffb640 1 -- 172.21.15.103:0/1777070746 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840005280 con 0x7f084c1007b0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c19c250 con 0x7f084c1007b0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.436+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c06a4c0 con 0x7f084c1007b0 2024-08-19T01:43:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.439+0000 7f0849ffb640 1 -- 172.21.15.103:0/1777070746 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0840005420 con 0x7f084c1007b0 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.439+0000 7f0849ffb640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f083c0a2c00 0x7f083c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.439+0000 7f0849ffb640 1 -- 172.21.15.103:0/1777070746 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f08400d0350 con 0x7f084c1007b0 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.439+0000 7f084bfff640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f083c0a2c00 0x7f083c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 Processor -- start 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f084bfff640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f083c0a2c00 0x7f083c0a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0838000c00 tx=0x7f0838009610 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 -- start start 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477f0210 0x55bc477f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0b30 0x55bc477faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bc47755980 con 0x55bc477fb3f0 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bc476eda70 con 0x55bc477f0b30 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f08551a4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bc47751940 con 0x55bc477f0210 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f0851343640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f0851343640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47166/0 (socket says 172.21.15.103:47166) 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f0850b42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477f0210 0x55bc477f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.440+0000 7f0851343640 1 -- 172.21.15.103:0/4233400929 learned_addr learned my addr 172.21.15.103:0/4233400929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f084bfff640 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0b30 0x55bc477faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0850b42640 1 -- 172.21.15.103:0/4233400929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0b30 msgr2=0x55bc477faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0850b42640 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0b30 0x55bc477faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0850b42640 1 -- 172.21.15.103:0/4233400929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 msgr2=0x55bc477fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0850b42640 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0850b42640 1 -- 172.21.15.103:0/4233400929 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc477506e0 con 0x55bc477f0210 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0851343640 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0850b42640 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477f0210 0x55bc477f05f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f08400cde80 tx=0x7f084009cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b52035edd6ef423 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0836ffd640 1 -- 172.21.15.103:0/4233400929 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840087050 con 0x55bc477f0210 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0836ffd640 1 -- 172.21.15.103:0/4233400929 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f084009cbe0 con 0x55bc477f0210 2024-08-19T01:43:29.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.441+0000 7f0836ffd640 1 -- 172.21.15.103:0/4233400929 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084001e040 con 0x55bc477f0210 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 -- 172.21.15.103:0/4233400929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477f0210 msgr2=0x55bc477f05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477f0210 0x55bc477f05f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f08400cde80 tx=0x7f084009cac0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 -- 172.21.15.103:0/4233400929 shutdown_connections 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0b30 0x55bc477faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 --2- 172.21.15.103:0/4233400929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477f0210 0x55bc477f05f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 -- 172.21.15.103:0/4233400929 >> 172.21.15.103:0/4233400929 conn(0x55bc477ec7a0 msgr2=0x55bc477ecb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 -- 172.21.15.103:0/4233400929 shutdown_connections 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.442+0000 7f08551a4740 1 -- 172.21.15.103:0/4233400929 wait complete. 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.443+0000 7f08551a4740 1 -- start start 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.443+0000 7f08551a4740 1 Processor -- start 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.443+0000 7f08551a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.443+0000 7f08551a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0b30 0x55bc477f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.443+0000 7f08551a4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f08551a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bc477fdfc0 con 0x55bc477f0b30 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f08551a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bc47710aa0 con 0x55bc477f0210 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f08551a4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bc477fe140 con 0x55bc477fb3f0 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0b30 0x55bc477f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0b30 0x55bc477f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47168/0 (socket says 172.21.15.103:47168) 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 -- 172.21.15.103:0/1296474860 learned_addr learned my addr 172.21.15.103:0/1296474860 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f08551a4740 10 client.0 _ll_register_callbacks cb 0x55bc476ffe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f0850b42640 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f0851343640 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 -- 172.21.15.103:0/1296474860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 msgr2=0x55bc477f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 -- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 msgr2=0x55bc477f82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477f82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 -- 172.21.15.103:0/1296474860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc476f4650 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f084bfff640 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0b30 0x55bc477f8830 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0838009930 tx=0x7f0838009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962471830ca13e6b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f082bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f082bfff640 1 -- 172.21.15.103:0/1296474860 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083804c070 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.444+0000 7f082bfff640 1 -- 172.21.15.103:0/1296474860 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0838005590 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084009cbe0 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f082bfff640 1 -- 172.21.15.103:0/1296474860 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0838058050 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 10 client.17403 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc474c73c0 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f082bfff640 1 -- 172.21.15.103:0/1296474860 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f084009cbe0 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f082bfff640 1 -- 172.21.15.103:0/1296474860 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f0838003680 con 0x55bc477f0b30 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 10 client.17403 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 20 client.17403 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc4780aef0 0x55bc4780d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 4 client.17403 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.445+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc47762ef0 con 0x55bc4780aef0 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.446+0000 7f0850b42640 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc4780aef0 0x55bc4780d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.446+0000 7f082bfff640 10 client.17403 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.446+0000 7f0850b42640 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc4780aef0 0x55bc4780d330 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b23c9c088bb90f server_cookie=f92cd421f786898e in_seq=0 out_seq=0 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f082bfff640 1 -- 172.21.15.103:0/1296474860 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc47762ef0 con 0x55bc4780aef0 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f082bfff640 10 client.17403 handle_command_reply: tid=0 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 client.17403 shutdown 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0b30 msgr2=0x55bc477f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0b30 0x55bc477f8830 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0838009930 tx=0x7f0838009260 comp rx=0 tx=0).stop 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 shutdown_connections 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc4780aef0 0x55bc4780d330 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0b30 0x55bc477f8830 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 --2- 172.21.15.103:0/1296474860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 >> 172.21.15.103:0/1296474860 conn(0x55bc477ec7a0 msgr2=0x55bc477f2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f082bfff640 0 client.17403 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 shutdown_connections 2024-08-19T01:43:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 1 -- 172.21.15.103:0/1296474860 wait complete. 2024-08-19T01:43:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.447+0000 7f08551a4740 20 client.17403 trim_cache size 0 max 16384 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 Processor -- start 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 -- start start 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc47812c00 0x55bc47813060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1aaf90 con 0x55bc477fb3f0 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1acb60 con 0x55bc477f0210 2024-08-19T01:43:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.467+0000 7f08551a4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1ab660 con 0x55bc47812c00 2024-08-19T01:43:29.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47180/0 (socket says 172.21.15.103:47180) 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 -- 172.21.15.103:0/4021576878 learned_addr learned my addr 172.21.15.103:0/4021576878 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f0850b42640 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f0851343640 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc47812c00 0x55bc47813060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 -- 172.21.15.103:0/4021576878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc47812c00 msgr2=0x55bc47813060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc47812c00 0x55bc47813060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 -- 172.21.15.103:0/4021576878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 msgr2=0x55bc477ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 -- 172.21.15.103:0/4021576878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc477f72a0 con 0x55bc477fb3f0 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f0850b42640 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f084bfff640 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fa150 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0838003aa0 tx=0x7f083802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226403249e853ca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f08367fc640 1 -- 172.21.15.103:0/4021576878 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083804c040 con 0x55bc477fb3f0 2024-08-19T01:43:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f08367fc640 1 -- 172.21.15.103:0/4021576878 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0838005410 con 0x55bc477fb3f0 2024-08-19T01:43:29.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.468+0000 7f08367fc640 1 -- 172.21.15.103:0/4021576878 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838035050 con 0x55bc477fb3f0 2024-08-19T01:43:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 -- 172.21.15.103:0/4021576878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 msgr2=0x55bc477fa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fa150 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0838003aa0 tx=0x7f083802a420 comp rx=0 tx=0).stop 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 -- 172.21.15.103:0/4021576878 shutdown_connections 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc47812c00 0x55bc47813060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477fb3f0 0x55bc477fa150 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 --2- 172.21.15.103:0/4021576878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc477f0210 0x55bc477ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 -- 172.21.15.103:0/4021576878 >> 172.21.15.103:0/4021576878 conn(0x55bc477ec7a0 msgr2=0x55bc477f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 -- 172.21.15.103:0/4021576878 shutdown_connections 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.469+0000 7f08551a4740 1 -- 172.21.15.103:0/4021576878 wait complete. 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 -- start start 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 Processor -- start 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0210 0x55bc477ff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc47812c00 0x55bc47874bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1ac490 con 0x55bc477f0210 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1abd30 con 0x55bc47812c00 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f084c1a8f70 con 0x55bc477fb3f0 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f08551a4740 10 client.0 _ll_register_callbacks cb 0x7f083803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f0850b42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0210 0x55bc477ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f0850b42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0210 0x55bc477ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47190/0 (socket says 172.21.15.103:47190) 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.470+0000 7f0850b42640 1 -- 172.21.15.103:0/786988051 learned_addr learned my addr 172.21.15.103:0/786988051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f0851343640 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc47812c00 0x55bc47874bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f084bfff640 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f0850b42640 1 -- 172.21.15.103:0/786988051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 msgr2=0x55bc477ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f0850b42640 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f0850b42640 1 -- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc47812c00 msgr2=0x55bc47874bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f0850b42640 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc47812c00 0x55bc47874bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f0850b42640 1 -- 172.21.15.103:0/786988051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc47875110 con 0x55bc477f0210 2024-08-19T01:43:29.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f0850b42640 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0210 0x55bc477ff820 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f08400cded0 tx=0x7f0840005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04aec850cb3d397 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f082bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f082bfff640 1 -- 172.21.15.103:0/786988051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840084060 con 0x55bc477f0210 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f082bfff640 1 -- 172.21.15.103:0/786988051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0840099330 con 0x55bc477f0210 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0838005410 con 0x55bc477f0210 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f082bfff640 1 -- 172.21.15.103:0/786988051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f084007fe60 con 0x55bc477f0210 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f08551a4740 10 client.17409 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.471+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bc47876380 con 0x55bc477f0210 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f082bfff640 1 -- 172.21.15.103:0/786988051 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0838005410 con 0x55bc477f0210 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f082bfff640 1 -- 172.21.15.103:0/786988051 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f0840098930 con 0x55bc477f0210 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f08551a4740 10 client.17409 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f08551a4740 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f08551a4740 20 client.17409 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:29.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f08551a4740 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc47876670 0x55bc47878a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f08551a4740 4 client.17409 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc4775d400 con 0x55bc47876670 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.472+0000 7f084bfff640 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc47876670 0x55bc47878a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f082bfff640 10 client.17409 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f084bfff640 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc47876670 0x55bc47878a60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aeb892aef49640 server_cookie=ca0e0d2618cd9ebb in_seq=0 out_seq=0 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f082bfff640 1 -- 172.21.15.103:0/786988051 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bc4775d400 con 0x55bc47876670 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f082bfff640 10 client.17409 handle_command_reply: tid=0 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 client.17409 shutdown 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0210 msgr2=0x55bc477ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0210 0x55bc477ff820 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f08400cded0 tx=0x7f0840005710 comp rx=0 tx=0).stop 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 shutdown_connections 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55bc47876670 0x55bc47878a60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bc47812c00 0x55bc47874bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f082bfff640 0 client.17409 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bc477fb3f0 0x55bc477ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 --2- 172.21.15.103:0/786988051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc477f0210 0x55bc477ff820 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 >> 172.21.15.103:0/786988051 conn(0x55bc477ec7a0 msgr2=0x55bc47811bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 shutdown_connections 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 1 -- 172.21.15.103:0/786988051 wait complete. 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.473+0000 7f08551a4740 20 client.17409 trim_cache size 0 max 16384 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:43:29.480 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:43:29.481 INFO:teuthology.orchestra.run.smithi103.stdout: "bb16fdb1-952a-4873-99b9-b05d6d9c6076": { 2024-08-19T01:43:29.481 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:43:29.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "bb16fdb1-952a-4873-99b9-b05d6d9c6076", 2024-08-19T01:43:29.481 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:43:29.481 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:43:29.481 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:43:29.481 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f083c0a2c00 msgr2=0x7f083c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f083c0a2c00 0x7f083c0a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0838000c00 tx=0x7f0838009610 comp rx=0 tx=0).stop 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 msgr2=0x7f084c196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c196fb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0840009900 tx=0x7f0840009dc0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 shutdown_connections 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f083c0a2c00 0x7f083c0a50c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f084c1067b0 0x7f084c19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c101150 0x7f084c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 --2- 172.21.15.103:0/1777070746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f084c1007b0 0x7f084c196fb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 >> 172.21.15.103:0/1777070746 conn(0x7f084c0fc4d0 msgr2=0x7f084c0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 shutdown_connections 2024-08-19T01:43:29.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:43:29.474+0000 7f0852da6640 1 -- 172.21.15.103:0/1777070746 wait complete. 2024-08-19T01:43:30.226 INFO:tasks.cephfs.filesystem:scrub status for tag:bb16fdb1-952a-4873-99b9-b05d6d9c6076 - {'path': '/', 'tag': 'bb16fdb1-952a-4873-99b9-b05d6d9c6076', 'options': 'recursive,force'} 2024-08-19T01:43:30.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:29 smithi103 ceph-mon[18436]: pgmap v570: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 26 op/s 2024-08-19T01:43:30.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:29 smithi190 ceph-mon[22032]: pgmap v570: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 26 op/s 2024-08-19T01:43:30.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:29 smithi146 ceph-mon[22098]: pgmap v570: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 26 op/s 2024-08-19T01:43:31.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:30 smithi103 ceph-mon[18436]: pgmap v571: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 35 op/s 2024-08-19T01:43:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:30 smithi190 ceph-mon[22032]: pgmap v571: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 35 op/s 2024-08-19T01:43:31.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:30 smithi146 ceph-mon[22098]: pgmap v571: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 35 op/s 2024-08-19T01:43:34.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:33 smithi103 ceph-mon[18436]: pgmap v572: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 49 op/s 2024-08-19T01:43:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:33 smithi190 ceph-mon[22032]: pgmap v572: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 49 op/s 2024-08-19T01:43:34.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:33 smithi146 ceph-mon[22098]: pgmap v572: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 30 MiB/s wr, 49 op/s 2024-08-19T01:43:35.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:35 smithi103 ceph-mon[18436]: pgmap v573: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-08-19T01:43:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:35 smithi190 ceph-mon[22032]: pgmap v573: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-08-19T01:43:35.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:35 smithi146 ceph-mon[22098]: pgmap v573: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 -- 172.21.15.103:0/1020477852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 msgr2=0x7fd740101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 --2- 172.21.15.103:0/1020477852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740101c90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd73002a410 comp rx=0 tx=0).stop 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 -- 172.21.15.103:0/1020477852 shutdown_connections 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 --2- 172.21.15.103:0/1020477852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740103440 0x7fd740109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 --2- 172.21.15.103:0/1020477852 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd740102a80 0x7fd740102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 --2- 172.21.15.103:0/1020477852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740101c90 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 -- 172.21.15.103:0/1020477852 >> 172.21.15.103:0/1020477852 conn(0x7fd7400fd010 msgr2=0x7fd7400ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 -- 172.21.15.103:0/1020477852 shutdown_connections 2024-08-19T01:43:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 -- 172.21.15.103:0/1020477852 wait complete. 2024-08-19T01:43:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.567+0000 7fd745020640 1 Processor -- start 2024-08-19T01:43:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd745020640 1 -- start start 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd745020640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd745020640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd740102a80 0x7fd740074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd745020640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740103440 0x7fd740074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd745020640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd74010d440 con 0x7fd740101890 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd745020640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd74010d2c0 con 0x7fd740102a80 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd745020640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd74010d5c0 con 0x7fd740103440 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740075cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49886/0 (socket says 172.21.15.103:49886) 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 -- 172.21.15.103:0/2857197466 learned_addr learned my addr 172.21.15.103:0/2857197466 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73e59c640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd740102a80 0x7fd740074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73f59e640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740103440 0x7fd740074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 -- 172.21.15.103:0/2857197466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740103440 msgr2=0x7fd740074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740103440 0x7fd740074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 -- 172.21.15.103:0/2857197466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd740102a80 msgr2=0x7fd740074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd740102a80 0x7fd740074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73ed9d640 1 -- 172.21.15.103:0/2857197466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7400750e0 con 0x7fd740101890 2024-08-19T01:43:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73f59e640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740103440 0x7fd740074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.568+0000 7fd73e59c640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd740102a80 0x7fd740074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.569+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740075cd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd7300314e0 tx=0x7fd7300042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.569+0000 7fd727fff640 1 -- 172.21.15.103:0/2857197466 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73003f030 con 0x7fd740101890 2024-08-19T01:43:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.569+0000 7fd727fff640 1 -- 172.21.15.103:0/2857197466 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd730003cb0 con 0x7fd740101890 2024-08-19T01:43:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.569+0000 7fd727fff640 1 -- 172.21.15.103:0/2857197466 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730035070 con 0x7fd740101890 2024-08-19T01:43:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.569+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7401a1bb0 con 0x7fd740101890 2024-08-19T01:43:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.569+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7401a2100 con 0x7fd740101890 2024-08-19T01:43:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.571+0000 7fd727fff640 1 -- 172.21.15.103:0/2857197466 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd730042030 con 0x7fd740101890 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 Processor -- start 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 -- start start 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6dda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e6e5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fd0e6409d0 con 0x55fd0e6e3050 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fd0e5e8820 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd747dda740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd0e63c990 con 0x55fd0e6dd680 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e6e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e6e5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49898/0 (socket says 172.21.15.103:49898) 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd73e59c640 1 -- 172.21.15.103:0/564655719 learned_addr learned my addr 172.21.15.103:0/564655719 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd73f59e640 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6dda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.572+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd73e59c640 1 -- 172.21.15.103:0/564655719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 msgr2=0x55fd0e6dda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd727fff640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd7280a2c30 0x7fd7280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd73e59c640 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6dda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd73e59c640 1 -- 172.21.15.103:0/564655719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6ddfc0 msgr2=0x55fd0e6e2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd73e59c640 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd73e59c640 1 -- 172.21.15.103:0/564655719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0e354210 con 0x55fd0e6e3050 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd73f59e640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd7280a2c30 0x7fd7280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd727fff640 1 -- 172.21.15.103:0/2857197466 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd7300ef700 con 0x7fd740101890 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd73e59c640 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e6e5510 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd734007800 tx=0x7fd734007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6d64098fd277f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd7257fa640 1 -- 172.21.15.103:0/564655719 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400d950 con 0x55fd0e6e3050 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd7257fa640 1 -- 172.21.15.103:0/564655719 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd734018070 con 0x55fd0e6e3050 2024-08-19T01:43:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.573+0000 7fd7257fa640 1 -- 172.21.15.103:0/564655719 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340123e0 con 0x55fd0e6e3050 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 -- 172.21.15.103:0/564655719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 msgr2=0x55fd0e6e5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd73f59e640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd7280a2c30 0x7fd7280a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd738005fd0 tx=0x7fd738004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e6e5510 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd734007800 tx=0x7fd734007cc0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 -- 172.21.15.103:0/564655719 shutdown_connections 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e6e5510 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 --2- 172.21.15.103:0/564655719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6dda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 -- 172.21.15.103:0/564655719 >> 172.21.15.103:0/564655719 conn(0x55fd0e6d8cf0 msgr2=0x55fd0e6d90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 -- 172.21.15.103:0/564655719 shutdown_connections 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.574+0000 7fd747dda740 1 -- 172.21.15.103:0/564655719 wait complete. 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 -- start start 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 Processor -- start 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6e3050 0x55fd0e6eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fd0e6e6020 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fd0e5fc520 con 0x55fd0e6dd680 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fd0e6e61a0 con 0x55fd0e6e3050 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd747dda740 10 client.0 _ll_register_callbacks cb 0x55fd0e353a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49910/0 (socket says 172.21.15.103:49910) 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 -- 172.21.15.103:0/2041464993 learned_addr learned my addr 172.21.15.103:0/2041464993 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73f59e640 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6e3050 0x55fd0e6eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 -- 172.21.15.103:0/2041464993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6e3050 msgr2=0x55fd0e6eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6e3050 0x55fd0e6eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 -- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 msgr2=0x55fd0e6e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.575+0000 7fd73e59c640 1 -- 172.21.15.103:0/2041464993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0e420620 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd73e59c640 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e7cc0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd734004700 tx=0x7fd734013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171e87304e013ee3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd722ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd722ff5640 1 -- 172.21.15.103:0/2041464993 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73400de20 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd722ff5640 1 -- 172.21.15.103:0/2041464993 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd734024420 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd734018070 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd722ff5640 1 -- 172.21.15.103:0/2041464993 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd734024620 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd747dda740 10 client.17421 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd0e6f1af0 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd722ff5640 1 -- 172.21.15.103:0/2041464993 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd734018070 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd722ff5640 1 -- 172.21.15.103:0/2041464993 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fd734030820 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd747dda740 10 client.17421 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.576+0000 7fd747dda740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.577+0000 7fd747dda740 20 client.17421 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.577+0000 7fd747dda740 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e6f2f60 0x55fd0e6f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.577+0000 7fd747dda740 4 client.17421 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.577+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd0e64e100 con 0x55fd0e6f2f60 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.577+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e6f2f60 0x55fd0e6f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.577+0000 7fd722ff5640 10 client.17421 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.577+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e6f2f60 0x55fd0e6f5350 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec6e9f7af39eae9 server_cookie=d92b883e0d508b2c in_seq=0 out_seq=0 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd722ff5640 1 -- 172.21.15.103:0/2041464993 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd0e64e100 con 0x55fd0e6f2f60 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd722ff5640 10 client.17421 handle_command_reply: tid=0 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 client.17421 shutdown 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 msgr2=0x55fd0e6e7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e7cc0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd734004700 tx=0x7fd734013730 comp rx=0 tx=0).stop 2024-08-19T01:43:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 shutdown_connections 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e6f2f60 0x55fd0e6f5350 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6e3050 0x55fd0e6eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6e7cc0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 --2- 172.21.15.103:0/2041464993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 >> 172.21.15.103:0/2041464993 conn(0x55fd0e6d8cf0 msgr2=0x55fd0e6da500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd722ff5640 0 client.17421 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 shutdown_connections 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 1 -- 172.21.15.103:0/2041464993 wait complete. 2024-08-19T01:43:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.578+0000 7fd747dda740 20 client.17421 trim_cache size 0 max 16384 2024-08-19T01:43:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.594+0000 7fd747dda740 1 Processor -- start 2024-08-19T01:43:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.594+0000 7fd747dda740 1 -- start start 2024-08-19T01:43:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd747dda740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6d90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd747dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd747dda740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6e3050 0x55fd0e6d9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd747dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401aaee0 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd747dda740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401acab0 con 0x55fd0e6e3050 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd747dda740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401ab5b0 con 0x55fd0e6dd680 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49918/0 (socket says 172.21.15.103:49918) 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 -- 172.21.15.103:0/1017880853 learned_addr learned my addr 172.21.15.103:0/1017880853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6d90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73f59e640 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6e3050 0x55fd0e6d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 -- 172.21.15.103:0/1017880853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 msgr2=0x55fd0e6d90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6d90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 -- 172.21.15.103:0/1017880853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6e3050 msgr2=0x55fd0e6d9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6e3050 0x55fd0e6d9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 -- 172.21.15.103:0/1017880853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0e6efe10 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6d90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73f59e640 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6e3050 0x55fd0e6d9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.595+0000 7fd73e59c640 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6eecc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd734006fc0 tx=0x7fd7340245b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2f42cc8ad85226 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd724ff9640 1 -- 172.21.15.103:0/1017880853 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734035030 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd724ff9640 1 -- 172.21.15.103:0/1017880853 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd734018040 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd724ff9640 1 -- 172.21.15.103:0/1017880853 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734024a20 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 -- 172.21.15.103:0/1017880853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 msgr2=0x55fd0e6eecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6eecc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd734006fc0 tx=0x7fd7340245b0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 -- 172.21.15.103:0/1017880853 shutdown_connections 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6e3050 0x55fd0e6d9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6eecc0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 --2- 172.21.15.103:0/1017880853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6dd680 0x55fd0e6d90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 -- 172.21.15.103:0/1017880853 >> 172.21.15.103:0/1017880853 conn(0x55fd0e6d8cf0 msgr2=0x55fd0e6e7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 -- 172.21.15.103:0/1017880853 shutdown_connections 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.596+0000 7fd747dda740 1 -- 172.21.15.103:0/1017880853 wait complete. 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 -- start start 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 Processor -- start 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e75bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401ac3e0 con 0x55fd0e6e3050 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401abc80 con 0x55fd0e6dd680 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7401a8ec0 con 0x55fd0e6ddfc0 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd747dda740 10 client.0 _ll_register_callbacks cb 0x7fd7300f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd73ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e75bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd73ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6eb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:34548/0 (socket says 172.21.15.103:34548) 2024-08-19T01:43:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd73ed9d640 1 -- 172.21.15.103:0/807474492 learned_addr learned my addr 172.21.15.103:0/807474492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd73f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e75bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49926/0 (socket says 172.21.15.103:49926) 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.597+0000 7fd73e59c640 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd73ed9d640 1 -- 172.21.15.103:0/807474492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6ddfc0 msgr2=0x55fd0e6ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6ebe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd73ed9d640 1 -- 172.21.15.103:0/807474492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 msgr2=0x55fd0e75bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e75bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd73ed9d640 1 -- 172.21.15.103:0/807474492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0e6eef20 con 0x55fd0e6dd680 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd73f59e640 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e75bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd73ed9d640 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6eb8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd7300ed230 tx=0x7fd7300f5550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3490396b029290 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd722ff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd722ff5640 1 -- 172.21.15.103:0/807474492 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730049040 con 0x55fd0e6dd680 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd722ff5640 1 -- 172.21.15.103:0/807474492 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fd7300ba290 con 0x55fd0e6dd680 2024-08-19T01:43:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd734018040 con 0x55fd0e6dd680 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd722ff5640 1 -- 172.21.15.103:0/807474492 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd7300bb8f0 con 0x55fd0e6dd680 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd747dda740 10 client.26696 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.598+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd0e75d0c0 con 0x55fd0e6dd680 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd722ff5640 1 -- 172.21.15.103:0/807474492 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd734018040 con 0x55fd0e6dd680 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd722ff5640 1 -- 172.21.15.103:0/807474492 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fd73003bc90 con 0x55fd0e6dd680 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd747dda740 10 client.26696 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd747dda740 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd747dda740 20 client.26696 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd747dda740 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e75da40 0x55fd0e75fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd747dda740 4 client.26696 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd0e648450 con 0x55fd0e75da40 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd73e59c640 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e75da40 0x55fd0e75fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd722ff5640 10 client.26696 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.599+0000 7fd73e59c640 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e75da40 0x55fd0e75fe60 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1412e1f3f71582c6 server_cookie=170cc7da33f97e8 in_seq=0 out_seq=0 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.606+0000 7fd722ff5640 1 -- 172.21.15.103:0/807474492 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd0e648450 con 0x55fd0e75da40 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.606+0000 7fd722ff5640 10 client.26696 handle_command_reply: tid=0 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.606+0000 7fd747dda740 1 client.26696 shutdown 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 msgr2=0x55fd0e6eb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6eb8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd7300ed230 tx=0x7fd7300f5550 comp rx=0 tx=0).stop 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 shutdown_connections 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55fd0e75da40 0x55fd0e75fe60 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd0e6e3050 0x55fd0e75bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd722ff5640 0 client.26696 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fd0e6ddfc0 0x55fd0e6ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 --2- 172.21.15.103:0/807474492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd0e6dd680 0x55fd0e6eb8f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 >> 172.21.15.103:0/807474492 conn(0x55fd0e6d8cf0 msgr2=0x55fd0e6d7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 shutdown_connections 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 1 -- 172.21.15.103:0/807474492 wait complete. 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd747dda740 20 client.26696 trim_cache size 0 max 16384 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:43:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:43:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:43:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd7280a2c30 msgr2=0x7fd7280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd745020640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd7280a2c30 0x7fd7280a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd738005fd0 tx=0x7fd738004a40 comp rx=0 tx=0).stop 2024-08-19T01:43:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 msgr2=0x7fd740075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.607+0000 7fd745020640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740075cd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd7300314e0 tx=0x7fd7300042e0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 shutdown_connections 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fd7280a2c30 0x7fd7280a50f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd740103440 0x7fd740074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd740102a80 0x7fd740074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 --2- 172.21.15.103:0/2857197466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd740101890 0x7fd740075cd0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 >> 172.21.15.103:0/2857197466 conn(0x7fd7400fd010 msgr2=0x7fd740105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 shutdown_connections 2024-08-19T01:43:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:35.608+0000 7fd745020640 1 -- 172.21.15.103:0/2857197466 wait complete. 2024-08-19T01:43:38.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:37 smithi103 ceph-mon[18436]: pgmap v574: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:43:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:37 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:43:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:37 smithi190 ceph-mon[22032]: pgmap v574: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:43:38.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:37 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:43:38.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:37 smithi146 ceph-mon[22098]: pgmap v574: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:43:38.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:37 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:43:39.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:43:39.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:43:39.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:43:39.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:43:39.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:43:39.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:43:39.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:43:39.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:43:39.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:43:40.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:39 smithi103 ceph-mon[18436]: pgmap v575: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:43:40.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:39 smithi190 ceph-mon[22032]: pgmap v575: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:43:40.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:39 smithi146 ceph-mon[22098]: pgmap v575: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 47 op/s 2024-08-19T01:43:41.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:40 smithi103 ceph-mon[18436]: pgmap v576: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-08-19T01:43:41.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:40 smithi190 ceph-mon[22032]: pgmap v576: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-08-19T01:43:41.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:40 smithi146 ceph-mon[22098]: pgmap v576: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 45 op/s 2024-08-19T01:43:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:41 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:43:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:41 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:43:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:41 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:43:42.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:41 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:43:42.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:41 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:43:42.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:41 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:43:42.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:41 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:43:42.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:41 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:43:42.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:41 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:43:42.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:41 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:43:42.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:41 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:43:42.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:41 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:43:43.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:42 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:43:43.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:42 smithi103 ceph-mon[18436]: pgmap v577: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 38 op/s 2024-08-19T01:43:43.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:42 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:43:43.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:42 smithi190 ceph-mon[22032]: pgmap v577: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 38 op/s 2024-08-19T01:43:43.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:42 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:43:43.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:42 smithi146 ceph-mon[22098]: pgmap v577: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 MiB/s wr, 38 op/s 2024-08-19T01:43:46.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:45 smithi103 ceph-mon[18436]: pgmap v578: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:43:46.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:45 smithi190 ceph-mon[22032]: pgmap v578: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:43:46.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:45 smithi146 ceph-mon[22098]: pgmap v578: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:43:47.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:46 smithi103 ceph-mon[18436]: pgmap v579: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:43:47.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:46 smithi190 ceph-mon[22032]: pgmap v579: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:43:47.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:46 smithi146 ceph-mon[22098]: pgmap v579: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 32 MiB/s wr, 25 op/s 2024-08-19T01:43:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 -- 172.21.15.103:0/419425094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88102320 msgr2=0x7f6a88108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 --2- 172.21.15.103:0/419425094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88102320 0x7f6a88108bb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009ca0 tx=0x7f6a8002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 -- 172.21.15.103:0/419425094 shutdown_connections 2024-08-19T01:43:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 --2- 172.21.15.103:0/419425094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88102320 0x7f6a88108bb0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 --2- 172.21.15.103:0/419425094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88101960 0x7f6a88101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 --2- 172.21.15.103:0/419425094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a88100770 0x7f6a88100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 -- 172.21.15.103:0/419425094 >> 172.21.15.103:0/419425094 conn(0x7f6a880fbf10 msgr2=0x7f6a880fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 -- 172.21.15.103:0/419425094 shutdown_connections 2024-08-19T01:43:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.719+0000 7f6a9018f640 1 -- 172.21.15.103:0/419425094 wait complete. 2024-08-19T01:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 Processor -- start 2024-08-19T01:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 -- start start 2024-08-19T01:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88100770 0x7f6a88197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88101960 0x7f6a88197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a88102320 0x7f6a8819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810c320 con 0x7f6a88100770 2024-08-19T01:43:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810c1a0 con 0x7f6a88101960 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a9018f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810c4a0 con 0x7f6a88102320 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88100770 0x7f6a88197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88100770 0x7f6a88197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43954/0 (socket says 172.21.15.103:43954) 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 -- 172.21.15.103:0/3816811713 learned_addr learned my addr 172.21.15.103:0/3816811713 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a88102320 0x7f6a8819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88101960 0x7f6a88197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 -- 172.21.15.103:0/3816811713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a88102320 msgr2=0x7f6a8819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a88102320 0x7f6a8819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 -- 172.21.15.103:0/3816811713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88101960 msgr2=0x7f6a88197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88101960 0x7f6a88197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8df2b640 1 -- 172.21.15.103:0/3816811713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8819f690 con 0x7f6a88100770 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.720+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88101960 0x7f6a88197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.721+0000 7f6a8df2b640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88100770 0x7f6a88197a70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400e9c0 tx=0x7f6a8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.721+0000 7f6a7effd640 1 -- 172.21.15.103:0/3816811713 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400cd80 con 0x7f6a88100770 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.721+0000 7f6a7effd640 1 -- 172.21.15.103:0/3816811713 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a840035f0 con 0x7f6a88100770 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.721+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8819f980 con 0x7f6a88100770 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.721+0000 7f6a7effd640 1 -- 172.21.15.103:0/3816811713 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84010690 con 0x7f6a88100770 2024-08-19T01:43:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.721+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8819fed0 con 0x7f6a88100770 2024-08-19T01:43:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 Processor -- start 2024-08-19T01:43:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 -- start start 2024-08-19T01:43:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee271a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271fc0 0x55deee276a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee277050 0x55deee279510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a7effd640 1 -- 172.21.15.103:0/3816811713 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6a840037a0 con 0x7f6a88100770 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55deee1d49d0 con 0x55deee271680 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55deee17c820 con 0x55deee277050 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a93025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55deee1d0990 con 0x55deee271fc0 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.724+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee271a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee271a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43968/0 (socket says 172.21.15.103:43968) 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2063270731 learned_addr learned my addr 172.21.15.103:0/2063270731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee277050 0x55deee279510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8df2b640 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271fc0 0x55deee276a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2063270731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271fc0 msgr2=0x55deee276a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271fc0 0x55deee276a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2063270731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee277050 msgr2=0x55deee279510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee277050 0x55deee279510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2063270731 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deedee8210 con 0x55deee271680 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8df2b640 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271fc0 0x55deee276a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee271a80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009380 tx=0x7f6a800093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fb8994941f1453 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a7effd640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6a740a2cb0 0x7f6a740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a73fff640 1 -- 172.21.15.103:0/2063270731 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80031700 con 0x55deee271680 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a73fff640 1 -- 172.21.15.103:0/2063270731 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a80040070 con 0x55deee271680 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a73fff640 1 -- 172.21.15.103:0/2063270731 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a800044b0 con 0x55deee271680 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a7effd640 1 -- 172.21.15.103:0/3816811713 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6a840cf610 con 0x7f6a88100770 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.725+0000 7f6a8df2b640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6a740a2cb0 0x7f6a740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 -- 172.21.15.103:0/2063270731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 msgr2=0x55deee271a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee271a80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009380 tx=0x7f6a800093b0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a8df2b640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6a740a2cb0 0x7f6a740a5170 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a840cd4b0 tx=0x7f6a840d3040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 -- 172.21.15.103:0/2063270731 shutdown_connections 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee277050 0x55deee279510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271fc0 0x55deee276a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 --2- 172.21.15.103:0/2063270731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee271a80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 -- 172.21.15.103:0/2063270731 >> 172.21.15.103:0/2063270731 conn(0x55deee26ccf0 msgr2=0x55deee26d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 -- 172.21.15.103:0/2063270731 shutdown_connections 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.726+0000 7f6a93025740 1 -- 172.21.15.103:0/2063270731 wait complete. 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 -- start start 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 Processor -- start 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271680 0x55deee27b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee27bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee277050 0x55deee282d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55deee279fc0 con 0x55deee277050 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55deee190520 con 0x55deee271fc0 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55deee27a140 con 0x55deee271680 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a93025740 10 client.0 _ll_register_callbacks cb 0x55deedee7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee277050 0x55deee282d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee277050 0x55deee282d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43978/0 (socket says 172.21.15.103:43978) 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a8e72c640 1 -- 172.21.15.103:0/4143603238 learned_addr learned my addr 172.21.15.103:0/4143603238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.727+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271680 0x55deee27b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee27bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a8e72c640 1 -- 172.21.15.103:0/4143603238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271680 msgr2=0x55deee27b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271680 0x55deee27b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a8e72c640 1 -- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 msgr2=0x55deee27bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee27bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a8e72c640 1 -- 172.21.15.103:0/4143603238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deedfb4620 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee277050 0x55deee282d50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x55deee27ada0 tx=0x7f6a80003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a59c51d4dd1d22e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a717fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a717fa640 1 -- 172.21.15.103:0/4143603238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a800044f0 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a717fa640 1 -- 172.21.15.103:0/4143603238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6a80031d50 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a80040070 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a717fa640 1 -- 172.21.15.103:0/4143603238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6a80004b40 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a93025740 10 client.17439 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55deee279fc0 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.728+0000 7f6a717fa640 1 -- 172.21.15.103:0/4143603238 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a80040070 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a717fa640 1 -- 172.21.15.103:0/4143603238 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f6a800309f0 con 0x55deee277050 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a93025740 10 client.17439 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a93025740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a93025740 20 client.17439 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a93025740 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee286dd0 0x55deee2891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a93025740 4 client.17439 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:43:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deee1e2100 con 0x55deee286dd0 2024-08-19T01:43:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee286dd0 0x55deee2891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee286dd0 0x55deee2891c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac0b26216ffcb52 server_cookie=fa5865d619ae066f in_seq=0 out_seq=0 2024-08-19T01:43:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.729+0000 7f6a717fa640 10 client.17439 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a717fa640 1 -- 172.21.15.103:0/4143603238 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55deee1e2100 con 0x55deee286dd0 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a717fa640 10 client.17439 handle_command_reply: tid=0 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 client.17439 shutdown 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee277050 msgr2=0x55deee282d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee277050 0x55deee282d50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x55deee27ada0 tx=0x7f6a80003ed0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 shutdown_connections 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee286dd0 0x55deee2891c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee277050 0x55deee282d50 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee27bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 --2- 172.21.15.103:0/4143603238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee271680 0x55deee27b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a717fa640 0 client.17439 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 >> 172.21.15.103:0/4143603238 conn(0x55deee26ccf0 msgr2=0x55deee26e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.730+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 shutdown_connections 2024-08-19T01:43:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.731+0000 7f6a93025740 1 -- 172.21.15.103:0/4143603238 wait complete. 2024-08-19T01:43:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.731+0000 7f6a93025740 20 client.17439 trim_cache size 0 max 16384 2024-08-19T01:43:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 Processor -- start 2024-08-19T01:43:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 -- start start 2024-08-19T01:43:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee26dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee282d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 0x55deee26e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881a8c20 con 0x55deee271680 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881aa7f0 con 0x55deee271fc0 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a93025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881a92f0 con 0x55deee277050 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee26dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee26dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43994/0 (socket says 172.21.15.103:43994) 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2402816103 learned_addr learned my addr 172.21.15.103:0/2402816103 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee282d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2402816103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 msgr2=0x55deee26e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 0x55deee26e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2402816103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 msgr2=0x55deee282d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee282d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8e72c640 1 -- 172.21.15.103:0/2402816103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deee26cb10 con 0x55deee271680 2024-08-19T01:43:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.753+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee282d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:43:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee26dba0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009620 tx=0x7f6a80009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e4e057e6f83860 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a737fe640 1 -- 172.21.15.103:0/2402816103 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a800044d0 con 0x55deee271680 2024-08-19T01:43:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a737fe640 1 -- 172.21.15.103:0/2402816103 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a80040040 con 0x55deee271680 2024-08-19T01:43:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a737fe640 1 -- 172.21.15.103:0/2402816103 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80045ba0 con 0x55deee271680 2024-08-19T01:43:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a93025740 1 -- 172.21.15.103:0/2402816103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 msgr2=0x55deee26dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a93025740 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee26dba0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009620 tx=0x7f6a80009a20 comp rx=0 tx=0).stop 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a93025740 1 -- 172.21.15.103:0/2402816103 shutdown_connections 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a93025740 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 0x55deee26e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a93025740 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee282d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a93025740 1 --2- 172.21.15.103:0/2402816103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee26dba0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.754+0000 7f6a93025740 1 -- 172.21.15.103:0/2402816103 >> 172.21.15.103:0/2402816103 conn(0x55deee26ccf0 msgr2=0x55deee27b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 -- 172.21.15.103:0/2402816103 shutdown_connections 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 -- 172.21.15.103:0/2402816103 wait complete. 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 -- start start 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 Processor -- start 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee280020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee280560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 0x55deee281570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881aa120 con 0x55deee271680 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881a99c0 con 0x55deee271fc0 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881a6c00 con 0x55deee277050 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a93025740 10 client.0 _ll_register_callbacks cb 0x7f6a7800a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee280020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a8e72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee280020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44004/0 (socket says 172.21.15.103:44004) 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.755+0000 7f6a8e72c640 1 -- 172.21.15.103:0/845017400 learned_addr learned my addr 172.21.15.103:0/845017400 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee280560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 0x55deee281570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8e72c640 1 -- 172.21.15.103:0/845017400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 msgr2=0x55deee281570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 0x55deee281570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8e72c640 1 -- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 msgr2=0x55deee280560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee280560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8e72c640 1 -- 172.21.15.103:0/845017400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deee281b80 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee280560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a8e72c640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee280020 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6a8002ade0 tx=0x7f6a80005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c38a89866e346c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a717fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a717fa640 1 -- 172.21.15.103:0/845017400 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80003970 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a717fa640 1 -- 172.21.15.103:0/845017400 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6a8004e040 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a80040040 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a717fa640 1 -- 172.21.15.103:0/845017400 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6a80003b10 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a93025740 10 client.17445 fetch_fsmap learned FSMap version 38 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.756+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55deee282e00 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a717fa640 1 -- 172.21.15.103:0/845017400 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a80040040 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a717fa640 1 -- 172.21.15.103:0/845017400 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f6a80030830 con 0x55deee271680 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a93025740 10 client.17445 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a93025740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:43:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a93025740 20 client.17445 populate_metadata read hostname 'smithi103' 2024-08-19T01:43:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a93025740 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee2f1e30 0x55deee2f4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:43:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a93025740 4 client.17445 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:43:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55deee1dc450 con 0x55deee2f1e30 2024-08-19T01:43:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.757+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee2f1e30 0x55deee2f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:43:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.758+0000 7f6a8d72a640 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee2f1e30 0x55deee2f4250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b8d35b5d980187 server_cookie=6303094f2ca9212 in_seq=0 out_seq=0 2024-08-19T01:43:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.758+0000 7f6a717fa640 10 client.17445 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a717fa640 1 -- 172.21.15.103:0/845017400 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55deee1dc450 con 0x55deee2f1e30 2024-08-19T01:43:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a717fa640 10 client.17445 handle_command_reply: tid=0 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 client.17445 shutdown 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 msgr2=0x55deee280020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee280020 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6a8002ade0 tx=0x7f6a80005f70 comp rx=0 tx=0).stop 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 shutdown_connections 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55deee2f1e30 0x55deee2f4250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55deee277050 0x55deee281570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55deee271fc0 0x55deee280560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 --2- 172.21.15.103:0/845017400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55deee271680 0x55deee280020 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 >> 172.21.15.103:0/845017400 conn(0x55deee26ccf0 msgr2=0x55deee26b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a717fa640 0 client.17445 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 shutdown_connections 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.764+0000 7f6a93025740 1 -- 172.21.15.103:0/845017400 wait complete. 2024-08-19T01:43:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a93025740 20 client.17445 trim_cache size 0 max 16384 2024-08-19T01:43:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:43:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:43:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:43:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6a740a2cb0 msgr2=0x7f6a740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6a740a2cb0 0x7f6a740a5170 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a840cd4b0 tx=0x7f6a840d3040 comp rx=0 tx=0).stop 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88100770 msgr2=0x7f6a88197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88100770 0x7f6a88197a70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400e9c0 tx=0x7f6a8400ee80 comp rx=0 tx=0).stop 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 shutdown_connections 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6a740a2cb0 0x7f6a740a5170 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a88102320 0x7f6a8819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88101960 0x7f6a88197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 --2- 172.21.15.103:0/3816811713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a88100770 0x7f6a88197a70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 >> 172.21.15.103:0/3816811713 conn(0x7f6a880fbf10 msgr2=0x7f6a880fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 shutdown_connections 2024-08-19T01:43:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:43:48.765+0000 7f6a9018f640 1 -- 172.21.15.103:0/3816811713 wait complete. 2024-08-19T01:43:49.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:49 smithi103 ceph-mon[18436]: pgmap v580: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:43:49.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:49 smithi190 ceph-mon[22032]: pgmap v580: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:43:49.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:49 smithi146 ceph-mon[22098]: pgmap v580: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-08-19T01:43:51.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:51 smithi103 ceph-mon[18436]: pgmap v581: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:43:51.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:51 smithi190 ceph-mon[22032]: pgmap v581: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:43:51.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:51 smithi146 ceph-mon[22098]: pgmap v581: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-08-19T01:43:53.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:53 smithi103 ceph-mon[18436]: pgmap v582: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:43:53.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:53 smithi190 ceph-mon[22032]: pgmap v582: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:43:53.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:53 smithi146 ceph-mon[22098]: pgmap v582: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-08-19T01:43:55.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:55 smithi103 ceph-mon[18436]: pgmap v583: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:43:55.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:55 smithi190 ceph-mon[22032]: pgmap v583: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:43:55.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:55 smithi146 ceph-mon[22098]: pgmap v583: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-08-19T01:43:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:57 smithi103 ceph-mon[18436]: pgmap v584: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:43:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:57 smithi190 ceph-mon[22032]: pgmap v584: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:43:57.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:57 smithi146 ceph-mon[22098]: pgmap v584: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-08-19T01:43:59.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:43:59 smithi103 ceph-mon[18436]: pgmap v585: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:43:59.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:43:59 smithi190 ceph-mon[22032]: pgmap v585: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:43:59.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:43:59 smithi146 ceph-mon[22098]: pgmap v585: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-08-19T01:44:00.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:44:00.516 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:01.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:01 smithi103 ceph-mon[18436]: pgmap v586: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:44:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:01 smithi190 ceph-mon[22032]: pgmap v586: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:44:01.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:01 smithi146 ceph-mon[22098]: pgmap v586: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-08-19T01:44:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 -- 172.21.15.103:0/3980083396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8102320 msgr2=0x7f8da8108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 --2- 172.21.15.103:0/3980083396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8102320 0x7f8da8108bb0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009ca0 tx=0x7f8da002a8c0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 -- 172.21.15.103:0/3980083396 shutdown_connections 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 --2- 172.21.15.103:0/3980083396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8102320 0x7f8da8108bb0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 --2- 172.21.15.103:0/3980083396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da8101960 0x7f8da8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 --2- 172.21.15.103:0/3980083396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da8100770 0x7f8da8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 -- 172.21.15.103:0/3980083396 >> 172.21.15.103:0/3980083396 conn(0x7f8da80fbf10 msgr2=0x7f8da80fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 -- 172.21.15.103:0/3980083396 shutdown_connections 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.858+0000 7f8dadf78640 1 -- 172.21.15.103:0/3980083396 wait complete. 2024-08-19T01:44:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 Processor -- start 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 -- start start 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8100770 0x7f8da8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da8101960 0x7f8da8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da8102320 0x7f8da819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8da810c2a0 con 0x7f8da8100770 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8da810c120 con 0x7f8da8101960 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.859+0000 7f8dadf78640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8da810c420 con 0x7f8da8102320 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8100770 0x7f8da8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8100770 0x7f8da8197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55900/0 (socket says 172.21.15.103:55900) 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 -- 172.21.15.103:0/1043727475 learned_addr learned my addr 172.21.15.103:0/1043727475 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da7fff640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da8102320 0x7f8da819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da8101960 0x7f8da8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 -- 172.21.15.103:0/1043727475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da8102320 msgr2=0x7f8da819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da8102320 0x7f8da819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 -- 172.21.15.103:0/1043727475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da8101960 msgr2=0x7f8da8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da8101960 0x7f8da8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 -- 172.21.15.103:0/1043727475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da819f630 con 0x7f8da8100770 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da7fff640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da8102320 0x7f8da819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da8101960 0x7f8da8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da77fe640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8100770 0x7f8da8197a10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800d950 tx=0x7f8d9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da4ff9640 1 -- 172.21.15.103:0/1043727475 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98017070 con 0x7f8da8100770 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da819f920 con 0x7f8da8100770 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da4ff9640 1 -- 172.21.15.103:0/1043727475 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d98009c30 con 0x7f8da8100770 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8da4ff9640 1 -- 172.21.15.103:0/1043727475 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9800f620 con 0x7f8da8100770 2024-08-19T01:44:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.860+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da819fe20 con 0x7f8da8100770 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.863+0000 7f8da4ff9640 1 -- 172.21.15.103:0/1043727475 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8d9801a070 con 0x7f8da8100770 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.863+0000 7f8db0e0e740 1 Processor -- start 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.863+0000 7f8db0e0e740 1 -- start start 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd4524a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd4529a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da4ff9640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d900a2c30 0x7f8d900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd452c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abd44879d0 con 0x55abd452a050 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55abd442f820 con 0x55abd4524fc0 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55abd4483990 con 0x55abd4524680 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd452c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d900a2c30 0x7f8d900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd452c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55902/0 (socket says 172.21.15.103:55902) 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 -- 172.21.15.103:0/596107823 learned_addr learned my addr 172.21.15.103:0/596107823 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd4524a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da4ff9640 1 -- 172.21.15.103:0/1043727475 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8d980ceb20 con 0x7f8da8100770 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da77fe640 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd4529a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 -- 172.21.15.103:0/596107823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 msgr2=0x55abd4524a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd4524a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 -- 172.21.15.103:0/596107823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 msgr2=0x55abd4529a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd4529a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 -- 172.21.15.103:0/596107823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd419b210 con 0x55abd452a050 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d900a2c30 0x7f8d900a50f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8da8198f20 tx=0x7f8d9c005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.864+0000 7f8da7fff640 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd452c510 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8da0005e30 tx=0x7f8da0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a430d2be748757 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.865+0000 7f8d8dffb640 1 -- 172.21.15.103:0/596107823 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0004b40 con 0x55abd452a050 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.865+0000 7f8d8dffb640 1 -- 172.21.15.103:0/596107823 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8da0036040 con 0x55abd452a050 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.865+0000 7f8d8dffb640 1 -- 172.21.15.103:0/596107823 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0003e30 con 0x55abd452a050 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.865+0000 7f8db0e0e740 1 -- 172.21.15.103:0/596107823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 msgr2=0x55abd452c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.865+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd452c510 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8da0005e30 tx=0x7f8da0005c50 comp rx=0 tx=0).stop 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 -- 172.21.15.103:0/596107823 shutdown_connections 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd452c510 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd4529a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/596107823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd4524a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 -- 172.21.15.103:0/596107823 >> 172.21.15.103:0/596107823 conn(0x55abd451fcf0 msgr2=0x55abd45200f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 -- 172.21.15.103:0/596107823 shutdown_connections 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 -- 172.21.15.103:0/596107823 wait complete. 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 -- start start 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 Processor -- start 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd452e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd452eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.866+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd4535ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abd452ce40 con 0x55abd452a050 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55abd4443520 con 0x55abd4524fc0 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55abd452cfc0 con 0x55abd4524680 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8db0e0e740 10 client.0 _ll_register_callbacks cb 0x55abd419aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd4535ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd4535ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55912/0 (socket says 172.21.15.103:55912) 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 -- 172.21.15.103:0/1895945083 learned_addr learned my addr 172.21.15.103:0/1895945083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd452eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da77fe640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd452e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 -- 172.21.15.103:0/1895945083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 msgr2=0x55abd452e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd452e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 -- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 msgr2=0x55abd452eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd452eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 -- 172.21.15.103:0/1895945083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd4267620 con 0x55abd452a050 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da77fe640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd452e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8da7fff640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd4535ba0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009ca0 tx=0x7f8da0003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984863390fb54139 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8d86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8d86ffd640 1 -- 172.21.15.103:0/1895945083 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da004f020 con 0x55abd452a050 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8d86ffd640 1 -- 172.21.15.103:0/1895945083 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8da0003fd0 con 0x55abd452a050 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.867+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da0036040 con 0x55abd452a050 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8d86ffd640 1 -- 172.21.15.103:0/1895945083 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8da00041d0 con 0x55abd452a050 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 10 client.17457 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55abd452ce40 con 0x55abd452a050 2024-08-19T01:44:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8d86ffd640 1 -- 172.21.15.103:0/1895945083 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8da0036040 con 0x55abd452a050 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8d86ffd640 1 -- 172.21.15.103:0/1895945083 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f8da0030bd0 con 0x55abd452a050 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 10 client.17457 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 20 client.17457 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd4539cb0 0x55abd453c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 4 client.17457 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd4495100 con 0x55abd4539cb0 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.868+0000 7f8da77fe640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd4539cb0 0x55abd453c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.869+0000 7f8d86ffd640 10 client.17457 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.869+0000 7f8da77fe640 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd4539cb0 0x55abd453c0a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca08e40cf347f27c server_cookie=1d1d0b8b474e1857 in_seq=0 out_seq=0 2024-08-19T01:44:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.869+0000 7f8d86ffd640 1 -- 172.21.15.103:0/1895945083 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abd4495100 con 0x55abd4539cb0 2024-08-19T01:44:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.869+0000 7f8d86ffd640 10 client.17457 handle_command_reply: tid=0 2024-08-19T01:44:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 client.17457 shutdown 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 msgr2=0x55abd4535ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd4535ba0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009ca0 tx=0x7f8da0003de0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 shutdown_connections 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd4539cb0 0x55abd453c0a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd452a050 0x55abd4535ba0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd4524fc0 0x55abd452eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8d86ffd640 0 client.17457 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/1895945083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd452e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 >> 172.21.15.103:0/1895945083 conn(0x55abd451fcf0 msgr2=0x55abd4521500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 shutdown_connections 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 1 -- 172.21.15.103:0/1895945083 wait complete. 2024-08-19T01:44:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.870+0000 7f8db0e0e740 20 client.17457 trim_cache size 0 max 16384 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 Processor -- start 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 -- start start 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd451f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524fc0 0x55abd4535b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd45212b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81a8ae0 con 0x55abd4524fc0 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81aa6b0 con 0x55abd452a050 2024-08-19T01:44:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81a91b0 con 0x55abd4524680 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524fc0 0x55abd4535b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524fc0 0x55abd4535b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55914/0 (socket says 172.21.15.103:55914) 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 -- 172.21.15.103:0/3850638705 learned_addr learned my addr 172.21.15.103:0/3850638705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da77fe640 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd451f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da7fff640 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd45212b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 -- 172.21.15.103:0/3850638705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 msgr2=0x55abd451f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd451f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 -- 172.21.15.103:0/3850638705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 msgr2=0x55abd45212b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd45212b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da6ffd640 1 -- 172.21.15.103:0/3850638705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd4520980 con 0x55abd4524fc0 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da77fe640 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd451f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.888+0000 7f8da7fff640 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd45212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.889+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524fc0 0x55abd4535b60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00ee80 tx=0x7f8d9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4859369c936c30 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.889+0000 7f8d8d7fa640 1 -- 172.21.15.103:0/3850638705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c023070 con 0x55abd4524fc0 2024-08-19T01:44:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.889+0000 7f8d8d7fa640 1 -- 172.21.15.103:0/3850638705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d9c017040 con 0x55abd4524fc0 2024-08-19T01:44:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.889+0000 7f8d8d7fa640 1 -- 172.21.15.103:0/3850638705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c01f420 con 0x55abd4524fc0 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3850638705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524fc0 msgr2=0x55abd4535b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524fc0 0x55abd4535b60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00ee80 tx=0x7f8d9c011000 comp rx=0 tx=0).stop 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3850638705 shutdown_connections 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd45212b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524fc0 0x55abd4535b60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3850638705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524680 0x55abd451f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3850638705 >> 172.21.15.103:0/3850638705 conn(0x55abd451fcf0 msgr2=0x55abd4520ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3850638705 shutdown_connections 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3850638705 wait complete. 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 -- start start 2024-08-19T01:44:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.890+0000 7f8db0e0e740 1 Processor -- start 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524680 0x55abd45338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524fc0 0x55abd4533e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd4534e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81a9fe0 con 0x55abd4524680 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81a9880 con 0x55abd452a050 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81a6ac0 con 0x55abd4524fc0 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 10 client.0 _ll_register_callbacks cb 0x7f8da0031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524680 0x55abd45338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524680 0x55abd45338c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55918/0 (socket says 172.21.15.103:55918) 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524fc0 0x55abd4533e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 -- 172.21.15.103:0/3459482429 learned_addr learned my addr 172.21.15.103:0/3459482429 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da7fff640 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd4534e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 -- 172.21.15.103:0/3459482429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524fc0 msgr2=0x55abd4533e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524fc0 0x55abd4533e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 -- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 msgr2=0x55abd4534e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd4534e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 -- 172.21.15.103:0/3459482429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd4535420 con 0x55abd4524680 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8da77fe640 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524680 0x55abd45338c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8d9809b340 tx=0x7f8d980ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d245ae887220a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8d86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8d86ffd640 1 -- 172.21.15.103:0/3459482429 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980af070 con 0x55abd4524680 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8d86ffd640 1 -- 172.21.15.103:0/3459482429 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8d98097a20 con 0x55abd4524680 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.891+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d9c017040 con 0x55abd4524680 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8d86ffd640 1 -- 172.21.15.103:0/3459482429 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8d98021040 con 0x55abd4524680 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 10 client.17463 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55abd45a5040 con 0x55abd4524680 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8d86ffd640 1 -- 172.21.15.103:0/3459482429 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d9c017040 con 0x55abd4524680 2024-08-19T01:44:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8d86ffd640 1 -- 172.21.15.103:0/3459482429 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f8d980b4110 con 0x55abd4524680 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 10 client.17463 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 10 client.17463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 20 client.17463 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd45a5970 0x55abd45a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 4 client.17463 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abd448f450 con 0x55abd45a5970 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.892+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd45a5970 0x55abd45a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.893+0000 7f8d86ffd640 10 client.17463 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.893+0000 7f8da6ffd640 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd45a5970 0x55abd45a7d90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd1a1c47863c65f server_cookie=c87469258377f372 in_seq=0 out_seq=0 2024-08-19T01:44:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8d86ffd640 1 -- 172.21.15.103:0/3459482429 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abd448f450 con 0x55abd45a5970 2024-08-19T01:44:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8d86ffd640 10 client.17463 handle_command_reply: tid=0 2024-08-19T01:44:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 client.17463 shutdown 2024-08-19T01:44:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524680 msgr2=0x55abd45338c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524680 0x55abd45338c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8d9809b340 tx=0x7f8d980ce040 comp rx=0 tx=0).stop 2024-08-19T01:44:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 shutdown_connections 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55abd45a5970 0x55abd45a7d90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55abd452a050 0x55abd4534e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8d86ffd640 0 client.17463 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abd4524fc0 0x55abd4533e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 --2- 172.21.15.103:0/3459482429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abd4524680 0x55abd45338c0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 >> 172.21.15.103:0/3459482429 conn(0x55abd451fcf0 msgr2=0x55abd4521370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 shutdown_connections 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 1 -- 172.21.15.103:0/3459482429 wait complete. 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.900+0000 7f8db0e0e740 20 client.17463 trim_cache size 0 max 16384 2024-08-19T01:44:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:44:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:44:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:44:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:44:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d900a2c30 msgr2=0x7f8d900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d900a2c30 0x7f8d900a50f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8da8198f20 tx=0x7f8d9c005e50 comp rx=0 tx=0).stop 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8100770 msgr2=0x7f8da8197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8100770 0x7f8da8197a10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800d950 tx=0x7f8d9800de10 comp rx=0 tx=0).stop 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 shutdown_connections 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8d900a2c30 0x7f8d900a50f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8da8102320 0x7f8da819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da8101960 0x7f8da8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 --2- 172.21.15.103:0/1043727475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8da8100770 0x7f8da8197a10 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 >> 172.21.15.103:0/1043727475 conn(0x7f8da80fbf10 msgr2=0x7f8da80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 shutdown_connections 2024-08-19T01:44:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:01.901+0000 7f8dadf78640 1 -- 172.21.15.103:0/1043727475 wait complete. 2024-08-19T01:44:03.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:03 smithi103 ceph-mon[18436]: pgmap v587: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:44:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:03 smithi190 ceph-mon[22032]: pgmap v587: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:44:03.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:03 smithi146 ceph-mon[22098]: pgmap v587: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:44:04.774 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:44:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:44:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:44:04.775 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:44:04.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 -- 172.21.15.103:0/3068048586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 msgr2=0x7f0bb4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3068048586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb4100f30 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009900 tx=0x7f0ba402a480 comp rx=0 tx=0).stop 2024-08-19T01:44:04.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 -- 172.21.15.103:0/3068048586 shutdown_connections 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3068048586 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb4102c20 0x7f0bb410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3068048586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb41022a0 0x7f0bb41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3068048586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb4100f30 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 -- 172.21.15.103:0/3068048586 >> 172.21.15.103:0/3068048586 conn(0x7f0bb40fc500 msgr2=0x7f0bb40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 -- 172.21.15.103:0/3068048586 shutdown_connections 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.107+0000 7f0bb8968640 1 -- 172.21.15.103:0/3068048586 wait complete. 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 Processor -- start 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 -- start start 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb40fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb41022a0 0x7f0bb40fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb4102c20 0x7f0bb40fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4110c00 con 0x7f0bb4100b50 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4110a80 con 0x7f0bb41022a0 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb8968640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4110d80 con 0x7f0bb4102c20 2024-08-19T01:44:04.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb40fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb40fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55936/0 (socket says 172.21.15.103:55936) 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 -- 172.21.15.103:0/3915583104 learned_addr learned my addr 172.21.15.103:0/3915583104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb41022a0 0x7f0bb40fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 -- 172.21.15.103:0/3915583104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb4102c20 msgr2=0x7f0bb40fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb4102c20 0x7f0bb40fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 -- 172.21.15.103:0/3915583104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb41022a0 msgr2=0x7f0bb40fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb41022a0 0x7f0bb40fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb259c640 1 -- 172.21.15.103:0/3915583104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb41a66f0 con 0x7f0bb4100b50 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.108+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb41022a0 0x7f0bb40fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.109+0000 7f0bb259c640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb40fc9b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4006010 tx=0x7f0ba40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.109+0000 7f0ba37fe640 1 -- 172.21.15.103:0/3915583104 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba403f070 con 0x7f0bb4100b50 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.109+0000 7f0ba37fe640 1 -- 172.21.15.103:0/3915583104 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ba4003aa0 con 0x7f0bb4100b50 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.109+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb41a6980 con 0x7f0bb4100b50 2024-08-19T01:44:04.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.109+0000 7f0ba37fe640 1 -- 172.21.15.103:0/3915583104 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4035030 con 0x7f0bb4100b50 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.109+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb41a6eb0 con 0x7f0bb4100b50 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.112+0000 7f0ba37fe640 1 -- 172.21.15.103:0/3915583104 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0ba4042070 con 0x7f0bb4100b50 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.112+0000 7f0bbad66740 1 Processor -- start 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.112+0000 7f0bbad66740 1 -- start start 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.112+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c7b210 0x5585c7c7b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.112+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7bb30 0x5585c7c85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.112+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c863f0 0x5585c7c887e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.112+0000 7f0bbad66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5585c7be0980 con 0x5585c7c863f0 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bbad66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5585c7b78a70 con 0x5585c7c7b210 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bbad66740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5585c7bdc940 con 0x5585c7c7bb30 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c863f0 0x5585c7c887e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c863f0 0x5585c7c887e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55938/0 (socket says 172.21.15.103:55938) 2024-08-19T01:44:04.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/4277625817 learned_addr learned my addr 172.21.15.103:0/4277625817 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb259c640 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7bb30 0x5585c7c85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c7b210 0x5585c7c7b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0ba37fe640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0b9c0a2c00 0x7f0b9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/4277625817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7bb30 msgr2=0x5585c7c85eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7bb30 0x5585c7c85eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/4277625817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c7b210 msgr2=0x5585c7c7b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c7b210 0x5585c7c7b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/4277625817 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c7bdb6e0 con 0x5585c7c863f0 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0b9c0a2c00 0x7f0b9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0ba37fe640 1 -- 172.21.15.103:0/3915583104 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0ba40efa30 con 0x7f0bb4100b50 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c863f0 0x5585c7c887e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800a9d0 tx=0x7f0ba800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97b7697ab638fec server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.113+0000 7f0ba0ff9640 1 -- 172.21.15.103:0/4277625817 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8015070 con 0x5585c7c863f0 2024-08-19T01:44:04.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.114+0000 7f0ba0ff9640 1 -- 172.21.15.103:0/4277625817 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ba800cc60 con 0x5585c7c863f0 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.114+0000 7f0ba0ff9640 1 -- 172.21.15.103:0/4277625817 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80105b0 con 0x5585c7c863f0 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0b9c0a2c00 0x7f0b9c0a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009e80 tx=0x7f0bac004b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 -- 172.21.15.103:0/4277625817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c863f0 msgr2=0x5585c7c887e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c863f0 0x5585c7c887e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800a9d0 tx=0x7f0ba800ae90 comp rx=0 tx=0).stop 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 -- 172.21.15.103:0/4277625817 shutdown_connections 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c863f0 0x5585c7c887e0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7bb30 0x5585c7c85eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 --2- 172.21.15.103:0/4277625817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c7b210 0x5585c7c7b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 -- 172.21.15.103:0/4277625817 >> 172.21.15.103:0/4277625817 conn(0x5585c7c777a0 msgr2=0x5585c7c77b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 -- 172.21.15.103:0/4277625817 shutdown_connections 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 -- 172.21.15.103:0/4277625817 wait complete. 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 -- start start 2024-08-19T01:44:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.115+0000 7f0bbad66740 1 Processor -- start 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c83230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c7bb30 0x5585c7c83770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c83cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bbad66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5585c7c88fc0 con 0x5585c7c7bb30 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bbad66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5585c7b9baa0 con 0x5585c7c863f0 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bbad66740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5585c7c89140 con 0x5585c7c7b210 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bbad66740 10 client.0 _ll_register_callbacks cb 0x5585c7b8ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c7bb30 0x5585c7c83770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c83230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c7bb30 0x5585c7c83770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55952/0 (socket says 172.21.15.103:55952) 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c83230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43572/0 (socket says 172.21.15.103:43572) 2024-08-19T01:44:04.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/1276447293 learned_addr learned my addr 172.21.15.103:0/1276447293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c83cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/1276447293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 msgr2=0x5585c7c83230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c83230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 msgr2=0x5585c7c83cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c83cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.116+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/1276447293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c7b7f650 con 0x5585c7c7bb30 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c7bb30 0x5585c7c83770 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8008010 tx=0x7f0ba8011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369cf19398fd0a74 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0b99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0b99ffb640 1 -- 172.21.15.103:0/1276447293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8020070 con 0x5585c7c7bb30 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba800cc60 con 0x5585c7c7bb30 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0b99ffb640 1 -- 172.21.15.103:0/1276447293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0ba801a660 con 0x5585c7c7bb30 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0b99ffb640 1 -- 172.21.15.103:0/1276447293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0ba8015040 con 0x5585c7c7bb30 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 10 client.17475 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:04.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5585c79523c0 con 0x5585c7c7bb30 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0b99ffb640 1 -- 172.21.15.103:0/1276447293 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ba800cc60 con 0x5585c7c7bb30 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0b99ffb640 1 -- 172.21.15.103:0/1276447293 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f0ba802f900 con 0x5585c7c7bb30 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 10 client.17475 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 20 client.17475 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7c95e80 0x5585c7c98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 4 client.17475 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.117+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585c7bedef0 con 0x5585c7c95e80 2024-08-19T01:44:04.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.118+0000 7f0bb259c640 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7c95e80 0x5585c7c98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.118+0000 7f0b99ffb640 10 client.17475 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.118+0000 7f0bb259c640 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7c95e80 0x5585c7c98270 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36121969e43d5167 server_cookie=162e2705874f707e in_seq=0 out_seq=0 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0b99ffb640 1 -- 172.21.15.103:0/1276447293 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585c7bedef0 con 0x5585c7c95e80 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0b99ffb640 10 client.17475 handle_command_reply: tid=0 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 client.17475 shutdown 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c7bb30 msgr2=0x5585c7c83770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c7bb30 0x5585c7c83770 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8008010 tx=0x7f0ba8011bd0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 shutdown_connections 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7c95e80 0x5585c7c98270 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c83cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c7bb30 0x5585c7c83770 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 --2- 172.21.15.103:0/1276447293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c83230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 >> 172.21.15.103:0/1276447293 conn(0x5585c7c777a0 msgr2=0x5585c7c7d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:04.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0b99ffb640 0 client.17475 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 shutdown_connections 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 1 -- 172.21.15.103:0/1276447293 wait complete. 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.119+0000 7f0bbad66740 20 client.17475 trim_cache size 0 max 16384 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 Processor -- start 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 -- start start 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c9d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c9de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c81e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41af2a0 con 0x5585c7c819d0 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41b0e70 con 0x5585c7c863f0 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.139+0000 7f0bbad66740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41af970 con 0x5585c7c7b210 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c81e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c81e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55958/0 (socket says 172.21.15.103:55958) 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 -- 172.21.15.103:0/3871925322 learned_addr learned my addr 172.21.15.103:0/3871925322 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:04.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c9de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb259c640 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c9d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 -- 172.21.15.103:0/3871925322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 msgr2=0x5585c7c9d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c9d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 -- 172.21.15.103:0/3871925322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 msgr2=0x5585c7c9de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c9de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 -- 172.21.15.103:0/3871925322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c7c775d0 con 0x5585c7c819d0 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c9de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c81e30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0bac017550 tx=0x7f0bac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2c761ccf4cb39d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.140+0000 7f0b9bfff640 1 -- 172.21.15.103:0/3871925322 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac015400 con 0x5585c7c819d0 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0b9bfff640 1 -- 172.21.15.103:0/3871925322 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bac026070 con 0x5585c7c819d0 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0b9bfff640 1 -- 172.21.15.103:0/3871925322 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac021860 con 0x5585c7c819d0 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 -- 172.21.15.103:0/3871925322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 msgr2=0x5585c7c81e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c81e30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0bac017550 tx=0x7f0bac013000 comp rx=0 tx=0).stop 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 -- 172.21.15.103:0/3871925322 shutdown_connections 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c81e30 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c9de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3871925322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c9d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 -- 172.21.15.103:0/3871925322 >> 172.21.15.103:0/3871925322 conn(0x5585c7c777a0 msgr2=0x5585c7c7d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 -- 172.21.15.103:0/3871925322 shutdown_connections 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.141+0000 7f0bbad66740 1 -- 172.21.15.103:0/3871925322 wait complete. 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 -- start start 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 Processor -- start 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c8e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c8e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41b07a0 con 0x5585c7c819d0 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41b0040 con 0x5585c7c863f0 2024-08-19T01:44:04.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41ad2a0 con 0x5585c7c7b210 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bbad66740 10 client.0 _ll_register_callbacks cb 0x7f0ba800a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c8e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c8e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55964/0 (socket says 172.21.15.103:55964) 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/3023064587 learned_addr learned my addr 172.21.15.103:0/3023064587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb259c640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c8e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/3023064587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 msgr2=0x5585c7c8e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c8e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 msgr2=0x5585c7c893b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c893b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.142+0000 7f0bb1d9b640 1 -- 172.21.15.103:0/3023064587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c7c7ee30 con 0x5585c7c819d0 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0bb2d9d640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:44:04.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0bb1d9b640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c8e940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0ba802fda0 tx=0x7f0ba80117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d454b6a4bc926a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0b99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0b99ffb640 1 -- 172.21.15.103:0/3023064587 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8020040 con 0x5585c7c819d0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0b99ffb640 1 -- 172.21.15.103:0/3023064587 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0ba800cc60 con 0x5585c7c819d0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bac026070 con 0x5585c7c819d0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0b99ffb640 1 -- 172.21.15.103:0/3023064587 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0ba803e050 con 0x5585c7c819d0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0bbad66740 10 client.17481 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5585c7c88ea0 con 0x5585c7c819d0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.143+0000 7f0b99ffb640 1 -- 172.21.15.103:0/3023064587 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bac026070 con 0x5585c7c819d0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0b99ffb640 1 -- 172.21.15.103:0/3023064587 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f0ba8024850 con 0x5585c7c819d0 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bbad66740 10 client.17481 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bbad66740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bbad66740 20 client.17481 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7d014e0 0x5585c7d03900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:04.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bbad66740 4 client.17481 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585c7be8400 con 0x5585c7d014e0 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bb259c640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7d014e0 0x5585c7d03900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0bb259c640 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7d014e0 0x5585c7d03900 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c518d484ae45fc server_cookie=45638d478013baf1 in_seq=0 out_seq=0 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0b99ffb640 10 client.17481 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0b99ffb640 1 -- 172.21.15.103:0/3023064587 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5585c7be8400 con 0x5585c7d014e0 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.144+0000 7f0b99ffb640 10 client.17481 handle_command_reply: tid=0 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 client.17481 shutdown 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 msgr2=0x5585c7c8e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c8e940 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0ba802fda0 tx=0x7f0ba80117d0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 shutdown_connections 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5585c7d014e0 0x5585c7d03900 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5585c7c863f0 0x5585c7c893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5585c7c819d0 0x5585c7c8e940 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 --2- 172.21.15.103:0/3023064587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5585c7c7b210 0x5585c7c8e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 >> 172.21.15.103:0/3023064587 conn(0x5585c7c777a0 msgr2=0x5585c7c7d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0b99ffb640 0 client.17481 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 shutdown_connections 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 1 -- 172.21.15.103:0/3023064587 wait complete. 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bbad66740 20 client.17481 trim_cache size 0 max 16384 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0b9c0a2c00 msgr2=0x7f0b9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0b9c0a2c00 0x7f0b9c0a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009e80 tx=0x7f0bac004b50 comp rx=0 tx=0).stop 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 msgr2=0x7f0bb40fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.145+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb40fc9b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4006010 tx=0x7f0ba40042f0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 shutdown_connections 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f0b9c0a2c00 0x7f0b9c0a50c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bb4102c20 0x7f0bb40fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0bb41022a0 0x7f0bb40fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 --2- 172.21.15.103:0/3915583104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bb4100b50 0x7f0bb40fc9b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 >> 172.21.15.103:0/3915583104 conn(0x7f0bb40fc500 msgr2=0x7f0bb4069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 shutdown_connections 2024-08-19T01:44:04.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:04.146+0000 7f0bb8968640 1 -- 172.21.15.103:0/3915583104 wait complete. 2024-08-19T01:44:05.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:05 smithi103 ceph-mon[18436]: pgmap v588: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:44:05.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:05 smithi190 ceph-mon[22032]: pgmap v588: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:44:05.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:05 smithi146 ceph-mon[22098]: pgmap v588: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-08-19T01:44:06.277 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:44:06.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:44:06.573 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:07.149 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:07 smithi103 ceph-mon[18436]: pgmap v589: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:44:07.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 -- 172.21.15.103:0/4220794459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1078b0 msgr2=0x7fecfc107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 --2- 172.21.15.103:0/4220794459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1078b0 0x7fecfc107c90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009930 tx=0x7fecf802a420 comp rx=0 tx=0).stop 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 -- 172.21.15.103:0/4220794459 shutdown_connections 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 --2- 172.21.15.103:0/4220794459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecfc102250 0x7fecfc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 --2- 172.21.15.103:0/4220794459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecfc1018b0 0x7fecfc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 --2- 172.21.15.103:0/4220794459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1078b0 0x7fecfc107c90 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 -- 172.21.15.103:0/4220794459 >> 172.21.15.103:0/4220794459 conn(0x7fecfc0fd5b0 msgr2=0x7fecfc0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 -- 172.21.15.103:0/4220794459 shutdown_connections 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 -- 172.21.15.103:0/4220794459 wait complete. 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 Processor -- start 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.138+0000 7fed0405c640 1 -- start start 2024-08-19T01:44:07.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed0405c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1018b0 0x7fecfc074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed0405c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecfc102250 0x7fecfc074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed0405c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecfc1078b0 0x7fecfc074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed0405c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc111b20 con 0x7fecfc1018b0 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed0405c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc1119a0 con 0x7fecfc102250 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed0405c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fecfc111ca0 con 0x7fecfc1078b0 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1018b0 0x7fecfc074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1018b0 0x7fecfc074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55992/0 (socket says 172.21.15.103:55992) 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 -- 172.21.15.103:0/2194772973 learned_addr learned my addr 172.21.15.103:0/2194772973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed025f9640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecfc1078b0 0x7fecfc074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed015f7640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecfc102250 0x7fecfc074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 -- 172.21.15.103:0/2194772973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecfc1078b0 msgr2=0x7fecfc074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecfc1078b0 0x7fecfc074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 -- 172.21.15.103:0/2194772973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecfc102250 msgr2=0x7fecfc074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecfc102250 0x7fecfc074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 -- 172.21.15.103:0/2194772973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecfc1a65e0 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed025f9640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecfc1078b0 0x7fecfc074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fed01df8640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1018b0 0x7fecfc074220 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005940 tx=0x7fecf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.139+0000 7fecf2ffd640 1 -- 172.21.15.103:0/2194772973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf803f030 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.140+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecfc1a6870 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.140+0000 7fecf2ffd640 1 -- 172.21.15.103:0/2194772973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecf80318a0 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.140+0000 7fecf2ffd640 1 -- 172.21.15.103:0/2194772973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8035070 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.140+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecfc1a6da0 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.141+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecfc101d10 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.142+0000 7fecf2ffd640 1 -- 172.21.15.103:0/2194772973 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fecf8042030 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.143+0000 7fecf2ffd640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fece80a2b80 0x7fece80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.143+0000 7fed015f7640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fece80a2b80 0x7fece80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.143+0000 7fecf2ffd640 1 -- 172.21.15.103:0/2194772973 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fecf80ef860 con 0x7fecfc1018b0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.143+0000 7fed015f7640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fece80a2b80 0x7fece80a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fecfc075cb0 tx=0x7fecec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:07.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.148+0000 7fecf2ffd640 1 -- 172.21.15.103:0/2194772973 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fecf80bc7f0 con 0x7fecfc1018b0 2024-08-19T01:44:07.282 INFO:tasks.workunit.client.0.smithi103.stdout:fs setup took 443 secs 2024-08-19T01:44:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:07 smithi190 ceph-mon[22032]: pgmap v589: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:44:07.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.381+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecfc075750 con 0x7fecfc1018b0 2024-08-19T01:44:07.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.381+0000 7fecf2ffd640 1 -- 172.21.15.103:0/2194772973 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11994 (secure 0 0 0) 0x7fecf80bc0f0 con 0x7fecfc1018b0 2024-08-19T01:44:07.382 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:44:07.383 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":38,"btime":"2024-08-19T01:43:11:862098+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:43:11.573718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:44:07.383 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 38 2024-08-19T01:44:07.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.383+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fece80a2b80 msgr2=0x7fece80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.383+0000 7fed0405c640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fece80a2b80 0x7fece80a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fecfc075cb0 tx=0x7fecec005f90 comp rx=0 tx=0).stop 2024-08-19T01:44:07.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.383+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1018b0 msgr2=0x7fecfc074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.383+0000 7fed0405c640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1018b0 0x7fecfc074220 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005940 tx=0x7fecf80042f0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.383+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 shutdown_connections 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.383+0000 7fed0405c640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fece80a2b80 0x7fece80a5040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.383+0000 7fed0405c640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fecfc1078b0 0x7fecfc074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.384+0000 7fed0405c640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecfc102250 0x7fecfc074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.384+0000 7fed0405c640 1 --2- 172.21.15.103:0/2194772973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecfc1018b0 0x7fecfc074220 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.384+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 >> 172.21.15.103:0/2194772973 conn(0x7fecfc0fd5b0 msgr2=0x7fecfc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.384+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 shutdown_connections 2024-08-19T01:44:07.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:07.384+0000 7fed0405c640 1 -- 172.21.15.103:0/2194772973 wait complete. 2024-08-19T01:44:07.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:07 smithi146 ceph-mon[22098]: pgmap v589: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-08-19T01:44:08.071 INFO:tasks.workunit.client.0.smithi103.stdout:Syncing()...0 sec 2024-08-19T01:44:08.073 INFO:tasks.workunit.client.0.smithi103.stdout:Starting Actual Benchmark At: Mon Aug 19 01:44:08 2024 2024-08-19T01:44:08.073 INFO:tasks.workunit.client.0.smithi103.stdout: 2024-08-19T01:44:08.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:08 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2194772973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:44:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:08 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2194772973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:44:08.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:08 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2194772973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:44:08.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:44:09.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:09 smithi103 ceph-mon[18436]: pgmap v590: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 23 op/s 2024-08-19T01:44:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:09 smithi190 ceph-mon[22032]: pgmap v590: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 23 op/s 2024-08-19T01:44:09.399 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:09.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:09 smithi146 ceph-mon[22098]: pgmap v590: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 23 op/s 2024-08-19T01:44:10.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.988+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/866145936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78103ac0 msgr2=0x7fbd78103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.988+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/866145936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78103ac0 0x7fbd78103f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009890 tx=0x7fbd6c02a380 comp rx=0 tx=0).stop 2024-08-19T01:44:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.989+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/866145936 shutdown_connections 2024-08-19T01:44:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.989+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/866145936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78104460 0x7fbd78110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.989+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/866145936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78103ac0 0x7fbd78103f20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.989+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/866145936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd78109ac0 0x7fbd78109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.989+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/866145936 >> 172.21.15.103:0/866145936 conn(0x7fbd780ff7a0 msgr2=0x7fbd78101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:10.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.990+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/866145936 shutdown_connections 2024-08-19T01:44:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.990+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/866145936 wait complete. 2024-08-19T01:44:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.990+0000 7fbd7f4c5640 1 Processor -- start 2024-08-19T01:44:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7f4c5640 1 -- start start 2024-08-19T01:44:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7f4c5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78103ac0 0x7fbd7819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7f4c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78104460 0x7fbd7819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7f4c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd78109ac0 0x7fbd7819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7f4c5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78113dc0 con 0x7fbd78109ac0 2024-08-19T01:44:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7f4c5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78113c40 con 0x7fbd78103ac0 2024-08-19T01:44:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7f4c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78113f40 con 0x7fbd78104460 2024-08-19T01:44:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78103ac0 0x7fbd7819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78103ac0 0x7fbd7819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:43826/0 (socket says 172.21.15.103:43826) 2024-08-19T01:44:10.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7da62640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd78109ac0 0x7fbd7819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7d261640 1 -- 172.21.15.103:0/1133845568 learned_addr learned my addr 172.21.15.103:0/1133845568 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.991+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78104460 0x7fbd7819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7d261640 1 -- 172.21.15.103:0/1133845568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78104460 msgr2=0x7fbd7819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7d261640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78104460 0x7fbd7819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7d261640 1 -- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd78109ac0 msgr2=0x7fbd7819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7d261640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd78109ac0 0x7fbd7819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7d261640 1 -- 172.21.15.103:0/1133845568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7819f2f0 con 0x7fbd78103ac0 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7d261640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78103ac0 0x7fbd7819a340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800ebc0 tx=0x7fbd6800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd767fc640 1 -- 172.21.15.103:0/1133845568 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68010430 con 0x7fbd78103ac0 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd767fc640 1 -- 172.21.15.103:0/1133845568 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd68010a80 con 0x7fbd78103ac0 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7819f5e0 con 0x7fbd78103ac0 2024-08-19T01:44:10.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd767fc640 1 -- 172.21.15.103:0/1133845568 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68018a50 con 0x7fbd78103ac0 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.992+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd781a6ec0 con 0x7fbd78103ac0 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.996+0000 7fbd767fc640 1 -- 172.21.15.103:0/1133845568 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbd68010c30 con 0x7fbd78103ac0 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.997+0000 7fbd818c3740 1 Processor -- start 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd818c3740 1 -- start start 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd767fc640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd600a2cc0 0x7fbd600a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca3ff0 0x55ce13ca43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13ca4910 0x55ce13caec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13caf1d0 0x55ce13cb15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd818c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ce13c09a40 con 0x55ce13ca4910 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd818c3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ce13ba19d0 con 0x55ce13caf1d0 2024-08-19T01:44:10.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd818c3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ce13c05a00 con 0x55ce13ca3ff0 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd7da62640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd600a2cc0 0x7fbd600a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd767fc640 1 -- 172.21.15.103:0/1133845568 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbd680cf6b0 con 0x7fbd78103ac0 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca3ff0 0x55ce13ca43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca3ff0 0x55ce13ca43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43630/0 (socket says 172.21.15.103:43630) 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd7d261640 1 -- 172.21.15.103:0/4041521386 learned_addr learned my addr 172.21.15.103:0/4041521386 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.998+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13ca4910 0x55ce13caec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7da62640 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13caf1d0 0x55ce13cb15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7ca60640 1 -- 172.21.15.103:0/4041521386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca3ff0 msgr2=0x55ce13ca43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca3ff0 0x55ce13ca43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7ca60640 1 -- 172.21.15.103:0/4041521386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13caf1d0 msgr2=0x55ce13cb15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13caf1d0 0x55ce13cb15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7ca60640 1 -- 172.21.15.103:0/4041521386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce13ba8650 con 0x55ce13ca4910 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7da62640 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13caf1d0 0x55ce13cb15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:09.999+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13ca4910 0x55ce13caec90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009860 tx=0x7fbd6c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04792adb4c2845f server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.000+0000 7fbd7da62640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd600a2cc0 0x7fbd600a5180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009fd0 tx=0x7fbd70011940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.000+0000 7fbd5f7fe640 1 -- 172.21.15.103:0/4041521386 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c041020 con 0x55ce13ca4910 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.000+0000 7fbd5f7fe640 1 -- 172.21.15.103:0/4041521386 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd6c030e50 con 0x55ce13ca4910 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.000+0000 7fbd5f7fe640 1 -- 172.21.15.103:0/4041521386 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c037040 con 0x55ce13ca4910 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 -- 172.21.15.103:0/4041521386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13ca4910 msgr2=0x55ce13caec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13ca4910 0x55ce13caec90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009860 tx=0x7fbd6c009520 comp rx=0 tx=0).stop 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 -- 172.21.15.103:0/4041521386 shutdown_connections 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13caf1d0 0x55ce13cb15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13ca4910 0x55ce13caec90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 --2- 172.21.15.103:0/4041521386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca3ff0 0x55ce13ca43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 -- 172.21.15.103:0/4041521386 >> 172.21.15.103:0/4041521386 conn(0x55ce13ca0580 msgr2=0x55ce13ca0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 -- 172.21.15.103:0/4041521386 shutdown_connections 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.001+0000 7fbd818c3740 1 -- 172.21.15.103:0/4041521386 wait complete. 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.002+0000 7fbd818c3740 1 -- start start 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.002+0000 7fbd818c3740 1 Processor -- start 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13caf1d0 0x55ce13cac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cbb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd818c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ce13cb1da0 con 0x55ce13cacc70 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd818c3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ce13bc4b20 con 0x55ce13ca3ff0 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd818c3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ce13cb1f20 con 0x55ce13caf1d0 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd818c3740 10 client.0 _ll_register_callbacks cb 0x55ce13bc2bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:10.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13caf1d0 0x55ce13cac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13caf1d0 0x55ce13cac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43644/0 (socket says 172.21.15.103:43644) 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1487277215 learned_addr learned my addr 172.21.15.103:0/1487277215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7d261640 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7da62640 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cbb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 msgr2=0x55ce13cac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1487277215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 msgr2=0x55ce13cbb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cbb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.003+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1487277215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1397b3c0 con 0x55ce13caf1d0 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13caf1d0 0x55ce13cac730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c02a3c0 tx=0x7fbd6c0309e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1f9085de17b994 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:10.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd5cff9640 1 -- 172.21.15.103:0/1487277215 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c04a070 con 0x55ce13caf1d0 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd5cff9640 1 -- 172.21.15.103:0/1487277215 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbd6c004450 con 0x55ce13caf1d0 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6c030e50 con 0x55ce13caf1d0 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd5cff9640 1 -- 172.21.15.103:0/1487277215 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbd6c055050 con 0x55ce13caf1d0 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd818c3740 10 client.26692 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.004+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ce13cb1df0 con 0x55ce13caf1d0 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd5cff9640 1 -- 172.21.15.103:0/1487277215 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd6c0054e0 con 0x55ce13caf1d0 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd5cff9640 1 -- 172.21.15.103:0/1487277215 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fbd6c004ca0 con 0x55ce13caf1d0 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd818c3740 10 client.26692 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd818c3740 10 client.26692 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd818c3740 20 client.26692 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13cbf570 0x55ce13cc1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd818c3740 4 client.26692 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:10.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbd6c030e50 con 0x55ce13cbf570 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.005+0000 7fbd7d261640 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13cbf570 0x55ce13cc1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.006+0000 7fbd7d261640 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13cbf570 0x55ce13cc1990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=991e70fed6a4618 server_cookie=581155ff5405fb4a in_seq=0 out_seq=0 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.006+0000 7fbd5cff9640 10 client.26692 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd5cff9640 1 -- 172.21.15.103:0/1487277215 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbd6c030e50 con 0x55ce13cbf570 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd5cff9640 10 client.26692 handle_command_reply: tid=0 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 client.26692 shutdown 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13caf1d0 msgr2=0x55ce13cac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13caf1d0 0x55ce13cac730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c02a3c0 tx=0x7fbd6c0309e0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 shutdown_connections 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13cbf570 0x55ce13cc1990 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cbb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13caf1d0 0x55ce13cac730 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1487277215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 >> 172.21.15.103:0/1487277215 conn(0x55ce13ca0580 msgr2=0x55ce13ca6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd5cff9640 0 client.26692 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:44:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 shutdown_connections 2024-08-19T01:44:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 1 -- 172.21.15.103:0/1487277215 wait complete. 2024-08-19T01:44:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.007+0000 7fbd818c3740 20 client.26692 trim_cache size 0 max 16384 2024-08-19T01:44:10.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 Processor -- start 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 -- start start 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cbb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cc7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cc77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781aaf30 con 0x55ce13cacc70 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781acb00 con 0x55ce13ca3ff0 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd818c3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781ab600 con 0x55ce13ca0580 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cbb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cc7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cc7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:43842/0 (socket says 172.21.15.103:43842) 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cbb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43650/0 (socket says 172.21.15.103:43650) 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1979439104 learned_addr learned my addr 172.21.15.103:0/1979439104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7da62640 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cc77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1979439104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 msgr2=0x55ce13cbb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cbb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1979439104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 msgr2=0x55ce13cc77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cc77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.030+0000 7fbd7ca60640 1 -- 172.21.15.103:0/1979439104 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce13ca4740 con 0x55ce13ca3ff0 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.031+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cc7280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c003730 tx=0x7fbd6c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da1f85b12e90d69 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.031+0000 7fbd5effd640 1 -- 172.21.15.103:0/1979439104 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c041070 con 0x55ce13ca3ff0 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.031+0000 7fbd5effd640 1 -- 172.21.15.103:0/1979439104 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd6c030c50 con 0x55ce13ca3ff0 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.031+0000 7fbd5effd640 1 -- 172.21.15.103:0/1979439104 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c04a040 con 0x55ce13ca3ff0 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 -- 172.21.15.103:0/1979439104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 msgr2=0x55ce13cc7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cc7280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c003730 tx=0x7fbd6c005d30 comp rx=0 tx=0).stop 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 -- 172.21.15.103:0/1979439104 shutdown_connections 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cc77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cc7280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 --2- 172.21.15.103:0/1979439104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cbb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 -- 172.21.15.103:0/1979439104 >> 172.21.15.103:0/1979439104 conn(0x55ce13cac890 msgr2=0x55ce13c9b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 -- 172.21.15.103:0/1979439104 shutdown_connections 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.032+0000 7fbd818c3740 1 -- 172.21.15.103:0/1979439104 wait complete. 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.033+0000 7fbd818c3740 1 -- start start 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.033+0000 7fbd818c3740 1 Processor -- start 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.033+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cb2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cb2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd818c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd818c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781ac430 con 0x55ce13cacc70 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd818c3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781abcd0 con 0x55ce13ca3ff0 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd818c3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781a8f30 con 0x55ce13ca0580 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cb2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cb2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43654/0 (socket says 172.21.15.103:43654) 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 -- 172.21.15.103:0/2143543123 learned_addr learned my addr 172.21.15.103:0/2143543123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd818c3740 10 client.0 _ll_register_callbacks cb 0x7fbd680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cb2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7da62640 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cb2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 -- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 msgr2=0x55ce13cb2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cb2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 -- 172.21.15.103:0/2143543123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 msgr2=0x55ce13cb2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cb2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.034+0000 7fbd7d261640 1 -- 172.21.15.103:0/2143543123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce13d28410 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd7d261640 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cb2130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd6805d960 tx=0x7fbd68010f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333bffa08327bdbf server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd5cff9640 1 -- 172.21.15.103:0/2143543123 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680a4b50 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd5cff9640 1 -- 172.21.15.103:0/2143543123 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fbd680cd290 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6c030c50 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd5cff9640 1 -- 172.21.15.103:0/2143543123 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbd68098550 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd818c3740 10 client.26698 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ce13d296b0 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.035+0000 7fbd5cff9640 1 -- 172.21.15.103:0/2143543123 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd6c030c50 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd5cff9640 1 -- 172.21.15.103:0/2143543123 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fbd6809b330 con 0x55ce13ca0580 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd818c3740 10 client.26698 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:10.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd818c3740 10 client.26698 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd818c3740 20 client.26698 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd818c3740 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13caf1d0 0x55ce13d2be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd818c3740 4 client.26698 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce13c16fb0 con 0x55ce13caf1d0 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13caf1d0 0x55ce13d2be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd5cff9640 10 client.26698 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.036+0000 7fbd7ca60640 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13caf1d0 0x55ce13d2be90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d684b6ffec0d040 server_cookie=9a18c977ba450e4c in_seq=0 out_seq=0 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd5cff9640 1 -- 172.21.15.103:0/2143543123 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce13c16fb0 con 0x55ce13caf1d0 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd5cff9640 10 client.26698 handle_command_reply: tid=0 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 client.26698 shutdown 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 msgr2=0x55ce13cb2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cb2130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbd6805d960 tx=0x7fbd68010f20 comp rx=0 tx=0).stop 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 shutdown_connections 2024-08-19T01:44:10.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x55ce13caf1d0 0x55ce13d2be90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce13cacc70 0x55ce13cb2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd5cff9640 0 client.26698 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:44:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ce13ca3ff0 0x55ce13cb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 --2- 172.21.15.103:0/2143543123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce13ca0580 0x55ce13cb2130 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 >> 172.21.15.103:0/2143543123 conn(0x55ce13cac890 msgr2=0x55ce13caaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:10.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 shutdown_connections 2024-08-19T01:44:10.456 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:44:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 1 -- 172.21.15.103:0/2143543123 wait complete. 2024-08-19T01:44:10.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.037+0000 7fbd818c3740 20 client.26698 trim_cache size 0 max 16384 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd600a2cc0 msgr2=0x7fbd600a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd600a2cc0 0x7fbd600a5180 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009fd0 tx=0x7fbd70011940 comp rx=0 tx=0).stop 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78103ac0 msgr2=0x7fbd7819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78103ac0 0x7fbd7819a340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800ebc0 tx=0x7fbd6800c2d0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 shutdown_connections 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fbd600a2cc0 0x7fbd600a5180 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd78109ac0 0x7fbd7819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd78104460 0x7fbd7819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 --2- 172.21.15.103:0/1133845568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd78103ac0 0x7fbd7819a340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 >> 172.21.15.103:0/1133845568 conn(0x7fbd780ff7a0 msgr2=0x7fbd781000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 shutdown_connections 2024-08-19T01:44:10.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:10.038+0000 7fbd7f4c5640 1 -- 172.21.15.103:0/1133845568 wait complete. 2024-08-19T01:44:11.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:11 smithi103 ceph-mon[18436]: pgmap v591: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 311 op/s 2024-08-19T01:44:11.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:11 smithi190 ceph-mon[22032]: pgmap v591: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 311 op/s 2024-08-19T01:44:11.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:11 smithi146 ceph-mon[22098]: pgmap v591: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 311 op/s 2024-08-19T01:44:13.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:13 smithi103 ceph-mon[18436]: pgmap v592: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.50k op/s 2024-08-19T01:44:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:13 smithi190 ceph-mon[22032]: pgmap v592: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.50k op/s 2024-08-19T01:44:13.381 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:44:13.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:13 smithi146 ceph-mon[22098]: pgmap v592: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 1.50k op/s 2024-08-19T01:44:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:14.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.368+0000 7fc9b159a640 1 -- 172.21.15.103:0/3792141224 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8031880 con 0x7fc9b4101090 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.368+0000 7fc9b8d12640 1 -- 172.21.15.103:0/3792141224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 msgr2=0x7fc9b410dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.368+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/3792141224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410dcc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009900 tx=0x7fc9a802a3f0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.369+0000 7fc9b8d12640 1 -- 172.21.15.103:0/3792141224 shutdown_connections 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.369+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/3792141224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b410e200 0x7fc9b4110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.369+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/3792141224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410dcc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.369+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/3792141224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b41006e0 0x7fc9b4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.369+0000 7fc9b8d12640 1 -- 172.21.15.103:0/3792141224 >> 172.21.15.103:0/3792141224 conn(0x7fc9b40fc500 msgr2=0x7fc9b40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.369+0000 7fc9b8d12640 1 -- 172.21.15.103:0/3792141224 shutdown_connections 2024-08-19T01:44:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.369+0000 7fc9b8d12640 1 -- 172.21.15.103:0/3792141224 wait complete. 2024-08-19T01:44:14.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 Processor -- start 2024-08-19T01:44:14.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 -- start start 2024-08-19T01:44:14.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b41006e0 0x7fc9b410c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b410e200 0x7fc9b410cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41034b0 con 0x7fc9b410e200 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4103330 con 0x7fc9b41006e0 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b8d12640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4103630 con 0x7fc9b4101090 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b41006e0 0x7fc9b410c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b1d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b1d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58446/0 (socket says 172.21.15.103:58446) 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b1d9b640 1 -- 172.21.15.103:0/220700563 learned_addr learned my addr 172.21.15.103:0/220700563 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b41006e0 0x7fc9b410c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:60934/0 (socket says 172.21.15.103:60934) 2024-08-19T01:44:14.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.370+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b410e200 0x7fc9b410cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9b259c640 1 -- 172.21.15.103:0/220700563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 msgr2=0x7fc9b410c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9b259c640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9b259c640 1 -- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b410e200 msgr2=0x7fc9b410cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9b259c640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b410e200 0x7fc9b410cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9b259c640 1 -- 172.21.15.103:0/220700563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b41aaae0 con 0x7fc9b41006e0 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9b259c640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b41006e0 0x7fc9b410c180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400e9f0 tx=0x7fc9a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.371+0000 7fc9a37fe640 1 -- 172.21.15.103:0/220700563 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400ccc0 con 0x7fc9b41006e0 2024-08-19T01:44:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.372+0000 7fc9a37fe640 1 -- 172.21.15.103:0/220700563 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a40035f0 con 0x7fc9b41006e0 2024-08-19T01:44:14.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.373+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b41aadd0 con 0x7fc9b41006e0 2024-08-19T01:44:14.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.373+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b41ab280 con 0x7fc9b41006e0 2024-08-19T01:44:14.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.373+0000 7fc9a37fe640 1 -- 172.21.15.103:0/220700563 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400ce60 con 0x7fc9b41006e0 2024-08-19T01:44:14.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.376+0000 7fc9bb110740 1 Processor -- start 2024-08-19T01:44:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.377+0000 7fc9bb110740 1 -- start start 2024-08-19T01:44:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.377+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x5629454133d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.377+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294541dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.377+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x5629454205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.377+0000 7fc9bb110740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562945378a40 con 0x562945412ff0 2024-08-19T01:44:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.377+0000 7fc9bb110740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5629453109d0 con 0x56294541e1d0 2024-08-19T01:44:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.377+0000 7fc9bb110740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562945374a00 con 0x562945413910 2024-08-19T01:44:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.378+0000 7fc9b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x5629454133d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.378+0000 7fc9a37fe640 1 -- 172.21.15.103:0/220700563 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc9a4004000 con 0x7fc9b41006e0 2024-08-19T01:44:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.378+0000 7fc9b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x5629454133d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56936/0 (socket says 172.21.15.103:56936) 2024-08-19T01:44:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.378+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/754173809 learned_addr learned my addr 172.21.15.103:0/754173809 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.378+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x5629454205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.378+0000 7fc9b259c640 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294541dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.378+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/754173809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 msgr2=0x56294541dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:14.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294541dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/754173809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 msgr2=0x5629454205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:14.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x5629454205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/754173809 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562945317650 con 0x562945412ff0 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9a37fe640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc99c0a2b20 0x7fc99c0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x5629454205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b259c640 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294541dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9a37fe640 1 -- 172.21.15.103:0/220700563 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9a40cf090 con 0x7fc9b41006e0 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b259c640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc99c0a2b20 0x7fc99c0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x5629454133d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009f60 tx=0x7fc9ac009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4026c0e8dfec051b server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9a0ff9640 1 -- 172.21.15.103:0/754173809 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac01e070 con 0x562945412ff0 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9a0ff9640 1 -- 172.21.15.103:0/754173809 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9ac004030 con 0x562945412ff0 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.379+0000 7fc9a0ff9640 1 -- 172.21.15.103:0/754173809 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00f6e0 con 0x562945412ff0 2024-08-19T01:44:14.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.380+0000 7fc9b259c640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc99c0a2b20 0x7fc99c0a4fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40ccf30 tx=0x7fc9a40d2b70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 -- 172.21.15.103:0/754173809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 msgr2=0x5629454133d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x5629454133d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009f60 tx=0x7fc9ac009f90 comp rx=0 tx=0).stop 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 -- 172.21.15.103:0/754173809 shutdown_connections 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x5629454205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294541dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 --2- 172.21.15.103:0/754173809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x5629454133d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 -- 172.21.15.103:0/754173809 >> 172.21.15.103:0/754173809 conn(0x56294540f580 msgr2=0x56294540f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:14.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.381+0000 7fc9bb110740 1 -- 172.21.15.103:0/754173809 shutdown_connections 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.382+0000 7fc9bb110740 1 -- 172.21.15.103:0/754173809 wait complete. 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.382+0000 7fc9bb110740 1 -- start start 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.382+0000 7fc9bb110740 1 Processor -- start 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.383+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945412ff0 0x56294541dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.383+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945413910 0x56294541afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.383+0000 7fc9b1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945413910 0x56294541afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.383+0000 7fc9b1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945413910 0x56294541afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56948/0 (socket says 172.21.15.103:56948) 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b1d9b640 1 -- 172.21.15.103:0/921892847 learned_addr learned my addr 172.21.15.103:0/921892847 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9bb110740 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x56294541b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562945420da0 con 0x562945413910 2024-08-19T01:44:14.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562945333b20 con 0x562945412ff0 2024-08-19T01:44:14.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562945420f20 con 0x56294541e1d0 2024-08-19T01:44:14.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945412ff0 0x56294541dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x56294541b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9bb110740 10 client.0 _ll_register_callbacks cb 0x562945331bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:14.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 msgr2=0x56294541b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:14.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x56294541b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/921892847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945413910 msgr2=0x56294541afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945413910 0x56294541afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.384+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/921892847 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629450ea3c0 con 0x562945412ff0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945412ff0 0x56294541dc50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009f30 tx=0x7fc9ac004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38baf5be11cc0069 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945413910 0x56294541afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc999ffb640 1 -- 172.21.15.103:0/921892847 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac01e040 con 0x562945412ff0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc999ffb640 1 -- 172.21.15.103:0/921892847 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9ac00f480 con 0x562945412ff0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ac004030 con 0x562945412ff0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc999ffb640 1 -- 172.21.15.103:0/921892847 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9ac00f6e0 con 0x562945412ff0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.385+0000 7fc9bb110740 10 client.26765 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56294542c870 con 0x562945412ff0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc999ffb640 1 -- 172.21.15.103:0/921892847 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9ac004030 con 0x562945412ff0 2024-08-19T01:44:14.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc999ffb640 1 -- 172.21.15.103:0/921892847 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fc9ac024070 con 0x562945412ff0 2024-08-19T01:44:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9bb110740 10 client.26765 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9bb110740 10 client.26765 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:44:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9bb110740 20 client.26765 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9bb110740 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x56294542de80 0x562945430340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9bb110740 4 client.26765 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562945385fb0 con 0x56294542de80 2024-08-19T01:44:14.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.386+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x56294542de80 0x562945430340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.387+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x56294542de80 0x562945430340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d3f83a9b4bdd5dc server_cookie=c712077345095ced in_seq=0 out_seq=0 2024-08-19T01:44:14.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.387+0000 7fc999ffb640 10 client.26765 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:44:14.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.388+0000 7fc999ffb640 1 -- 172.21.15.103:0/921892847 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562945385fb0 con 0x56294542de80 2024-08-19T01:44:14.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.388+0000 7fc999ffb640 10 client.26765 handle_command_reply: tid=0 2024-08-19T01:44:14.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 client.26765 shutdown 2024-08-19T01:44:14.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945412ff0 msgr2=0x56294541dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945412ff0 0x56294541dc50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009f30 tx=0x7fc9ac004900 comp rx=0 tx=0).stop 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 shutdown_connections 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x56294542de80 0x562945430340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x56294541b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945413910 0x56294541afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 --2- 172.21.15.103:0/921892847 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945412ff0 0x56294541dc50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc999ffb640 0 client.26765 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 >> 172.21.15.103:0/921892847 conn(0x56294540f580 msgr2=0x562945415260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.389+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 shutdown_connections 2024-08-19T01:44:14.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.390+0000 7fc9bb110740 1 -- 172.21.15.103:0/921892847 wait complete. 2024-08-19T01:44:14.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.390+0000 7fc9bb110740 20 client.26765 trim_cache size 0 max 16384 2024-08-19T01:44:14.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.410+0000 7fc9bb110740 1 Processor -- start 2024-08-19T01:44:14.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.410+0000 7fc9bb110740 1 -- start start 2024-08-19T01:44:14.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x56294542a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294540f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x562945418200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9bb110740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b35c0 con 0x562945412ff0 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9bb110740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b5190 con 0x56294541e1d0 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9bb110740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b3c90 con 0x562945413910 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x56294542a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x56294542a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56954/0 (socket says 172.21.15.103:56954) 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/2428113056 learned_addr learned my addr 172.21.15.103:0/2428113056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.411+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294540f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.412+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x562945418200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.412+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/2428113056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 msgr2=0x56294540f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:14.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.412+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294540f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.412+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/2428113056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 msgr2=0x562945418200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:14.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.412+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x562945418200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.412+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/2428113056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562945410170 con 0x562945412ff0 2024-08-19T01:44:14.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.412+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294540f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:14.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.413+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x56294542a180 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0327b0 tx=0x7fc9ac005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3a92b0ce00bfc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:14.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.413+0000 7fc99bfff640 1 -- 172.21.15.103:0/2428113056 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac01e040 con 0x562945412ff0 2024-08-19T01:44:14.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.413+0000 7fc99bfff640 1 -- 172.21.15.103:0/2428113056 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9ac02c070 con 0x562945412ff0 2024-08-19T01:44:14.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.413+0000 7fc99bfff640 1 -- 172.21.15.103:0/2428113056 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac032b30 con 0x562945412ff0 2024-08-19T01:44:14.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 -- 172.21.15.103:0/2428113056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 msgr2=0x56294542a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x56294542a180 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0327b0 tx=0x7fc9ac005f70 comp rx=0 tx=0).stop 2024-08-19T01:44:14.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 -- 172.21.15.103:0/2428113056 shutdown_connections 2024-08-19T01:44:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56294541e1d0 0x562945418200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562945413910 0x56294540f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 --2- 172.21.15.103:0/2428113056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x56294542a180 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 -- 172.21.15.103:0/2428113056 >> 172.21.15.103:0/2428113056 conn(0x56294540f580 msgr2=0x56294540a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.414+0000 7fc9bb110740 1 -- 172.21.15.103:0/2428113056 shutdown_connections 2024-08-19T01:44:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.415+0000 7fc9bb110740 1 -- 172.21.15.103:0/2428113056 wait complete. 2024-08-19T01:44:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.415+0000 7fc9bb110740 1 -- start start 2024-08-19T01:44:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.415+0000 7fc9bb110740 1 Processor -- start 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x562945430ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945413910 0x5629454313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9bb110740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x5629454969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x562945430ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x562945430ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56964/0 (socket says 172.21.15.103:56964) 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/3807351744 learned_addr learned my addr 172.21.15.103:0/3807351744 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b4ac0 con 0x562945412ff0 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b4360 con 0x562945413910 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b41b15a0 con 0x56294541e1d0 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x5629454969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.416+0000 7fc9bb110740 10 client.0 _ll_register_callbacks cb 0x7fc9ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.417+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 msgr2=0x5629454969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:14.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.417+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x5629454969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.417+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/3807351744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945413910 msgr2=0x5629454313e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:44:14.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.417+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945413910 0x5629454313e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.417+0000 7fc9b2d9d640 1 -- 172.21.15.103:0/3807351744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562945496f70 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.417+0000 7fc9b2d9d640 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x562945430ea0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac013040 tx=0x7fc9ac00fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1fbae689d2c63c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.417+0000 7fc999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.418+0000 7fc999ffb640 1 -- 172.21.15.103:0/3807351744 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac009820 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.418+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ac02c070 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.418+0000 7fc999ffb640 1 -- 172.21.15.103:0/3807351744 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9ac032ae0 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.418+0000 7fc999ffb640 1 -- 172.21.15.103:0/3807351744 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc9ac032ce0 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.418+0000 7fc9bb110740 10 client.17523 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.418+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5629454981f0 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc999ffb640 1 -- 172.21.15.103:0/3807351744 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9ac02c070 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc999ffb640 1 -- 172.21.15.103:0/3807351744 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fc9ac024070 con 0x562945412ff0 2024-08-19T01:44:14.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc9bb110740 10 client.17523 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc9bb110740 10 client.17523 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:44:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc9bb110740 20 client.17523 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc9bb110740 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5629454988a0 0x56294549ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc9bb110740 4 client.17523 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:44:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.419+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629453804c0 con 0x5629454988a0 2024-08-19T01:44:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.420+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5629454988a0 0x56294549ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.420+0000 7fc999ffb640 10 client.17523 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.420+0000 7fc9b1d9b640 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5629454988a0 0x56294549ac90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11faff87a35087c3 server_cookie=488022e906e8a351 in_seq=0 out_seq=0 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.420+0000 7fc999ffb640 1 -- 172.21.15.103:0/3807351744 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629453804c0 con 0x5629454988a0 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.420+0000 7fc999ffb640 10 client.17523 handle_command_reply: tid=0 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.420+0000 7fc9bb110740 1 client.17523 shutdown 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 msgr2=0x562945430ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x562945430ea0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac013040 tx=0x7fc9ac00fe30 comp rx=0 tx=0).stop 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 shutdown_connections 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x5629454988a0 0x56294549ac90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56294541e1d0 0x5629454969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562945413910 0x5629454313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 --2- 172.21.15.103:0/3807351744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562945412ff0 0x562945430ea0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 >> 172.21.15.103:0/3807351744 conn(0x56294540f580 msgr2=0x56294540f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:14.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc999ffb640 0 client.17523 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:44:14.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.421+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 shutdown_connections 2024-08-19T01:44:14.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.422+0000 7fc9bb110740 1 -- 172.21.15.103:0/3807351744 wait complete. 2024-08-19T01:44:14.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.422+0000 7fc9bb110740 20 client.17523 trim_cache size 0 max 16384 2024-08-19T01:44:14.425 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.422+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc99c0a2b20 msgr2=0x7fc99c0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.422+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc99c0a2b20 0x7fc99c0a4fe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40ccf30 tx=0x7fc9a40d2b70 comp rx=0 tx=0).stop 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.422+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b41006e0 msgr2=0x7fc9b410c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.422+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b41006e0 0x7fc9b410c180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400e9f0 tx=0x7fc9a400edf0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 shutdown_connections 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc99c0a2b20 0x7fc99c0a4fe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b410e200 0x7fc9b410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b4101090 0x7fc9b410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 --2- 172.21.15.103:0/220700563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9b41006e0 0x7fc9b410c180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 >> 172.21.15.103:0/220700563 conn(0x7fc9b40fc500 msgr2=0x7fc9b40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 shutdown_connections 2024-08-19T01:44:14.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:14.423+0000 7fc9b8d12640 1 -- 172.21.15.103:0/220700563 wait complete. 2024-08-19T01:44:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.039+0000 7fed45229640 1 -- 172.21.15.103:0/2762615392 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38031e40 con 0x7fed40103440 2024-08-19T01:44:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 -- 172.21.15.103:0/2762615392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40103440 msgr2=0x7fed40109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 --2- 172.21.15.103:0/2762615392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40103440 0x7fed40109cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed38009ca0 tx=0x7fed3802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 -- 172.21.15.103:0/2762615392 shutdown_connections 2024-08-19T01:44:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 --2- 172.21.15.103:0/2762615392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40103440 0x7fed40109cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 --2- 172.21.15.103:0/2762615392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40102a80 0x7fed40102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 --2- 172.21.15.103:0/2762615392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed40101890 0x7fed40101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 -- 172.21.15.103:0/2762615392 >> 172.21.15.103:0/2762615392 conn(0x7fed400fcff0 msgr2=0x7fed400ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 -- 172.21.15.103:0/2762615392 shutdown_connections 2024-08-19T01:44:15.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.040+0000 7fed4848f640 1 -- 172.21.15.103:0/2762615392 wait complete. 2024-08-19T01:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 Processor -- start 2024-08-19T01:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 -- start start 2024-08-19T01:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40101890 0x7fed40075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40102a80 0x7fed40074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed40103440 0x7fed40074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010d470 con 0x7fed40101890 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed45a2a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40102a80 0x7fed40074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed45a2a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40102a80 0x7fed40074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58500/0 (socket says 172.21.15.103:58500) 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4622b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40101890 0x7fed40075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed45a2a640 1 -- 172.21.15.103:0/1782143485 learned_addr learned my addr 172.21.15.103:0/1782143485 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed46a2c640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed40103440 0x7fed40074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010d2f0 con 0x7fed40103440 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.042+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010d5f0 con 0x7fed40102a80 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4622b640 1 -- 172.21.15.103:0/1782143485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40102a80 msgr2=0x7fed40074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4622b640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40102a80 0x7fed40074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4622b640 1 -- 172.21.15.103:0/1782143485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed40103440 msgr2=0x7fed40074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4622b640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed40103440 0x7fed40074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4622b640 1 -- 172.21.15.103:0/1782143485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4019b990 con 0x7fed40101890 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed45a2a640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40102a80 0x7fed40074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4622b640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40101890 0x7fed40075cd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00c940 tx=0x7fed3c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed377fe640 1 -- 172.21.15.103:0/1782143485 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c004220 con 0x7fed40101890 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed377fe640 1 -- 172.21.15.103:0/1782143485 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3c0043c0 con 0x7fed40101890 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed377fe640 1 -- 172.21.15.103:0/1782143485 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00f450 con 0x7fed40101890 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4019bc80 con 0x7fed40101890 2024-08-19T01:44:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.043+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4019c180 con 0x7fed40101890 2024-08-19T01:44:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.046+0000 7fed377fe640 1 -- 172.21.15.103:0/1782143485 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fed3c0079b0 con 0x7fed40101890 2024-08-19T01:44:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.048+0000 7fed377fe640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fed2c0a2cb0 0x7fed2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.048+0000 7fed45a2a640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fed2c0a2cb0 0x7fed2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.049+0000 7fed377fe640 1 -- 172.21.15.103:0/1782143485 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fed3c0cec80 con 0x7fed40101890 2024-08-19T01:44:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.049+0000 7fed45a2a640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fed2c0a2cb0 0x7fed2c0a5170 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fed40075a40 tx=0x7fed300091c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.058+0000 7fed4b24d740 1 Processor -- start 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.058+0000 7fed4b24d740 1 -- start start 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dbea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dc6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4b24d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5634a7d219d0 con 0x5634a7dbe680 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4b24d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5634a7cc9820 con 0x5634a7dc4050 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4b24d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5634a7d1d990 con 0x5634a7dbefc0 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dbea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dbea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57010/0 (socket says 172.21.15.103:57010) 2024-08-19T01:44:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 -- 172.21.15.103:0/1053890833 learned_addr learned my addr 172.21.15.103:0/1053890833 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed46a2c640 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dc6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed45a2a640 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 -- 172.21.15.103:0/1053890833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 msgr2=0x5634a7dc3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 -- 172.21.15.103:0/1053890833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 msgr2=0x5634a7dc6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dc6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.059+0000 7fed4622b640 1 -- 172.21.15.103:0/1053890833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a7a35210 con 0x5634a7dbe680 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.060+0000 7fed4622b640 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dbea80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fed3c097f90 tx=0x7fed3c09b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed25e540f1484d2e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.060+0000 7fed34bfe640 1 -- 172.21.15.103:0/1053890833 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c01d040 con 0x5634a7dbe680 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.060+0000 7fed34bfe640 1 -- 172.21.15.103:0/1053890833 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3c09f8c0 con 0x5634a7dbe680 2024-08-19T01:44:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.060+0000 7fed34bfe640 1 -- 172.21.15.103:0/1053890833 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c09aa10 con 0x5634a7dbe680 2024-08-19T01:44:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 -- 172.21.15.103:0/1053890833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 msgr2=0x5634a7dbea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dbea80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fed3c097f90 tx=0x7fed3c09b450 comp rx=0 tx=0).stop 2024-08-19T01:44:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 -- 172.21.15.103:0/1053890833 shutdown_connections 2024-08-19T01:44:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dc6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1053890833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dbea80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 -- 172.21.15.103:0/1053890833 >> 172.21.15.103:0/1053890833 conn(0x5634a7db9cf0 msgr2=0x5634a7dba0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 -- 172.21.15.103:0/1053890833 shutdown_connections 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.061+0000 7fed4b24d740 1 -- 172.21.15.103:0/1053890833 wait complete. 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 -- start start 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 Processor -- start 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dc86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dcfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5634a7dc6fc0 con 0x5634a7dbe680 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5634a7cdd520 con 0x5634a7dc4050 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5634a7dc7140 con 0x5634a7dbefc0 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4b24d740 10 client.0 _ll_register_callbacks cb 0x5634a7a34a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4622b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dc86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4622b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dc86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57026/0 (socket says 172.21.15.103:57026) 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed4622b640 1 -- 172.21.15.103:0/248460454 learned_addr learned my addr 172.21.15.103:0/248460454 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.062+0000 7fed45a2a640 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed46a2c640 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dcfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed4622b640 1 -- 172.21.15.103:0/248460454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 msgr2=0x5634a7dc8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed4622b640 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed4622b640 1 -- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 msgr2=0x5634a7dcfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed4622b640 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dcfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed4622b640 1 -- 172.21.15.103:0/248460454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a7b01620 con 0x5634a7dbe680 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed4622b640 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dc86c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fed3c097a20 tx=0x7fed3c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946d2554c1a9ad5d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed29ffb640 1 -- 172.21.15.103:0/248460454 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0b9060 con 0x5634a7dbe680 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c09f8c0 con 0x5634a7dbe680 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.063+0000 7fed29ffb640 1 -- 172.21.15.103:0/248460454 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fed3c0b4260 con 0x5634a7dbe680 2024-08-19T01:44:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed29ffb640 1 -- 172.21.15.103:0/248460454 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fed3c01d070 con 0x5634a7dbe680 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 10 client.17532 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5634a7dc6fc0 con 0x5634a7dbe680 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed29ffb640 1 -- 172.21.15.103:0/248460454 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3c09f8c0 con 0x5634a7dbe680 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed29ffb640 1 -- 172.21.15.103:0/248460454 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fed3c09a480 con 0x5634a7dbe680 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 10 client.17532 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 10 client.17532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 20 client.17532 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7dd4130 0x5634a7dd6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 4 client.17532 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.064+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634a7d2f100 con 0x5634a7dd4130 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.065+0000 7fed45a2a640 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7dd4130 0x5634a7dd6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.065+0000 7fed29ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.065+0000 7fed45a2a640 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7dd4130 0x5634a7dd6550 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19bc5d3e542a4b9e server_cookie=b848e147105b3dbe in_seq=0 out_seq=0 2024-08-19T01:44:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed29ffb640 1 -- 172.21.15.103:0/248460454 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634a7d2f100 con 0x5634a7dd4130 2024-08-19T01:44:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed29ffb640 10 client.17532 handle_command_reply: tid=0 2024-08-19T01:44:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 client.17532 shutdown 2024-08-19T01:44:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 msgr2=0x5634a7dc86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dc86c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fed3c097a20 tx=0x7fed3c004570 comp rx=0 tx=0).stop 2024-08-19T01:44:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 shutdown_connections 2024-08-19T01:44:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7dd4130 0x5634a7dd6550 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dc4050 0x5634a7dcfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbefc0 0x5634a7dc8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 --2- 172.21.15.103:0/248460454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dc86c0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 >> 172.21.15.103:0/248460454 conn(0x5634a7db9cf0 msgr2=0x5634a7dbb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.066+0000 7fed29ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.067+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 shutdown_connections 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.067+0000 7fed4b24d740 1 -- 172.21.15.103:0/248460454 wait complete. 2024-08-19T01:44:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.067+0000 7fed4b24d740 20 client.17532 trim_cache size 0 max 16384 2024-08-19T01:44:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.089+0000 7fed4b24d740 1 Processor -- start 2024-08-19T01:44:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.089+0000 7fed4b24d740 1 -- start start 2024-08-19T01:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dc4050 0x5634a7dba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4b24d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b78b0 con 0x5634a7dbe680 2024-08-19T01:44:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4622b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed45a2a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4b24d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b9480 con 0x5634a7dbefc0 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4b24d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b7f80 con 0x5634a7dc4050 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4622b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dba0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57038/0 (socket says 172.21.15.103:57038) 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed4622b640 1 -- 172.21.15.103:0/4002399071 learned_addr learned my addr 172.21.15.103:0/4002399071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed45a2a640 1 -- 172.21.15.103:0/4002399071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dc4050 msgr2=0x5634a7dba610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed45a2a640 1 --2- 172.21.15.103:0/4002399071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dc4050 0x5634a7dba610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed45a2a640 1 -- 172.21.15.103:0/4002399071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 msgr2=0x5634a7dba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed45a2a640 1 --2- 172.21.15.103:0/4002399071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.090+0000 7fed45a2a640 1 -- 172.21.15.103:0/4002399071 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a7dbfc50 con 0x5634a7dbefc0 2024-08-19T01:44:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.091+0000 7fed45a2a640 1 --2- 172.21.15.103:0/4002399071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcfc00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5634a7dd6dd0 tx=0x7fed30015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bad146a117cd184 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.091+0000 7fed2bfff640 1 -- 172.21.15.103:0/4002399071 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3001b040 con 0x5634a7dbefc0 2024-08-19T01:44:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.091+0000 7fed2bfff640 1 -- 172.21.15.103:0/4002399071 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3002e070 con 0x5634a7dbefc0 2024-08-19T01:44:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.091+0000 7fed2bfff640 1 -- 172.21.15.103:0/4002399071 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30015d50 con 0x5634a7dbefc0 2024-08-19T01:44:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.092+0000 7fed4b24d740 1 -- 172.21.15.103:0/4002399071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 msgr2=0x5634a7dcfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.092+0000 7fed4b24d740 1 --2- 172.21.15.103:0/4002399071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcfc00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5634a7dd6dd0 tx=0x7fed30015000 comp rx=0 tx=0).stop 2024-08-19T01:44:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 -- 172.21.15.103:0/4002399071 shutdown_connections 2024-08-19T01:44:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 --2- 172.21.15.103:0/4002399071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dc4050 0x5634a7dba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 --2- 172.21.15.103:0/4002399071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcfc00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 --2- 172.21.15.103:0/4002399071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dbe680 0x5634a7dba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 -- 172.21.15.103:0/4002399071 >> 172.21.15.103:0/4002399071 conn(0x5634a7db9cf0 msgr2=0x5634a7dc8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 -- 172.21.15.103:0/4002399071 shutdown_connections 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 -- 172.21.15.103:0/4002399071 wait complete. 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.093+0000 7fed4b24d740 1 -- start start 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.094+0000 7fed4b24d740 1 Processor -- start 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.094+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbe680 0x5634a7dcf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.094+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.094+0000 7fed4b24d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dc4050 0x5634a7dcc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.094+0000 7fed4b24d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b8db0 con 0x5634a7dc4050 2024-08-19T01:44:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4b24d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fed401b8650 con 0x5634a7dbefc0 2024-08-19T01:44:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4b24d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed40101cd0 con 0x5634a7dbe680 2024-08-19T01:44:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4b24d740 10 client.0 _ll_register_callbacks cb 0x7fed3c09adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4622b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbe680 0x5634a7dcf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed46a2c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dc4050 0x5634a7dcc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed46a2c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dc4050 0x5634a7dcc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57046/0 (socket says 172.21.15.103:57046) 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed46a2c640 1 -- 172.21.15.103:0/1434251534 learned_addr learned my addr 172.21.15.103:0/1434251534 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed45a2a640 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4622b640 1 -- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 msgr2=0x5634a7dcc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4622b640 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4622b640 1 -- 172.21.15.103:0/1434251534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dc4050 msgr2=0x5634a7dcc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4622b640 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dc4050 0x5634a7dcc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.095+0000 7fed4622b640 1 -- 172.21.15.103:0/1434251534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a7dcfe00 con 0x5634a7dbe680 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed4622b640 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbe680 0x5634a7dcf7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed3c09b090 tx=0x7fed3c09b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b540d8bf13ab2b52 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed29ffb640 1 -- 172.21.15.103:0/1434251534 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0b9070 con 0x5634a7dbe680 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed29ffb640 1 -- 172.21.15.103:0/1434251534 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fed3c097310 con 0x5634a7dbe680 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3002e070 con 0x5634a7dbe680 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed29ffb640 1 -- 172.21.15.103:0/1434251534 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fed3c097680 con 0x5634a7dbe680 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed4b24d740 10 client.26731 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5634a7e3dda0 con 0x5634a7dbe680 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.096+0000 7fed29ffb640 1 -- 172.21.15.103:0/1434251534 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3002e070 con 0x5634a7dbe680 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed29ffb640 1 -- 172.21.15.103:0/1434251534 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fed3c0b4260 con 0x5634a7dbe680 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed4b24d740 10 client.26731 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed4b24d740 10 client.26731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed4b24d740 20 client.26731 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7e3e590 0x5634a7e409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed4b24d740 4 client.26731 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634a7d29450 con 0x5634a7e3e590 2024-08-19T01:44:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.097+0000 7fed45a2a640 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7e3e590 0x5634a7e409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.098+0000 7fed29ffb640 10 client.26731 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.098+0000 7fed45a2a640 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7e3e590 0x5634a7e409b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af86fca6d0513e83 server_cookie=a5017693721c7795 in_seq=0 out_seq=0 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed29ffb640 1 -- 172.21.15.103:0/1434251534 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634a7d29450 con 0x5634a7e3e590 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed29ffb640 10 client.26731 handle_command_reply: tid=0 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed4b24d740 1 client.26731 shutdown 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbe680 msgr2=0x5634a7dcf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbe680 0x5634a7dcf7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed3c09b090 tx=0x7fed3c09b450 comp rx=0 tx=0).stop 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 shutdown_connections 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5634a7e3e590 0x5634a7e409b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5634a7dc4050 0x5634a7dcc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed29ffb640 0 client.26731 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.106+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5634a7dbefc0 0x5634a7dcc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.107+0000 7fed4b24d740 1 --2- 172.21.15.103:0/1434251534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634a7dbe680 0x5634a7dcf7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.107+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 >> 172.21.15.103:0/1434251534 conn(0x5634a7db9cf0 msgr2=0x5634a7db8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.107+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 shutdown_connections 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.107+0000 7fed4b24d740 1 -- 172.21.15.103:0/1434251534 wait complete. 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.107+0000 7fed4b24d740 20 client.26731 trim_cache size 0 max 16384 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:44:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:44:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fed2c0a2cb0 msgr2=0x7fed2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fed2c0a2cb0 0x7fed2c0a5170 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fed40075a40 tx=0x7fed300091c0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40101890 msgr2=0x7fed40075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40101890 0x7fed40075cd0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00c940 tx=0x7fed3c00ce00 comp rx=0 tx=0).stop 2024-08-19T01:44:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 shutdown_connections 2024-08-19T01:44:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fed2c0a2cb0 0x7fed2c0a5170 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fed40103440 0x7fed40074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed40102a80 0x7fed40074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 --2- 172.21.15.103:0/1782143485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40101890 0x7fed40075cd0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 >> 172.21.15.103:0/1782143485 conn(0x7fed400fcff0 msgr2=0x7fed40105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 shutdown_connections 2024-08-19T01:44:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:15.108+0000 7fed4848f640 1 -- 172.21.15.103:0/1782143485 wait complete. 2024-08-19T01:44:15.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:15 smithi103 ceph-mon[18436]: pgmap v593: 97 pgs: 97 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 2.26k op/s 2024-08-19T01:44:15.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:15 smithi190 ceph-mon[22032]: pgmap v593: 97 pgs: 97 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 2.26k op/s 2024-08-19T01:44:15.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:44:15.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:15 smithi146 ceph-mon[22098]: pgmap v593: 97 pgs: 97 active+clean; 13 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 2.26k op/s 2024-08-19T01:44:15.826 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:16.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 -- 172.21.15.103:0/1545331101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1010df60 msgr2=0x7f6f1010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 --2- 172.21.15.103:0/1545331101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1010df60 0x7f6f1010e340 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009930 tx=0x7f6f0c02a420 comp rx=0 tx=0).stop 2024-08-19T01:44:16.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 -- 172.21.15.103:0/1545331101 shutdown_connections 2024-08-19T01:44:16.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 --2- 172.21.15.103:0/1545331101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f10108880 0x7f6f10114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 --2- 172.21.15.103:0/1545331101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10107f20 0x7f6f10108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 --2- 172.21.15.103:0/1545331101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f1010df60 0x7f6f1010e340 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 -- 172.21.15.103:0/1545331101 >> 172.21.15.103:0/1545331101 conn(0x7f6f100778e0 msgr2=0x7f6f10079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:16.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 -- 172.21.15.103:0/1545331101 shutdown_connections 2024-08-19T01:44:16.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.398+0000 7f6f185ef640 1 -- 172.21.15.103:0/1545331101 wait complete. 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.399+0000 7f6f185ef640 1 Processor -- start 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.399+0000 7f6f185ef640 1 -- start start 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f185ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f10107f20 0x7f6f100825b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f185ef640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10108880 0x7f6f1007b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f185ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1010df60 0x7f6f1007bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f185ef640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10118160 con 0x7f6f10107f20 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f185ef640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10117fe0 con 0x7f6f10108880 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f185ef640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101182e0 con 0x7f6f1010df60 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f15b8a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10108880 0x7f6f1007b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f15b8a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10108880 0x7f6f1007b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:32806/0 (socket says 172.21.15.103:32806) 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f16b8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1010df60 0x7f6f1007bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f15b8a640 1 -- 172.21.15.103:0/2485609021 learned_addr learned my addr 172.21.15.103:0/2485609021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:16.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.400+0000 7f6f1638b640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f10107f20 0x7f6f100825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f15b8a640 1 -- 172.21.15.103:0/2485609021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1010df60 msgr2=0x7f6f1007bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1010df60 0x7f6f1007bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f15b8a640 1 -- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f10107f20 msgr2=0x7f6f100825b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f10107f20 0x7f6f100825b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f15b8a640 1 -- 172.21.15.103:0/2485609021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1007c190 con 0x7f6f10108880 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f16b8c640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1010df60 0x7f6f1007bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10108880 0x7f6f1007b640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000d950 tx=0x7f6f0000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f077fe640 1 -- 172.21.15.103:0/2485609021 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00017030 con 0x7f6f10108880 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f077fe640 1 -- 172.21.15.103:0/2485609021 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f00004020 con 0x7f6f10108880 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.401+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1007c480 con 0x7f6f10108880 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.402+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f100804a0 con 0x7f6f10108880 2024-08-19T01:44:16.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.402+0000 7f6f077fe640 1 -- 172.21.15.103:0/2485609021 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f000047a0 con 0x7f6f10108880 2024-08-19T01:44:16.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.406+0000 7f6f077fe640 1 -- 172.21.15.103:0/2485609021 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6f000041d0 con 0x7f6f10108880 2024-08-19T01:44:16.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.406+0000 7f6f1a9ed740 1 Processor -- start 2024-08-19T01:44:16.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.406+0000 7f6f1a9ed740 1 -- start start 2024-08-19T01:44:16.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 0x560e1d5e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e2910 0x560e1d5ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e1d547a40 con 0x560e1d5ed1d0 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560e1d4df9d0 con 0x560e1d5e2910 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560e1d543a00 con 0x560e1d5e1ff0 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5ef5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57072/0 (socket says 172.21.15.103:57072) 2024-08-19T01:44:16.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 -- 172.21.15.103:0/4081067770 learned_addr learned my addr 172.21.15.103:0/4081067770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:16.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 -- 172.21.15.103:0/4081067770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 msgr2=0x560e1d5e23d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:44:16.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/4081067770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 0x560e1d5e23d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 -- 172.21.15.103:0/4081067770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e2910 msgr2=0x560e1d5ecc90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/4081067770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e2910 0x560e1d5ecc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.407+0000 7f6f15b8a640 1 -- 172.21.15.103:0/4081067770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1d4e6650 con 0x560e1d5ed1d0 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.408+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/4081067770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5ef5c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6f000d2a30 tx=0x7f6f000d2ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d89d34f310f000 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.408+0000 7f6f04ff9640 1 -- 172.21.15.103:0/4081067770 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f000e2070 con 0x560e1d5ed1d0 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.408+0000 7f6f04ff9640 1 -- 172.21.15.103:0/4081067770 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f000d7c90 con 0x560e1d5ed1d0 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.408+0000 7f6f04ff9640 1 -- 172.21.15.103:0/4081067770 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f000dc600 con 0x560e1d5ed1d0 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.408+0000 7f6f077fe640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6efc0a2c00 0x7f6efc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.409+0000 7f6f16b8c640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6efc0a2c00 0x7f6efc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.409+0000 7f6f077fe640 1 -- 172.21.15.103:0/2485609021 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6f000cea20 con 0x7f6f10108880 2024-08-19T01:44:16.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4081067770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 msgr2=0x560e1d5ef5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f16b8c640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6efc0a2c00 0x7f6efc0a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f080092f0 tx=0x7f6f08009280 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:16.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4081067770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5ef5c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6f000d2a30 tx=0x7f6f000d2ef0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4081067770 shutdown_connections 2024-08-19T01:44:16.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4081067770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5ef5c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4081067770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e2910 0x560e1d5ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4081067770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 0x560e1d5e23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4081067770 >> 172.21.15.103:0/4081067770 conn(0x560e1d5de580 msgr2=0x560e1d5de960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4081067770 shutdown_connections 2024-08-19T01:44:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.410+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4081067770 wait complete. 2024-08-19T01:44:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 -- start start 2024-08-19T01:44:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 Processor -- start 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e2910 0x560e1d5ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5ed1d0 0x560e1d5eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e1d5efda0 con 0x560e1d5e2910 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560e1d502b20 con 0x560e1d5ed1d0 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560e1d5eff20 con 0x560e1d5e1ff0 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f1a9ed740 10 client.0 _ll_register_callbacks cb 0x560e1d500bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f15b8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e2910 0x560e1d5ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f15b8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e2910 0x560e1d5ea640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57088/0 (socket says 172.21.15.103:57088) 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f15b8a640 1 -- 172.21.15.103:0/393745197 learned_addr learned my addr 172.21.15.103:0/393745197 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.411+0000 7f6f16b8c640 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5ed1d0 0x560e1d5eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6f1638b640 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6f15b8a640 1 -- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 msgr2=0x560e1d5ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6f15b8a640 1 -- 172.21.15.103:0/393745197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5ed1d0 msgr2=0x560e1d5eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5ed1d0 0x560e1d5eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6f15b8a640 1 -- 172.21.15.103:0/393745197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1d2b93c0 con 0x560e1d5e2910 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e2910 0x560e1d5ea640 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6f000cc5a0 tx=0x7f6f000d2230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9adb8161182099 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/393745197 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00097550 con 0x560e1d5e2910 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.412+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/393745197 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6f000d7e40 con 0x560e1d5e2910 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.413+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f000d7c90 con 0x560e1d5e2910 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.413+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/393745197 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6f0009ac10 con 0x560e1d5e2910 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.413+0000 7f6f1a9ed740 10 client.17550 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.413+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560e1d5fb9e0 con 0x560e1d5e2910 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.413+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/393745197 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f000d7c90 con 0x560e1d5e2910 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.413+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/393745197 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f6f00097920 con 0x560e1d5e2910 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.414+0000 7f6f1a9ed740 10 client.17550 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.414+0000 7f6f1a9ed740 10 client.17550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.414+0000 7f6f1a9ed740 20 client.17550 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.414+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d5fcd30 0x560e1d5ff120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.414+0000 7f6f1a9ed740 4 client.17550 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.414+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e1d554fb0 con 0x560e1d5fcd30 2024-08-19T01:44:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.414+0000 7f6f1638b640 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d5fcd30 0x560e1d5ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.415+0000 7f6f1638b640 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d5fcd30 0x560e1d5ff120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4136afded5cefb4e server_cookie=6e5f0d550b4e1307 in_seq=0 out_seq=0 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.415+0000 7f6ef9ffb640 10 client.17550 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/393745197 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e1d554fb0 con 0x560e1d5fcd30 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6ef9ffb640 10 client.17550 handle_command_reply: tid=0 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 client.17550 shutdown 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e2910 msgr2=0x560e1d5ea640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e2910 0x560e1d5ea640 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6f000cc5a0 tx=0x7f6f000d2230 comp rx=0 tx=0).stop 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 shutdown_connections 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d5fcd30 0x560e1d5ff120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5ed1d0 0x560e1d5eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e2910 0x560e1d5ea640 secure :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6f000cc5a0 tx=0x7f6f000d2230 comp rx=0 tx=0).stop 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/393745197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6ef9ffb640 0 client.17550 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:44:16.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.416+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 >> 172.21.15.103:0/393745197 conn(0x560e1d5de580 msgr2=0x560e1d5e4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.417+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 shutdown_connections 2024-08-19T01:44:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.417+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/393745197 wait complete. 2024-08-19T01:44:16.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.417+0000 7f6f1a9ed740 20 client.17550 trim_cache size 0 max 16384 2024-08-19T01:44:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.440+0000 7f6f1a9ed740 1 Processor -- start 2024-08-19T01:44:16.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.440+0000 7f6f1a9ed740 1 -- start start 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.440+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.440+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.440+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e91f0 0x560e1d602140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101bc130 con 0x560e1d5ed1d0 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101bdd00 con 0x560e1d5e1ff0 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101bc800 con 0x560e1d5e91f0 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f15b8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e91f0 0x560e1d602140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f1638b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e91f0 0x560e1d602140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58562/0 (socket says 172.21.15.103:58562) 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 -- 172.21.15.103:0/1377576265 learned_addr learned my addr 172.21.15.103:0/1377576265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:16.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f1638b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:32814/0 (socket says 172.21.15.103:32814) 2024-08-19T01:44:16.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 -- 172.21.15.103:0/1377576265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 msgr2=0x560e1d5ea9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:16.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 -- 172.21.15.103:0/1377576265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 msgr2=0x560e1d5f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:16.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f16b8c640 1 -- 172.21.15.103:0/1377576265 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1d5df610 con 0x560e1d5e91f0 2024-08-19T01:44:16.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.441+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:16.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.442+0000 7f6f16b8c640 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e91f0 0x560e1d602140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6f08012790 tx=0x7f6f08013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5e86a0123977d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:16.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.442+0000 7f6efbfff640 1 -- 172.21.15.103:0/1377576265 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08011720 con 0x560e1d5e91f0 2024-08-19T01:44:16.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.442+0000 7f6efbfff640 1 -- 172.21.15.103:0/1377576265 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f08024070 con 0x560e1d5e91f0 2024-08-19T01:44:16.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.442+0000 7f6efbfff640 1 -- 172.21.15.103:0/1377576265 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08029bc0 con 0x560e1d5e91f0 2024-08-19T01:44:16.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/1377576265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e91f0 msgr2=0x560e1d602140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e91f0 0x560e1d602140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6f08012790 tx=0x7f6f08013000 comp rx=0 tx=0).stop 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/1377576265 shutdown_connections 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5e91f0 0x560e1d602140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5ed1d0 0x560e1d5f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/1377576265 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5ea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/1377576265 >> 172.21.15.103:0/1377576265 conn(0x560e1d5de580 msgr2=0x560e1d5f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/1377576265 shutdown_connections 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.443+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/1377576265 wait complete. 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 -- start start 2024-08-19T01:44:16.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 Processor -- start 2024-08-19T01:44:16.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5fff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e91f0 0x560e1d600490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5ed1d0 0x560e1d6009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101bd630 con 0x560e1d5e91f0 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101bced0 con 0x560e1d5e1ff0 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.444+0000 7f6f1a9ed740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ba110 con 0x560e1d5ed1d0 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f1a9ed740 10 client.0 _ll_register_callbacks cb 0x7f6f000e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f16b8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5ed1d0 0x560e1d6009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f16b8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5ed1d0 0x560e1d6009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58576/0 (socket says 172.21.15.103:58576) 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f16b8c640 1 -- 172.21.15.103:0/4262928920 learned_addr learned my addr 172.21.15.103:0/4262928920 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e91f0 0x560e1d600490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f1638b640 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f15b8a640 1 -- 172.21.15.103:0/4262928920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5ed1d0 msgr2=0x560e1d6009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5ed1d0 0x560e1d6009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f15b8a640 1 -- 172.21.15.103:0/4262928920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 msgr2=0x560e1d5fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f15b8a640 1 -- 172.21.15.103:0/4262928920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1d6666f0 con 0x560e1d5e91f0 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6f15b8a640 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e91f0 0x560e1d600490 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x560e1d5eb740 tx=0x7f6f0000df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f98c6ad0fd601c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.445+0000 7f6ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.446+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/4262928920 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0009adf0 con 0x560e1d5e91f0 2024-08-19T01:44:16.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.446+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/4262928920 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f6f000a3a90 con 0x560e1d5e91f0 2024-08-19T01:44:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.446+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f08024070 con 0x560e1d5e91f0 2024-08-19T01:44:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.446+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/4262928920 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6f0009b080 con 0x560e1d5e91f0 2024-08-19T01:44:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.446+0000 7f6f1a9ed740 10 client.17556 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.446+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560e1d6666f0 con 0x560e1d5e91f0 2024-08-19T01:44:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/4262928920 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f08024070 con 0x560e1d5e91f0 2024-08-19T01:44:16.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/4262928920 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f6f000bb0f0 con 0x560e1d5e91f0 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6f1a9ed740 10 client.17556 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6f1a9ed740 10 client.17556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6f1a9ed740 20 client.17556 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d667c30 0x560e1d66a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6f1a9ed740 4 client.17556 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e1d54f4c0 con 0x560e1d667c30 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.447+0000 7f6f1638b640 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d667c30 0x560e1d66a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.448+0000 7f6f1638b640 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d667c30 0x560e1d66a050 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25006351e4f0134f server_cookie=987b4418a77ce22e in_seq=0 out_seq=0 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.448+0000 7f6ef9ffb640 10 client.17556 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:44:16.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.448+0000 7f6ef9ffb640 1 -- 172.21.15.103:0/4262928920 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e1d54f4c0 con 0x560e1d667c30 2024-08-19T01:44:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.448+0000 7f6ef9ffb640 10 client.17556 handle_command_reply: tid=0 2024-08-19T01:44:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 client.17556 shutdown 2024-08-19T01:44:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e91f0 msgr2=0x560e1d600490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e91f0 0x560e1d600490 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x560e1d5eb740 tx=0x7f6f0000df40 comp rx=0 tx=0).stop 2024-08-19T01:44:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 shutdown_connections 2024-08-19T01:44:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x560e1d667c30 0x560e1d66a050 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560e1d5ed1d0 0x560e1d6009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e1d5e91f0 0x560e1d600490 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6ef9ffb640 0 client.17556 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:44:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 --2- 172.21.15.103:0/4262928920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560e1d5e1ff0 0x560e1d5fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 >> 172.21.15.103:0/4262928920 conn(0x560e1d5de580 msgr2=0x560e1d5dd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 shutdown_connections 2024-08-19T01:44:16.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 1 -- 172.21.15.103:0/4262928920 wait complete. 2024-08-19T01:44:16.453 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:44:16.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.449+0000 7f6f1a9ed740 20 client.17556 trim_cache size 0 max 16384 2024-08-19T01:44:16.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6efc0a2c00 msgr2=0x7f6efc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6efc0a2c00 0x7f6efc0a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f080092f0 tx=0x7f6f08009280 comp rx=0 tx=0).stop 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10108880 msgr2=0x7f6f1007b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10108880 0x7f6f1007b640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000d950 tx=0x7f6f0000de10 comp rx=0 tx=0).stop 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 shutdown_connections 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6efc0a2c00 0x7f6efc0a50c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1010df60 0x7f6f1007bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6f10108880 0x7f6f1007b640 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 --2- 172.21.15.103:0/2485609021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f10107f20 0x7f6f100825b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 >> 172.21.15.103:0/2485609021 conn(0x7f6f100778e0 msgr2=0x7f6f10079210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 shutdown_connections 2024-08-19T01:44:16.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:16.450+0000 7f6f185ef640 1 -- 172.21.15.103:0/2485609021 wait complete. 2024-08-19T01:44:17.266 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:17 smithi103 ceph-mon[18436]: pgmap v594: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 3.43k op/s 2024-08-19T01:44:17.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:44:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:17 smithi190 ceph-mon[22032]: pgmap v594: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 3.43k op/s 2024-08-19T01:44:17.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:17 smithi146 ceph-mon[22098]: pgmap v594: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 3.43k op/s 2024-08-19T01:44:17.738 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:19.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:19 smithi103 ceph-mon[18436]: pgmap v595: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 4.48k op/s 2024-08-19T01:44:19.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:19 smithi190 ceph-mon[22032]: pgmap v595: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 4.48k op/s 2024-08-19T01:44:19.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:19 smithi146 ceph-mon[22098]: pgmap v595: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 4.48k op/s 2024-08-19T01:44:19.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 -- 172.21.15.103:0/2846173111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 msgr2=0x7fca18103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 --2- 172.21.15.103:0/2846173111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 0x7fca18103f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fca00009890 tx=0x7fca0002a380 comp rx=0 tx=0).stop 2024-08-19T01:44:19.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 -- 172.21.15.103:0/2846173111 shutdown_connections 2024-08-19T01:44:19.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 --2- 172.21.15.103:0/2846173111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca18104460 0x7fca18110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 --2- 172.21.15.103:0/2846173111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 0x7fca18103f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 --2- 172.21.15.103:0/2846173111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca18109ac0 0x7fca18109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 -- 172.21.15.103:0/2846173111 >> 172.21.15.103:0/2846173111 conn(0x7fca180ff7e0 msgr2=0x7fca18101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:19.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 -- 172.21.15.103:0/2846173111 shutdown_connections 2024-08-19T01:44:19.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.835+0000 7fca1e908640 1 -- 172.21.15.103:0/2846173111 wait complete. 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.836+0000 7fca1e908640 1 Processor -- start 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.836+0000 7fca1e908640 1 -- start start 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.836+0000 7fca1e908640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 0x7fca18110900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca1e908640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca18104460 0x7fca1810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca1e908640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca18109ac0 0x7fca1810e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca1e908640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca18113d30 con 0x7fca18109ac0 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca1e908640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fca18113bb0 con 0x7fca18104460 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca1e908640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca18113eb0 con 0x7fca18103ac0 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca17fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 0x7fca18110900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca17fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 0x7fca18110900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58592/0 (socket says 172.21.15.103:58592) 2024-08-19T01:44:19.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca17fff640 1 -- 172.21.15.103:0/8465053 learned_addr learned my addr 172.21.15.103:0/8465053 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:19.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca18104460 0x7fca1810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca1cea5640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca18109ac0 0x7fca1810e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca0f7fe640 1 -- 172.21.15.103:0/8465053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 msgr2=0x7fca18110900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 0x7fca18110900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca0f7fe640 1 -- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca18109ac0 msgr2=0x7fca1810e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca18109ac0 0x7fca1810e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.837+0000 7fca0f7fe640 1 -- 172.21.15.103:0/8465053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca181aaa50 con 0x7fca18104460 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.838+0000 7fca1cea5640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca18109ac0 0x7fca1810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.838+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca18104460 0x7fca1810d9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca0002a8a0 tx=0x7fca00033b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.838+0000 7fca15ffb640 1 -- 172.21.15.103:0/8465053 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00037040 con 0x7fca18104460 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.838+0000 7fca15ffb640 1 -- 172.21.15.103:0/8465053 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca00033de0 con 0x7fca18104460 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.838+0000 7fca15ffb640 1 -- 172.21.15.103:0/8465053 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000035f0 con 0x7fca18104460 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.838+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca181aac20 con 0x7fca18104460 2024-08-19T01:44:19.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.839+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca181ab080 con 0x7fca18104460 2024-08-19T01:44:19.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.842+0000 7fca15ffb640 1 -- 172.21.15.103:0/8465053 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fca00044030 con 0x7fca18104460 2024-08-19T01:44:19.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.842+0000 7fca20d06740 1 Processor -- start 2024-08-19T01:44:19.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.843+0000 7fca15ffb640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fca040a2cd0 0x7fca040a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.843+0000 7fca15ffb640 1 -- 172.21.15.103:0/8465053 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fca000ee670 con 0x7fca18104460 2024-08-19T01:44:19.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.843+0000 7fca20d06740 1 -- start start 2024-08-19T01:44:19.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca20d06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbb13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca20d06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb1910 0x561d3bbbbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca20d06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbbe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca20d06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561d3bb16a40 con 0x561d3bbb0ff0 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca20d06740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561d3baae9d0 con 0x561d3bbb1910 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca20d06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561d3bb12a00 con 0x561d3bbbc1d0 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb1910 0x561d3bbbbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbb13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbb13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57124/0 (socket says 172.21.15.103:57124) 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 -- 172.21.15.103:0/469429636 learned_addr learned my addr 172.21.15.103:0/469429636 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca1cea5640 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbbe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca1cea5640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fca040a2cd0 0x7fca040a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 -- 172.21.15.103:0/469429636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 msgr2=0x561d3bbbe5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbbe5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 -- 172.21.15.103:0/469429636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb1910 msgr2=0x561d3bbbbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb1910 0x561d3bbbbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.844+0000 7fca17fff640 1 -- 172.21.15.103:0/469429636 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d3bab5650 con 0x561d3bbb0ff0 2024-08-19T01:44:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.845+0000 7fca17fff640 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbb13d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fca08009d00 tx=0x7fca08011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5b90938377191a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.845+0000 7fca0e7fc640 1 -- 172.21.15.103:0/469429636 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08004230 con 0x561d3bbb0ff0 2024-08-19T01:44:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.845+0000 7fca1cea5640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fca040a2cd0 0x7fca040a5190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x561d3bbae900 tx=0x7fca100094a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.845+0000 7fca0e7fc640 1 -- 172.21.15.103:0/469429636 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca080043d0 con 0x561d3bbb0ff0 2024-08-19T01:44:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.845+0000 7fca0e7fc640 1 -- 172.21.15.103:0/469429636 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08019810 con 0x561d3bbb0ff0 2024-08-19T01:44:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.846+0000 7fca20d06740 1 -- 172.21.15.103:0/469429636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 msgr2=0x561d3bbb13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.846+0000 7fca20d06740 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbb13d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fca08009d00 tx=0x7fca08011810 comp rx=0 tx=0).stop 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.846+0000 7fca20d06740 1 -- 172.21.15.103:0/469429636 shutdown_connections 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.846+0000 7fca20d06740 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbbe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.846+0000 7fca20d06740 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb1910 0x561d3bbbbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.846+0000 7fca20d06740 1 --2- 172.21.15.103:0/469429636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbb13d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.846+0000 7fca20d06740 1 -- 172.21.15.103:0/469429636 >> 172.21.15.103:0/469429636 conn(0x561d3bbad580 msgr2=0x561d3bbad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.847+0000 7fca20d06740 1 -- 172.21.15.103:0/469429636 shutdown_connections 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.848+0000 7fca20d06740 1 -- 172.21.15.103:0/469429636 wait complete. 2024-08-19T01:44:19.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.848+0000 7fca20d06740 1 -- start start 2024-08-19T01:44:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.848+0000 7fca20d06740 1 Processor -- start 2024-08-19T01:44:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca20d06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbbbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca20d06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb1910 0x561d3bbb8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca20d06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca20d06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561d3bbbeda0 con 0x561d3bbb1910 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca20d06740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561d3bad1b20 con 0x561d3bbb0ff0 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca20d06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561d3bbbef20 con 0x561d3bbbc1d0 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca20d06740 10 client.0 _ll_register_callbacks cb 0x561d3bacfbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb1910 0x561d3bbb8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca1cea5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb1910 0x561d3bbb8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57126/0 (socket says 172.21.15.103:57126) 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca0f7fe640 1 -- 172.21.15.103:0/570340960 learned_addr learned my addr 172.21.15.103:0/570340960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.849+0000 7fca17fff640 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbbbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fca0f7fe640 1 -- 172.21.15.103:0/570340960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 msgr2=0x561d3bbb9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fca0f7fe640 1 -- 172.21.15.103:0/570340960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 msgr2=0x561d3bbbbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbbbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fca0f7fe640 1 -- 172.21.15.103:0/570340960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d3b8883c0 con 0x561d3bbb1910 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb1910 0x561d3bbb8fb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fca000b7960 tx=0x7fca00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d160956ef2666658 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fc9f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fc9f77fe640 1 -- 172.21.15.103:0/570340960 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000d3020 con 0x561d3bbb1910 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fc9f77fe640 1 -- 172.21.15.103:0/570340960 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fca000bb1f0 con 0x561d3bbb1910 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.850+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca080043d0 con 0x561d3bbb1910 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fc9f77fe640 1 -- 172.21.15.103:0/570340960 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fca000bf340 con 0x561d3bbb1910 2024-08-19T01:44:19.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 10 client.17568 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561d3bbca8b0 con 0x561d3bbb1910 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fc9f77fe640 1 -- 172.21.15.103:0/570340960 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca080043d0 con 0x561d3bbb1910 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fc9f77fe640 1 -- 172.21.15.103:0/570340960 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fca0003a030 con 0x561d3bbb1910 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 10 client.17568 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 10 client.17568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 20 client.17568 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bbcbee0 0x561d3bbce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 4 client.17568 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.851+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d3bb23fb0 con 0x561d3bbcbee0 2024-08-19T01:44:19.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.852+0000 7fca17fff640 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bbcbee0 0x561d3bbce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.853+0000 7fca17fff640 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bbcbee0 0x561d3bbce340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d110ba747e42ea1d server_cookie=245287ff3b737544 in_seq=0 out_seq=0 2024-08-19T01:44:19.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.854+0000 7fc9f77fe640 10 client.17568 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.854+0000 7fc9f77fe640 1 -- 172.21.15.103:0/570340960 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d3bb23fb0 con 0x561d3bbcbee0 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.854+0000 7fc9f77fe640 10 client.17568 handle_command_reply: tid=0 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.854+0000 7fca20d06740 1 client.17568 shutdown 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb1910 msgr2=0x561d3bbb8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb1910 0x561d3bbb8fb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fca000b7960 tx=0x7fca00000bc0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 shutdown_connections 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bbcbee0 0x561d3bbce340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb1910 0x561d3bbb8fb0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 --2- 172.21.15.103:0/570340960 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbbbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 >> 172.21.15.103:0/570340960 conn(0x561d3bbad580 msgr2=0x561d3bbb32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:19.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fc9f77fe640 0 client.17568 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:44:19.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 shutdown_connections 2024-08-19T01:44:19.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 1 -- 172.21.15.103:0/570340960 wait complete. 2024-08-19T01:44:19.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.855+0000 7fca20d06740 20 client.17568 trim_cache size 0 max 16384 2024-08-19T01:44:19.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.878+0000 7fca20d06740 1 Processor -- start 2024-08-19T01:44:19.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.878+0000 7fca20d06740 1 -- start start 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.878+0000 7fca20d06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbd3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca20d06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbd4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca20d06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca17fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbd3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca17fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbd3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:32876/0 (socket says 172.21.15.103:32876) 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbd4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca17fff640 1 -- 172.21.15.103:0/2241764192 learned_addr learned my addr 172.21.15.103:0/2241764192 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca181b3690 con 0x561d3bbbc1d0 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fca181b5260 con 0x561d3bbb0ff0 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca1cea5640 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca181b3d60 con 0x561d3bbb1910 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca0f7fe640 1 -- 172.21.15.103:0/2241764192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 msgr2=0x561d3bbd3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbd3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca0f7fe640 1 -- 172.21.15.103:0/2241764192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbbc1d0 msgr2=0x561d3bbb6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.879+0000 7fca0f7fe640 1 -- 172.21.15.103:0/2241764192 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d3bbad3b0 con 0x561d3bbb1910 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.880+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbd4150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca000baf20 tx=0x7fca000b94a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9fce89fee288e57 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.880+0000 7fca0dffb640 1 -- 172.21.15.103:0/2241764192 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000d3030 con 0x561d3bbb1910 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.880+0000 7fca0dffb640 1 -- 172.21.15.103:0/2241764192 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca00030780 con 0x561d3bbb1910 2024-08-19T01:44:19.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.880+0000 7fca0dffb640 1 -- 172.21.15.103:0/2241764192 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000bb450 con 0x561d3bbb1910 2024-08-19T01:44:19.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 msgr2=0x561d3bbd4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbd4150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca000baf20 tx=0x7fca000b94a0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 shutdown_connections 2024-08-19T01:44:19.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbd4150 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 --2- 172.21.15.103:0/2241764192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbb0ff0 0x561d3bbd3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 >> 172.21.15.103:0/2241764192 conn(0x561d3bbad580 msgr2=0x561d3bbb80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 shutdown_connections 2024-08-19T01:44:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 -- 172.21.15.103:0/2241764192 wait complete. 2024-08-19T01:44:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.881+0000 7fca20d06740 1 -- start start 2024-08-19T01:44:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 1 Processor -- start 2024-08-19T01:44:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbc4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbc4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca181b4b90 con 0x561d3bbb0ff0 2024-08-19T01:44:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fca181b4430 con 0x561d3bbbc1d0 2024-08-19T01:44:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca181b1670 con 0x561d3bbb1910 2024-08-19T01:44:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca20d06740 10 client.0 _ll_register_callbacks cb 0x7fca000b7320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbc4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca1cea5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbc4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58642/0 (socket says 172.21.15.103:58642) 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.882+0000 7fca0f7fe640 1 -- 172.21.15.103:0/2775002155 learned_addr learned my addr 172.21.15.103:0/2775002155 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fca0f7fe640 1 -- 172.21.15.103:0/2775002155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbbc1d0 msgr2=0x561d3bbb59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fca0f7fe640 1 -- 172.21.15.103:0/2775002155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 msgr2=0x561d3bbc4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbc4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fca0f7fe640 1 -- 172.21.15.103:0/2775002155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d3bbb61e0 con 0x561d3bbb1910 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fca0f7fe640 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbc4980 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca000b9240 tx=0x7fca000db040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4b423de6a73d92 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fc9f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fc9f77fe640 1 -- 172.21.15.103:0/2775002155 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000d3070 con 0x561d3bbb1910 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.883+0000 7fc9f77fe640 1 -- 172.21.15.103:0/2775002155 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fca000d9b40 con 0x561d3bbb1910 2024-08-19T01:44:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca00030780 con 0x561d3bbb1910 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fc9f77fe640 1 -- 172.21.15.103:0/2775002155 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fca000a0070 con 0x561d3bbb1910 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 10 client.26764 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561d3bbb61e0 con 0x561d3bbb1910 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fc9f77fe640 1 -- 172.21.15.103:0/2775002155 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca00030780 con 0x561d3bbb1910 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fc9f77fe640 1 -- 172.21.15.103:0/2775002155 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fca0003a070 con 0x561d3bbb1910 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 10 client.26764 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 10 client.26764 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 20 client.26764 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bc36be0 0x561d3bc38fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 4 client.26764 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.884+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d3bb1e4c0 con 0x561d3bc36be0 2024-08-19T01:44:19.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.885+0000 7fca17fff640 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bc36be0 0x561d3bc38fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.885+0000 7fc9f77fe640 10 client.26764 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.885+0000 7fca17fff640 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bc36be0 0x561d3bc38fd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a7d00d5bf902d18 server_cookie=50c74d085608f76a in_seq=0 out_seq=0 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fc9f77fe640 1 -- 172.21.15.103:0/2775002155 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d3bb1e4c0 con 0x561d3bc36be0 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fc9f77fe640 10 client.26764 handle_command_reply: tid=0 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 client.26764 shutdown 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 msgr2=0x561d3bbc4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbc4980 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca000b9240 tx=0x7fca000db040 comp rx=0 tx=0).stop 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 shutdown_connections 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x561d3bc36be0 0x561d3bc38fd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fc9f77fe640 0 client.26764 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561d3bbbc1d0 0x561d3bbb59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d3bbb1910 0x561d3bbc4980 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 --2- 172.21.15.103:0/2775002155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561d3bbb0ff0 0x561d3bbc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 >> 172.21.15.103:0/2775002155 conn(0x561d3bbad580 msgr2=0x561d3bbc6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.886+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 shutdown_connections 2024-08-19T01:44:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.887+0000 7fca20d06740 1 -- 172.21.15.103:0/2775002155 wait complete. 2024-08-19T01:44:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.887+0000 7fca20d06740 20 client.26764 trim_cache size 0 max 16384 2024-08-19T01:44:19.890 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:44:19.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.888+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fca040a2cd0 msgr2=0x7fca040a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.888+0000 7fca1e908640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fca040a2cd0 0x7fca040a5190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x561d3bbae900 tx=0x7fca100094a0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.888+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca18104460 msgr2=0x7fca1810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:19.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.888+0000 7fca1e908640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca18104460 0x7fca1810d9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca0002a8a0 tx=0x7fca00033b30 comp rx=0 tx=0).stop 2024-08-19T01:44:19.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 shutdown_connections 2024-08-19T01:44:19.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fca040a2cd0 0x7fca040a5190 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca18109ac0 0x7fca1810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca18104460 0x7fca1810d9b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 --2- 172.21.15.103:0/8465053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca18103ac0 0x7fca18110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:19.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 >> 172.21.15.103:0/8465053 conn(0x7fca180ff7e0 msgr2=0x7fca18100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:19.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 shutdown_connections 2024-08-19T01:44:19.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:19.889+0000 7fca1e908640 1 -- 172.21.15.103:0/8465053 wait complete. 2024-08-19T01:44:20.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:44:21.217 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:21.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:21 smithi103 ceph-mon[18436]: pgmap v596: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 5.49k op/s 2024-08-19T01:44:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:21 smithi190 ceph-mon[22032]: pgmap v596: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 5.49k op/s 2024-08-19T01:44:21.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:21 smithi146 ceph-mon[22098]: pgmap v596: 97 pgs: 97 active+clean; 13 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 MiB/s wr, 5.49k op/s 2024-08-19T01:44:21.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 -- 172.21.15.103:0/3621959968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4104460 msgr2=0x7fe2d4110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 --2- 172.21.15.103:0/3621959968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4104460 0x7fe2d4110990 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009c10 tx=0x7fe2cc02a6f0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 -- 172.21.15.103:0/3621959968 shutdown_connections 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 --2- 172.21.15.103:0/3621959968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4104460 0x7fe2d4110990 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 --2- 172.21.15.103:0/3621959968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d4103ac0 0x7fe2d4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 --2- 172.21.15.103:0/3621959968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4109ac0 0x7fe2d4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 -- 172.21.15.103:0/3621959968 >> 172.21.15.103:0/3621959968 conn(0x7fe2d40ff7e0 msgr2=0x7fe2d4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 -- 172.21.15.103:0/3621959968 shutdown_connections 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.825+0000 7fe2d907f640 1 -- 172.21.15.103:0/3621959968 wait complete. 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.826+0000 7fe2d907f640 1 Processor -- start 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d907f640 1 -- start start 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d907f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4103ac0 0x7fe2d4076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d907f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d4104460 0x7fe2d4076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d907f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4109ac0 0x7fe2d4076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d907f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4113d70 con 0x7fe2d4103ac0 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d907f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4113bf0 con 0x7fe2d4104460 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d907f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4113ef0 con 0x7fe2d4109ac0 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4103ac0 0x7fe2d4076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4103ac0 0x7fe2d4076430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57168/0 (socket says 172.21.15.103:57168) 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.827+0000 7fe2d2d9d640 1 -- 172.21.15.103:0/81088733 learned_addr learned my addr 172.21.15.103:0/81088733 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d359e640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4109ac0 0x7fe2d4076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d359e640 1 -- 172.21.15.103:0/81088733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d4104460 msgr2=0x7fe2d4076970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d259c640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d4104460 0x7fe2d4076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d359e640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d4104460 0x7fe2d4076970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d359e640 1 -- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4103ac0 msgr2=0x7fe2d4076430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d359e640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4103ac0 0x7fe2d4076430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d359e640 1 -- 172.21.15.103:0/81088733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d41aa850 con 0x7fe2d4109ac0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4103ac0 0x7fe2d4076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.828+0000 7fe2d359e640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4109ac0 0x7fe2d4076eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc02ace0 tx=0x7fe2cc003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.829+0000 7fe2bbfff640 1 -- 172.21.15.103:0/81088733 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc041020 con 0x7fe2d4109ac0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.829+0000 7fe2bbfff640 1 -- 172.21.15.103:0/81088733 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2cc037040 con 0x7fe2d4109ac0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.829+0000 7fe2bbfff640 1 -- 172.21.15.103:0/81088733 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc036070 con 0x7fe2d4109ac0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.829+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d41aaae0 con 0x7fe2d4109ac0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.829+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d41aaf40 con 0x7fe2d4109ac0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.832+0000 7fe2db47d740 1 Processor -- start 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.832+0000 7fe2bbfff640 1 -- 172.21.15.103:0/81088733 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe2cc044020 con 0x7fe2d4109ac0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2db47d740 1 -- start start 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x561950223270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619502237b0 0x56195023eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56195023f1e0 0x5619502415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2db47d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561950188a40 con 0x56195023f1e0 2024-08-19T01:44:21.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2db47d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5619501209d0 con 0x5619502237b0 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2db47d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561950184a00 con 0x561950222e90 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x561950223270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x561950223270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58674/0 (socket says 172.21.15.103:58674) 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2d259c640 1 -- 172.21.15.103:0/3830179010 learned_addr learned my addr 172.21.15.103:0/3830179010 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619502237b0 0x56195023eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2bbfff640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2b80 0x7fe2bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.833+0000 7fe2d259c640 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56195023f1e0 0x5619502415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d2d9d640 1 -- 172.21.15.103:0/3830179010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 msgr2=0x561950223270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x561950223270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d2d9d640 1 -- 172.21.15.103:0/3830179010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56195023f1e0 msgr2=0x5619502415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56195023f1e0 0x5619502415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d2d9d640 1 -- 172.21.15.103:0/3830179010 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561950127650 con 0x5619502237b0 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d359e640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2b80 0x7fe2bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d259c640 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56195023f1e0 0x5619502415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619502237b0 0x56195023eca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400f3f0 tx=0x7fe2c400f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769b9c9f6a22a683 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2bbfff640 1 -- 172.21.15.103:0/81088733 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe2cc0ef8c0 con 0x7fe2d4109ac0 2024-08-19T01:44:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2b97fa640 1 -- 172.21.15.103:0/3830179010 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4005220 con 0x5619502237b0 2024-08-19T01:44:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2b97fa640 1 -- 172.21.15.103:0/3830179010 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c4004c50 con 0x5619502237b0 2024-08-19T01:44:21.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.834+0000 7fe2b97fa640 1 -- 172.21.15.103:0/3830179010 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4010c20 con 0x5619502237b0 2024-08-19T01:44:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.835+0000 7fe2d359e640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2b80 0x7fe2bc0a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0c4890 tx=0x7fe2cc0e6ce0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.835+0000 7fe2db47d740 1 -- 172.21.15.103:0/3830179010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619502237b0 msgr2=0x56195023eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.835+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619502237b0 0x56195023eca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400f3f0 tx=0x7fe2c400f8b0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.836+0000 7fe2db47d740 1 -- 172.21.15.103:0/3830179010 shutdown_connections 2024-08-19T01:44:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.836+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56195023f1e0 0x5619502415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.836+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5619502237b0 0x56195023eca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.836+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3830179010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x561950223270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.836+0000 7fe2db47d740 1 -- 172.21.15.103:0/3830179010 >> 172.21.15.103:0/3830179010 conn(0x56195021f580 msgr2=0x56195021f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.836+0000 7fe2db47d740 1 -- 172.21.15.103:0/3830179010 shutdown_connections 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.836+0000 7fe2db47d740 1 -- 172.21.15.103:0/3830179010 wait complete. 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.837+0000 7fe2db47d740 1 -- start start 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.837+0000 7fe2db47d740 1 Processor -- start 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x56195023c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x56195023c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x56195023cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x56195023c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x56195023c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58678/0 (socket says 172.21.15.103:58678) 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2d259c640 1 -- 172.21.15.103:0/3396510746 learned_addr learned my addr 172.21.15.103:0/3396510746 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561950241db0 con 0x5619502237b0 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561950143b20 con 0x56195023f1e0 2024-08-19T01:44:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561950241f30 con 0x561950222e90 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2d259c640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x56195023cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2db47d740 10 client.0 _ll_register_callbacks cb 0x561950141bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.838+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x56195023c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2d259c640 1 -- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 msgr2=0x56195023cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2d259c640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x56195023cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2d259c640 1 -- 172.21.15.103:0/3396510746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 msgr2=0x56195023c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2d259c640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x56195023c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2d259c640 1 -- 172.21.15.103:0/3396510746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56194fefa3c0 con 0x561950222e90 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x56195023c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2d259c640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x56195023c0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009830 tx=0x7fe2c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7b66a55b3282c9e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/3396510746 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8005850 con 0x561950222e90 2024-08-19T01:44:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/3396510746 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe2c80140d0 con 0x561950222e90 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c4004c50 con 0x561950222e90 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/3396510746 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe2c8005020 con 0x561950222e90 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2db47d740 10 client.26773 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.839+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56195024da30 con 0x561950222e90 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/3396510746 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c4004c50 con 0x561950222e90 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/3396510746 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fe2c801f030 con 0x561950222e90 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2db47d740 10 client.26773 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2db47d740 10 client.26773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2db47d740 20 client.26773 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56195024eef0 0x561950251350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2db47d740 4 client.26773 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.840+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561950195fb0 con 0x56195024eef0 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.841+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56195024eef0 0x561950251350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.841+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56195024eef0 0x561950251350 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2891856f31590bff server_cookie=ace5b0420399779e in_seq=0 out_seq=0 2024-08-19T01:44:21.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.841+0000 7fe2b6ff5640 10 client.26773 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/3396510746 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561950195fb0 con 0x56195024eef0 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2b6ff5640 10 client.26773 handle_command_reply: tid=0 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 client.26773 shutdown 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 msgr2=0x56195023c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x56195023c0c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009830 tx=0x7fe2c8004730 comp rx=0 tx=0).stop 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 shutdown_connections 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56195024eef0 0x561950251350 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x56195023cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x56195023c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 --2- 172.21.15.103:0/3396510746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x56195023c0c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 >> 172.21.15.103:0/3396510746 conn(0x56195021f580 msgr2=0x5619502252e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2b6ff5640 0 client.26773 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 shutdown_connections 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 1 -- 172.21.15.103:0/3396510746 wait complete. 2024-08-19T01:44:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.842+0000 7fe2db47d740 20 client.26773 trim_cache size 0 max 16384 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 Processor -- start 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 -- start start 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561950222e90 0x56195023e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619502237b0 0x561950250f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x561950256c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41ad1a0 con 0x561950222e90 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4073df0 con 0x56195023f1e0 2024-08-19T01:44:21.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.865+0000 7fe2db47d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41acad0 con 0x5619502237b0 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619502237b0 0x561950250f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x561950256c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x561950256c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39614/0 (socket says 172.21.15.103:39614) 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d2d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619502237b0 0x561950250f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:38080/0 (socket says 172.21.15.103:38080) 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 -- 172.21.15.103:0/2654596517 learned_addr learned my addr 172.21.15.103:0/2654596517 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561950222e90 0x56195023e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 -- 172.21.15.103:0/2654596517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619502237b0 msgr2=0x561950250f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619502237b0 0x561950250f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 -- 172.21.15.103:0/2654596517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 msgr2=0x561950256c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x561950256c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 -- 172.21.15.103:0/2654596517 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195023a610 con 0x561950222e90 2024-08-19T01:44:21.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.866+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561950222e90 0x56195023e820 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009940 tx=0x7fe2c8014d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b79a2dd8fe3b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:21.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.867+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2654596517 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c802f030 con 0x561950222e90 2024-08-19T01:44:21.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.867+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2654596517 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c8004e60 con 0x561950222e90 2024-08-19T01:44:21.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.867+0000 7fe2b8ff9640 1 -- 172.21.15.103:0/2654596517 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80140d0 con 0x561950222e90 2024-08-19T01:44:21.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 -- 172.21.15.103:0/2654596517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561950222e90 msgr2=0x56195023e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561950222e90 0x56195023e820 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009940 tx=0x7fe2c8014d80 comp rx=0 tx=0).stop 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 -- 172.21.15.103:0/2654596517 shutdown_connections 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x561950256c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5619502237b0 0x561950250f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2654596517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561950222e90 0x56195023e820 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 -- 172.21.15.103:0/2654596517 >> 172.21.15.103:0/2654596517 conn(0x56195021f580 msgr2=0x56195023b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 -- 172.21.15.103:0/2654596517 shutdown_connections 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 -- 172.21.15.103:0/2654596517 wait complete. 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.868+0000 7fe2db47d740 1 -- start start 2024-08-19T01:44:21.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 1 Processor -- start 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x5619502421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x561950242720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x5619502b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41abca0 con 0x5619502237b0 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41ab5d0 con 0x56195023f1e0 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41ac400 con 0x561950222e90 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2db47d740 10 client.0 _ll_register_callbacks cb 0x7fe2c8009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x5619502421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x5619502421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:38086/0 (socket says 172.21.15.103:38086) 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2d259c640 1 -- 172.21.15.103:0/2178380268 learned_addr learned my addr 172.21.15.103:0/2178380268 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x5619502b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.869+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x561950242720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2d259c640 1 -- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 msgr2=0x5619502b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x5619502b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2d259c640 1 -- 172.21.15.103:0/2178380268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 msgr2=0x561950242720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:21.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x561950242720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2d259c640 1 -- 172.21.15.103:0/2178380268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561950242fb0 con 0x561950222e90 2024-08-19T01:44:21.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2d259c640 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x5619502421e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8012590 tx=0x7fe2c8014b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a979885c2e112da server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/2178380268 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c802f070 con 0x561950222e90 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/2178380268 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe2c8004720 con 0x561950222e90 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.870+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619501904c0 con 0x561950222e90 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/2178380268 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe2c8004920 con 0x561950222e90 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2db47d740 10 client.26779 fetch_fsmap learned FSMap version 38 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5619502b91f0 con 0x561950222e90 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/2178380268 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c8005670 con 0x561950222e90 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/2178380268 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fe2c801f050 con 0x561950222e90 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2db47d740 10 client.26779 fetch_fsmap finished waiting for FSMap version 38 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2db47d740 10 client.26779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2db47d740 20 client.26779 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.871+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5619502b97a0 0x5619502bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:21.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.872+0000 7fe2db47d740 4 client.26779 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:44:21.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.872+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619501904c0 con 0x5619502b97a0 2024-08-19T01:44:21.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.872+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5619502b97a0 0x5619502bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:21.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.872+0000 7fe2d2d9d640 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5619502b97a0 0x5619502bbb90 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d098ed82918a68 server_cookie=a5cfd162463a6399 in_seq=0 out_seq=0 2024-08-19T01:44:21.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.872+0000 7fe2b6ff5640 10 client.26779 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:21.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.873+0000 7fe2b6ff5640 1 -- 172.21.15.103:0/2178380268 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619501904c0 con 0x5619502b97a0 2024-08-19T01:44:21.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.873+0000 7fe2b6ff5640 10 client.26779 handle_command_reply: tid=0 2024-08-19T01:44:21.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.873+0000 7fe2db47d740 1 client.26779 shutdown 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 msgr2=0x5619502421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x5619502421e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8012590 tx=0x7fe2c8014b00 comp rx=0 tx=0).stop 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 shutdown_connections 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x5619502b97a0 0x5619502bbb90 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56195023f1e0 0x5619502b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5619502237b0 0x561950242720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2b6ff5640 0 client.26779 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:21.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 --2- 172.21.15.103:0/2178380268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561950222e90 0x5619502421e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 >> 172.21.15.103:0/2178380268 conn(0x56195021f580 msgr2=0x5619502249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:21.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 shutdown_connections 2024-08-19T01:44:21.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.874+0000 7fe2db47d740 1 -- 172.21.15.103:0/2178380268 wait complete. 2024-08-19T01:44:21.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2db47d740 20 client.26779 trim_cache size 0 max 16384 2024-08-19T01:44:21.877 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:44:21.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2b80 msgr2=0x7fe2bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2b80 0x7fe2bc0a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0c4890 tx=0x7fe2cc0e6ce0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4109ac0 msgr2=0x7fe2d4076eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:21.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4109ac0 0x7fe2d4076eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc02ace0 tx=0x7fe2cc003f10 comp rx=0 tx=0).stop 2024-08-19T01:44:21.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 shutdown_connections 2024-08-19T01:44:21.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe2bc0a2b80 0x7fe2bc0a5040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2d4109ac0 0x7fe2d4076eb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2d4104460 0x7fe2d4076970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 --2- 172.21.15.103:0/81088733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d4103ac0 0x7fe2d4076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:21.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.875+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 >> 172.21.15.103:0/81088733 conn(0x7fe2d40ff7e0 msgr2=0x7fe2d4100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:21.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.876+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 shutdown_connections 2024-08-19T01:44:21.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:21.876+0000 7fe2d907f640 1 -- 172.21.15.103:0/81088733 wait complete. 2024-08-19T01:44:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:23 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:44:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:23 smithi190 ceph-mon[22032]: pgmap v597: 97 pgs: 97 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 7.24k op/s 2024-08-19T01:44:23.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:23 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:44:23.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:23 smithi146 ceph-mon[22098]: pgmap v597: 97 pgs: 97 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 7.24k op/s 2024-08-19T01:44:23.553 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:23 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:44:23.553 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:23 smithi103 ceph-mon[18436]: pgmap v597: 97 pgs: 97 active+clean; 13 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 7.24k op/s 2024-08-19T01:44:24.139 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-19T01:44:24.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-19T01:44:24.545 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:25.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 -- 172.21.15.103:0/2139007637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881067b0 msgr2=0x7f3688106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 --2- 172.21.15.103:0/2139007637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881067b0 0x7f3688106b90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3678009930 tx=0x7f367802a420 comp rx=0 tx=0).stop 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 -- 172.21.15.103:0/2139007637 shutdown_connections 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 --2- 172.21.15.103:0/2139007637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3688101150 0x7f368810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 --2- 172.21.15.103:0/2139007637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36881007b0 0x7f3688100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 --2- 172.21.15.103:0/2139007637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881067b0 0x7f3688106b90 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 -- 172.21.15.103:0/2139007637 >> 172.21.15.103:0/2139007637 conn(0x7f36880fc4d0 msgr2=0x7f36880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 -- 172.21.15.103:0/2139007637 shutdown_connections 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.124+0000 7f368da67640 1 -- 172.21.15.103:0/2139007637 wait complete. 2024-08-19T01:44:25.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.125+0000 7f368da67640 1 Processor -- start 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.125+0000 7f368da67640 1 -- start start 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f368da67640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881007b0 0x7f3688196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f368da67640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3688101150 0x7f3688197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f368da67640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36881067b0 0x7f368819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f368da67640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3688110a20 con 0x7f36881007b0 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f36877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881007b0 0x7f3688196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f36877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881007b0 0x7f3688196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32876/0 (socket says 172.21.15.103:32876) 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f368da67640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f36881108a0 con 0x7f36881067b0 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f368da67640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3688110ba0 con 0x7f3688101150 2024-08-19T01:44:25.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f36877fe640 1 -- 172.21.15.103:0/953019789 learned_addr learned my addr 172.21.15.103:0/953019789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3688101150 0x7f3688197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.126+0000 7f3687fff640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36881067b0 0x7f368819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f36877fe640 1 -- 172.21.15.103:0/953019789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3688101150 msgr2=0x7f3688197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f36877fe640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3688101150 0x7f3688197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f36877fe640 1 -- 172.21.15.103:0/953019789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36881067b0 msgr2=0x7f368819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f36877fe640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36881067b0 0x7f368819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f36877fe640 1 -- 172.21.15.103:0/953019789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368819bdd0 con 0x7f36881007b0 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f36877fe640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881007b0 0x7f3688196e20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3678009900 tx=0x7f3678038a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f3684ff9640 1 -- 172.21.15.103:0/953019789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678035070 con 0x7f36881007b0 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f3684ff9640 1 -- 172.21.15.103:0/953019789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36780318c0 con 0x7f36881007b0 2024-08-19T01:44:25.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368819c060 con 0x7f36881007b0 2024-08-19T01:44:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f3684ff9640 1 -- 172.21.15.103:0/953019789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678041620 con 0x7f36881007b0 2024-08-19T01:44:25.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.127+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368819c4d0 con 0x7f36881007b0 2024-08-19T01:44:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.130+0000 7f368fe65740 1 Processor -- start 2024-08-19T01:44:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.130+0000 7f368fe65740 1 -- start start 2024-08-19T01:44:25.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3684ff9640 1 -- 172.21.15.103:0/953019789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f36780035f0 con 0x7f36881007b0 2024-08-19T01:44:25.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f368fe65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f368fe65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae25d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f368fe65740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae26880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f368fe65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b8faddfe40 con 0x55b8fae2b440 2024-08-19T01:44:25.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f368fe65740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b8fad817f0 con 0x55b8fae26400 2024-08-19T01:44:25.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f368fe65740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b8fad28fb0 con 0x55b8fae2bd60 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32890/0 (socket says 172.21.15.103:32890) 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 -- 172.21.15.103:0/2759312798 learned_addr learned my addr 172.21.15.103:0/2759312798 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f36877fe640 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae25d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3686ffd640 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae26880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 -- 172.21.15.103:0/2759312798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 msgr2=0x55b8fae25d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae25d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 -- 172.21.15.103:0/2759312798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 msgr2=0x55b8fae26880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae26880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.131+0000 7f3687fff640 1 -- 172.21.15.103:0/2759312798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8faaf36a0 con 0x55b8fae2b440 2024-08-19T01:44:25.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.132+0000 7f3687fff640 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2b820 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3680009f90 tx=0x7f3680010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1854ad9a5d53295 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:25.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.132+0000 7f3684ff9640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f36700a2c00 0x7f36700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.132+0000 7f366dffb640 1 -- 172.21.15.103:0/2759312798 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680011700 con 0x55b8fae2b440 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.132+0000 7f36877fe640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f36700a2c00 0x7f36700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.132+0000 7f366dffb640 1 -- 172.21.15.103:0/2759312798 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3680011d50 con 0x55b8fae2b440 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.132+0000 7f366dffb640 1 -- 172.21.15.103:0/2759312798 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680019d10 con 0x55b8fae2b440 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.132+0000 7f3684ff9640 1 -- 172.21.15.103:0/953019789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f36780ef530 con 0x7f36881007b0 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 -- 172.21.15.103:0/2759312798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 msgr2=0x55b8fae2b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2b820 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3680009f90 tx=0x7f3680010860 comp rx=0 tx=0).stop 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f36877fe640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f36700a2c00 0x7f36700a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55b8fae19690 tx=0x7f367800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 -- 172.21.15.103:0/2759312798 shutdown_connections 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae26880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae25d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 --2- 172.21.15.103:0/2759312798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2b820 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 -- 172.21.15.103:0/2759312798 >> 172.21.15.103:0/2759312798 conn(0x55b8fae18240 msgr2=0x55b8fae18620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:25.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 -- 172.21.15.103:0/2759312798 shutdown_connections 2024-08-19T01:44:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.133+0000 7f368fe65740 1 -- 172.21.15.103:0/2759312798 wait complete. 2024-08-19T01:44:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.134+0000 7f368fe65740 1 -- start start 2024-08-19T01:44:25.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.134+0000 7f368fe65740 1 Processor -- start 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.134+0000 7f368fe65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae2f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.134+0000 7f368fe65740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2b440 0x55b8fae2f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f368fe65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2bd60 0x55b8fae2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2b440 0x55b8fae2f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2b440 0x55b8fae2f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39656/0 (socket says 172.21.15.103:39656) 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f3686ffd640 1 -- 172.21.15.103:0/1079867898 learned_addr learned my addr 172.21.15.103:0/1079867898 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b8fad3ccb0 con 0x55b8fae2bd60 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b8fad7d7b0 con 0x55b8fae2b440 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b8fae32820 con 0x55b8fae26400 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.135+0000 7f368fe65740 10 client.0 _ll_register_callbacks cb 0x55b8faa90ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:25.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.136+0000 7f3687fff640 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae2f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.136+0000 7f3687fff640 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2bd60 0x55b8fae2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.137+0000 7f3687fff640 1 -- 172.21.15.103:0/1079867898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2b440 msgr2=0x55b8fae2f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.137+0000 7f3687fff640 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2b440 0x55b8fae2f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.137+0000 7f3687fff640 1 -- 172.21.15.103:0/1079867898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2bd60 msgr2=0x55b8fae2fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.137+0000 7f3687fff640 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2bd60 0x55b8fae2fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.137+0000 7f3687fff640 1 -- 172.21.15.103:0/1079867898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8fad3bbc0 con 0x55b8fae26400 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.137+0000 7f3687fff640 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae2f2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ee0 tx=0x7f3680019870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b515b87e75b5a93 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.137+0000 7f3666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.138+0000 7f3666ffd640 1 -- 172.21.15.103:0/1079867898 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368002d060 con 0x55b8fae26400 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.138+0000 7f3666ffd640 1 -- 172.21.15.103:0/1079867898 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3680019a10 con 0x55b8fae26400 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.138+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3680011d50 con 0x55b8fae26400 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.138+0000 7f3666ffd640 1 -- 172.21.15.103:0/1079867898 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3680004e50 con 0x55b8fae26400 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.138+0000 7f368fe65740 10 client.26791 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:25.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.138+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8fae3e820 con 0x55b8fae26400 2024-08-19T01:44:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.139+0000 7f3666ffd640 1 -- 172.21.15.103:0/1079867898 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3680011d50 con 0x55b8fae26400 2024-08-19T01:44:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.139+0000 7f3666ffd640 1 -- 172.21.15.103:0/1079867898 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3680025030 con 0x55b8fae26400 2024-08-19T01:44:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.139+0000 7f368fe65740 10 client.26791 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.139+0000 7f368fe65740 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.139+0000 7f368fe65740 20 client.26791 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:25.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.140+0000 7f368fe65740 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8fae3fbd0 0x55b8fae41fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.140+0000 7f368fe65740 4 client.26791 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.140+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8fad8ed60 con 0x55b8fae3fbd0 2024-08-19T01:44:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.140+0000 7f3686ffd640 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8fae3fbd0 0x55b8fae41fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.140+0000 7f3686ffd640 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8fae3fbd0 0x55b8fae41fc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b981f0a21db581f3 server_cookie=db0cd38261f45c67 in_seq=0 out_seq=0 2024-08-19T01:44:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.141+0000 7f3666ffd640 10 client.26791 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:25.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.141+0000 7f3666ffd640 1 -- 172.21.15.103:0/1079867898 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8fad8ed60 con 0x55b8fae3fbd0 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.141+0000 7f3666ffd640 10 client.26791 handle_command_reply: tid=0 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 client.26791 shutdown 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 msgr2=0x55b8fae2f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae2f2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ee0 tx=0x7f3680019870 comp rx=0 tx=0).stop 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 shutdown_connections 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8fae3fbd0 0x55b8fae41fc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2bd60 0x55b8fae2fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2b440 0x55b8fae2f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 --2- 172.21.15.103:0/1079867898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae2f2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f3666ffd640 0 client.26791 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:25.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 >> 172.21.15.103:0/1079867898 conn(0x55b8fae18240 msgr2=0x55b8fae178f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:25.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 shutdown_connections 2024-08-19T01:44:25.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 1 -- 172.21.15.103:0/1079867898 wait complete. 2024-08-19T01:44:25.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.142+0000 7f368fe65740 20 client.26791 trim_cache size 0 max 16384 2024-08-19T01:44:25.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.173+0000 7f368fe65740 1 Processor -- start 2024-08-19T01:44:25.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.177+0000 7f368fe65740 1 -- start start 2024-08-19T01:44:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f368fe65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae25d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f368fe65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f368fe65740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2bd60 0x55b8fae47610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f368fe65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36881aaf50 con 0x55b8fae2b440 2024-08-19T01:44:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f368fe65740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f36881acb50 con 0x55b8fae2bd60 2024-08-19T01:44:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f368fe65740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ab620 con 0x55b8fae26400 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae25d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae25d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:38152/0 (socket says 172.21.15.103:38152) 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 -- 172.21.15.103:0/897409293 learned_addr learned my addr 172.21.15.103:0/897409293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2bd60 0x55b8fae47610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3686ffd640 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 -- 172.21.15.103:0/897409293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2bd60 msgr2=0x55b8fae47610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2bd60 0x55b8fae47610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 -- 172.21.15.103:0/897409293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 msgr2=0x55b8fae2fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.178+0000 7f3687fff640 1 -- 172.21.15.103:0/897409293 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8fae2e2a0 con 0x55b8fae26400 2024-08-19T01:44:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.179+0000 7f3686ffd640 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.179+0000 7f3687fff640 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae25d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3680009720 tx=0x7f3680013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4939898d9c067d1c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.179+0000 7f366d7fa640 1 -- 172.21.15.103:0/897409293 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680038040 con 0x55b8fae26400 2024-08-19T01:44:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.179+0000 7f366d7fa640 1 -- 172.21.15.103:0/897409293 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3680020040 con 0x55b8fae26400 2024-08-19T01:44:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.179+0000 7f366d7fa640 1 -- 172.21.15.103:0/897409293 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680011ba0 con 0x55b8fae26400 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.180+0000 7f368fe65740 1 -- 172.21.15.103:0/897409293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 msgr2=0x55b8fae25d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.180+0000 7f368fe65740 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae25d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3680009720 tx=0x7f3680013040 comp rx=0 tx=0).stop 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.180+0000 7f368fe65740 1 -- 172.21.15.103:0/897409293 shutdown_connections 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae2bd60 0x55b8fae47610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae2fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 --2- 172.21.15.103:0/897409293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae26400 0x55b8fae25d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 -- 172.21.15.103:0/897409293 >> 172.21.15.103:0/897409293 conn(0x55b8fae18240 msgr2=0x55b8fae178f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 -- 172.21.15.103:0/897409293 shutdown_connections 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 -- 172.21.15.103:0/897409293 wait complete. 2024-08-19T01:44:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 -- start start 2024-08-19T01:44:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.181+0000 7f368fe65740 1 Processor -- start 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f368fe65740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae38390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f368fe65740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae47140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f368fe65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae47880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f368fe65740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36881ac480 con 0x55b8fae2b440 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f368fe65740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f36881abcf0 con 0x55b8fae26400 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f368fe65740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36881a8f50 con 0x55b8fae2bd60 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f368fe65740 10 client.0 _ll_register_callbacks cb 0x7f367c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae38390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae38390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39668/0 (socket says 172.21.15.103:39668) 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 -- 172.21.15.103:0/477764984 learned_addr learned my addr 172.21.15.103:0/477764984 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3686ffd640 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae47140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae47880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 -- 172.21.15.103:0/477764984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 msgr2=0x55b8fae47880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae47880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 -- 172.21.15.103:0/477764984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 msgr2=0x55b8fae47140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae47140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.182+0000 7f3687fff640 1 -- 172.21.15.103:0/477764984 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8faea95e0 con 0x55b8fae26400 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f3687fff640 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae38390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3680010df0 tx=0x7f3680012fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ffb6f4648a63ad server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f3666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f3666ffd640 1 -- 172.21.15.103:0/477764984 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680048030 con 0x55b8fae26400 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3680020040 con 0x55b8fae26400 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f3666ffd640 1 -- 172.21.15.103:0/477764984 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f3680011820 con 0x55b8fae26400 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f3666ffd640 1 -- 172.21.15.103:0/477764984 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3680043020 con 0x55b8fae26400 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f368fe65740 10 client.26849 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8faeaa7c0 con 0x55b8fae26400 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.183+0000 7f3666ffd640 1 -- 172.21.15.103:0/477764984 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3680020040 con 0x55b8fae26400 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f3666ffd640 1 -- 172.21.15.103:0/477764984 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3680025070 con 0x55b8fae26400 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f368fe65740 10 client.26849 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f368fe65740 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f368fe65740 20 client.26849 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f368fe65740 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8faeaae20 0x55b8faead210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f368fe65740 4 client.26849 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b8fad89270 con 0x55b8faeaae20 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f3686ffd640 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8faeaae20 0x55b8faead210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f3666ffd640 10 client.26849 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:25.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.184+0000 7f3686ffd640 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8faeaae20 0x55b8faead210 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11667e87fa607ed0 server_cookie=375ecaa403193b80 in_seq=0 out_seq=0 2024-08-19T01:44:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.186+0000 7f3666ffd640 1 -- 172.21.15.103:0/477764984 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b8fad89270 con 0x55b8faeaae20 2024-08-19T01:44:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.186+0000 7f3666ffd640 10 client.26849 handle_command_reply: tid=0 2024-08-19T01:44:25.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.186+0000 7f368fe65740 1 client.26849 shutdown 2024-08-19T01:44:25.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.188+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 msgr2=0x55b8fae38390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.188+0000 7f368fe65740 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae38390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3680010df0 tx=0x7f3680012fd0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.191+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 shutdown_connections 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.191+0000 7f368fe65740 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55b8faeaae20 0x55b8faead210 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.191+0000 7f368fe65740 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b8fae2bd60 0x55b8fae47880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.191+0000 7f368fe65740 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8fae2b440 0x55b8fae47140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.191+0000 7f368fe65740 1 --2- 172.21.15.103:0/477764984 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b8fae26400 0x55b8fae38390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.191+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 >> 172.21.15.103:0/477764984 conn(0x55b8fae18240 msgr2=0x55b8fae3a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.191+0000 7f3666ffd640 0 client.26849 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.192+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 shutdown_connections 2024-08-19T01:44:25.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.192+0000 7f368fe65740 1 -- 172.21.15.103:0/477764984 wait complete. 2024-08-19T01:44:25.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.192+0000 7f368fe65740 20 client.26849 trim_cache size 0 max 16384 2024-08-19T01:44:25.195 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:44:25.195 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-08-19T01:44:25.195 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "e3392fef-511e-4f6b-96e5-9dbbf598ef2a", 2024-08-19T01:44:25.195 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-08-19T01:44:25.195 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f36700a2c00 msgr2=0x7f36700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f36700a2c00 0x7f36700a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55b8fae19690 tx=0x7f367800b040 comp rx=0 tx=0).stop 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881007b0 msgr2=0x7f3688196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881007b0 0x7f3688196e20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3678009900 tx=0x7f3678038a10 comp rx=0 tx=0).stop 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 shutdown_connections 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f36700a2c00 0x7f36700a50c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f36881067b0 0x7f368819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3688101150 0x7f3688197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 --2- 172.21.15.103:0/953019789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36881007b0 0x7f3688196e20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:25.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 >> 172.21.15.103:0/953019789 conn(0x7f36880fc4d0 msgr2=0x7f36880fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:25.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.194+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 shutdown_connections 2024-08-19T01:44:25.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:25.195+0000 7f368da67640 1 -- 172.21.15.103:0/953019789 wait complete. 2024-08-19T01:44:25.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:25 smithi103 ceph-mon[18436]: pgmap v598: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 MiB/s wr, 6.90k op/s 2024-08-19T01:44:25.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:25 smithi190 ceph-mon[22032]: pgmap v598: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 MiB/s wr, 6.90k op/s 2024-08-19T01:44:25.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:25 smithi146 ceph-mon[22098]: pgmap v598: 97 pgs: 97 active+clean; 13 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 MiB/s wr, 6.90k op/s 2024-08-19T01:44:26.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:44:26.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub queued for path: / 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: active paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:26 smithi103 ceph-mon[18436]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub queued for path: / 2024-08-19T01:44:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:44:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:44:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:44:26.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: active paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:26 smithi190 ceph-mon[22032]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub queued for path: / 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: active paths [/] 2024-08-19T01:44:26.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:26 smithi146 ceph-mon[22098]: scrub summary: idle+waiting paths [/] 2024-08-19T01:44:26.491 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:27.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.105+0000 7f1011d9b640 1 -- 172.21.15.103:0/3845570347 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c037070 con 0x7f101411ae70 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 -- 172.21.15.103:0/3845570347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 msgr2=0x7f101411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3845570347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f101411d260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f100c009860 tx=0x7f100c02a340 comp rx=0 tx=0).stop 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 -- 172.21.15.103:0/3845570347 shutdown_connections 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3845570347 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f101411d260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3845570347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1014109560 0x7f10141099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3845570347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1014109180 0x7f101410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 -- 172.21.15.103:0/3845570347 >> 172.21.15.103:0/3845570347 conn(0x7f101406de70 msgr2=0x7f101406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 -- 172.21.15.103:0/3845570347 shutdown_connections 2024-08-19T01:44:27.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.106+0000 7f1018d4c640 1 -- 172.21.15.103:0/3845570347 wait complete. 2024-08-19T01:44:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.107+0000 7f1018d4c640 1 Processor -- start 2024-08-19T01:44:27.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.107+0000 7f1018d4c640 1 -- start start 2024-08-19T01:44:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1018d4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1014109180 0x7f10141a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1018d4c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1014109560 0x7f10141a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1018d4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f10141ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1018d4c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1014120980 con 0x7f1014109560 2024-08-19T01:44:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1018d4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1014120800 con 0x7f101411ae70 2024-08-19T01:44:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1018d4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1014120b00 con 0x7f1014109180 2024-08-19T01:44:27.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1012d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f10141ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1012d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f10141ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39674/0 (socket says 172.21.15.103:39674) 2024-08-19T01:44:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1012d9d640 1 -- 172.21.15.103:0/3514771549 learned_addr learned my addr 172.21.15.103:0/3514771549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f1009d9b640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1014109560 0x7f10141a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.108+0000 7f101259c640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1014109180 0x7f10141a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.109+0000 7f1012d9d640 1 -- 172.21.15.103:0/3514771549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1014109180 msgr2=0x7f10141a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.109+0000 7f1012d9d640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1014109180 0x7f10141a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.109+0000 7f1012d9d640 1 -- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1014109560 msgr2=0x7f10141a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.109+0000 7f1012d9d640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1014109560 0x7f10141a73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.109+0000 7f1012d9d640 1 -- 172.21.15.103:0/3514771549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10141abe10 con 0x7f101411ae70 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.109+0000 7f1012d9d640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f10141ab730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f100c005e00 tx=0x7f100c033d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.110+0000 7f100bfff640 1 -- 172.21.15.103:0/3514771549 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c03f020 con 0x7f101411ae70 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.110+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10141ac0a0 con 0x7f101411ae70 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.110+0000 7f100bfff640 1 -- 172.21.15.103:0/3514771549 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f100c005500 con 0x7f101411ae70 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.110+0000 7f100bfff640 1 -- 172.21.15.103:0/3514771549 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c037040 con 0x7f101411ae70 2024-08-19T01:44:27.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.110+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10141b39e0 con 0x7f101411ae70 2024-08-19T01:44:27.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.113+0000 7f101b14a740 1 Processor -- start 2024-08-19T01:44:27.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.113+0000 7f100bfff640 1 -- 172.21.15.103:0/3514771549 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f100c042020 con 0x7f101411ae70 2024-08-19T01:44:27.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101b14a740 1 -- start start 2024-08-19T01:44:27.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101b14a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e6f210 0x564bc8e6f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101b14a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6fb30 0x564bc8e79eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101b14a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101b14a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564bc8dd4980 con 0x564bc8e6fb30 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101b14a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564bc8d6ca70 con 0x564bc8e6f210 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101b14a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564bc8dd0940 con 0x564bc8e7a3f0 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6fb30 0x564bc8e79eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e6f210 0x564bc8e6f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6fb30 0x564bc8e79eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32940/0 (socket says 172.21.15.103:32940) 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f101259c640 1 -- 172.21.15.103:0/4260993162 learned_addr learned my addr 172.21.15.103:0/4260993162 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.114+0000 7f1009d9b640 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f100bfff640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10000a2cd0 0x7f10000a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f100bfff640 1 -- 172.21.15.103:0/3514771549 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f100c0eed90 con 0x7f101411ae70 2024-08-19T01:44:27.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f1012d9d640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10000a2cd0 0x7f10000a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f101259c640 1 -- 172.21.15.103:0/4260993162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 msgr2=0x564bc8e7c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f101259c640 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e7c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f101259c640 1 -- 172.21.15.103:0/4260993162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e6f210 msgr2=0x564bc8e6f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f101259c640 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e6f210 0x564bc8e6f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f101259c640 1 -- 172.21.15.103:0/4260993162 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc8dcf6e0 con 0x564bc8e6fb30 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.115+0000 7f1009d9b640 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e7c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.116+0000 7f1012d9d640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10000a2cd0 0x7f10000a5190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f100c0b7a80 tx=0x7f100c02c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.116+0000 7f101259c640 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6fb30 0x564bc8e79eb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00bcb0 tx=0x7f0ffc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abc4d3755f27817 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.117+0000 7f1008d99640 1 -- 172.21.15.103:0/4260993162 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc015070 con 0x564bc8e6fb30 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.117+0000 7f1008d99640 1 -- 172.21.15.103:0/4260993162 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ffc01a910 con 0x564bc8e6fb30 2024-08-19T01:44:27.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.117+0000 7f1008d99640 1 -- 172.21.15.103:0/4260993162 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011990 con 0x564bc8e6fb30 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 -- 172.21.15.103:0/4260993162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6fb30 msgr2=0x564bc8e79eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6fb30 0x564bc8e79eb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00bcb0 tx=0x7f0ffc00f730 comp rx=0 tx=0).stop 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 -- 172.21.15.103:0/4260993162 shutdown_connections 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e7c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6fb30 0x564bc8e79eb0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 --2- 172.21.15.103:0/4260993162 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e6f210 0x564bc8e6f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 -- 172.21.15.103:0/4260993162 >> 172.21.15.103:0/4260993162 conn(0x564bc8e6b7a0 msgr2=0x564bc8e6bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 -- 172.21.15.103:0/4260993162 shutdown_connections 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 -- 172.21.15.103:0/4260993162 wait complete. 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.118+0000 7f101b14a740 1 -- start start 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 1 Processor -- start 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e77420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e7a3f0 0x564bc8e77960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e77ea0 0x564bc8e86490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564bc8e7cfc0 con 0x564bc8e77ea0 2024-08-19T01:44:27.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564bc8d8faa0 con 0x564bc8e7a3f0 2024-08-19T01:44:27.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564bc8e7d140 con 0x564bc8e6f210 2024-08-19T01:44:27.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101b14a740 10 client.0 _ll_register_callbacks cb 0x564bc8d7ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e77420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e77ea0 0x564bc8e86490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e77ea0 0x564bc8e86490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32950/0 (socket says 172.21.15.103:32950) 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f1009d9b640 1 -- 172.21.15.103:0/4292821608 learned_addr learned my addr 172.21.15.103:0/4292821608 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.119+0000 7f101259c640 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e7a3f0 0x564bc8e77960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f101259c640 1 -- 172.21.15.103:0/4292821608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 msgr2=0x564bc8e77420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f101259c640 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e77420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f101259c640 1 -- 172.21.15.103:0/4292821608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e77ea0 msgr2=0x564bc8e86490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f101259c640 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e77ea0 0x564bc8e86490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f101259c640 1 -- 172.21.15.103:0/4292821608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc8d73650 con 0x564bc8e7a3f0 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f1009d9b640 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e77ea0 0x564bc8e86490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f101259c640 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e7a3f0 0x564bc8e77960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x564bc8e6c700 tx=0x7f0ffc00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470dc73e16f1a167 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:27.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.120+0000 7f0ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:44:27.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.122+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/4292821608 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc015070 con 0x564bc8e7a3f0 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.122+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/4292821608 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f0ffc020070 con 0x564bc8e7a3f0 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.122+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ffc01a910 con 0x564bc8e7a3f0 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.122+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/4292821608 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ffc015070 con 0x564bc8e7a3f0 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.122+0000 7f101b14a740 10 client.26861 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.122+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564bc8b463c0 con 0x564bc8e7a3f0 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/4292821608 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ffc01a910 con 0x564bc8e7a3f0 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/4292821608 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0ffc011710 con 0x564bc8e7a3f0 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f101b14a740 10 client.26861 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f101b14a740 10 client.26861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f101b14a740 20 client.26861 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f101b14a740 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8e8a390 0x564bc8e8c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f101b14a740 4 client.26861 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:27.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc8de1ef0 con 0x564bc8e8a390 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.123+0000 7f1009d9b640 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8e8a390 0x564bc8e8c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.124+0000 7f0ff9ffb640 10 client.26861 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.124+0000 7f1009d9b640 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8e8a390 0x564bc8e8c780 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199e73a3fcfab556 server_cookie=454a2e217d0dc7db in_seq=0 out_seq=0 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.125+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/4292821608 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bc8de1ef0 con 0x564bc8e8a390 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.125+0000 7f0ff9ffb640 10 client.26861 handle_command_reply: tid=0 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.125+0000 7f101b14a740 1 client.26861 shutdown 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e7a3f0 msgr2=0x564bc8e77960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e7a3f0 0x564bc8e77960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x564bc8e6c700 tx=0x7f0ffc00bfd0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 shutdown_connections 2024-08-19T01:44:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8e8a390 0x564bc8e8c780 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e77ea0 0x564bc8e86490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e7a3f0 0x564bc8e77960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 --2- 172.21.15.103:0/4292821608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e77420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f0ff9ffb640 0 client.26861 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 >> 172.21.15.103:0/4292821608 conn(0x564bc8e6b7a0 msgr2=0x564bc8e6a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 shutdown_connections 2024-08-19T01:44:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 1 -- 172.21.15.103:0/4292821608 wait complete. 2024-08-19T01:44:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.126+0000 7f101b14a740 20 client.26861 trim_cache size 0 max 16384 2024-08-19T01:44:27.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.158+0000 7f101b14a740 1 Processor -- start 2024-08-19T01:44:27.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.158+0000 7f101b14a740 1 -- start start 2024-08-19T01:44:27.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101b14a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6f210 0x564bc8e8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101b14a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e86280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101b14a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101b14a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10141bc0e0 con 0x564bc8e6f210 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101b14a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f10141bdce0 con 0x564bc8e77ea0 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101b14a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f10141bc7b0 con 0x564bc8e7a3f0 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6f210 0x564bc8e8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6f210 0x564bc8e8c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32960/0 (socket says 172.21.15.103:32960) 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f1009d9b640 1 -- 172.21.15.103:0/4186311867 learned_addr learned my addr 172.21.15.103:0/4186311867 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f1009d9b640 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101259c640 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e86280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101259c640 1 -- 172.21.15.103:0/4186311867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 msgr2=0x564bc8e762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101259c640 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101259c640 1 -- 172.21.15.103:0/4186311867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6f210 msgr2=0x564bc8e8c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101259c640 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6f210 0x564bc8e8c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f101259c640 1 -- 172.21.15.103:0/4186311867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc8e75970 con 0x564bc8e77ea0 2024-08-19T01:44:27.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.159+0000 7f1009d9b640 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6f210 0x564bc8e8c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:27.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.160+0000 7f101259c640 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e86280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc011bf0 tx=0x7f0ffc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdabc980bbc918a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:27.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.160+0000 7f0ffbfff640 1 -- 172.21.15.103:0/4186311867 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc015070 con 0x564bc8e77ea0 2024-08-19T01:44:27.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.160+0000 7f0ffbfff640 1 -- 172.21.15.103:0/4186311867 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ffc011440 con 0x564bc8e77ea0 2024-08-19T01:44:27.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.160+0000 7f0ffbfff640 1 -- 172.21.15.103:0/4186311867 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc01a8d0 con 0x564bc8e77ea0 2024-08-19T01:44:27.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 -- 172.21.15.103:0/4186311867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 msgr2=0x564bc8e86280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e86280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc011bf0 tx=0x7f0ffc005850 comp rx=0 tx=0).stop 2024-08-19T01:44:27.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 -- 172.21.15.103:0/4186311867 shutdown_connections 2024-08-19T01:44:27.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e7a3f0 0x564bc8e762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e86280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 --2- 172.21.15.103:0/4186311867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e6f210 0x564bc8e8c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 -- 172.21.15.103:0/4186311867 >> 172.21.15.103:0/4186311867 conn(0x564bc8e6b7a0 msgr2=0x564bc8e75bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:27.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.161+0000 7f101b14a740 1 -- 172.21.15.103:0/4186311867 shutdown_connections 2024-08-19T01:44:27.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.162+0000 7f101b14a740 1 -- 172.21.15.103:0/4186311867 wait complete. 2024-08-19T01:44:27.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.162+0000 7f101b14a740 1 -- start start 2024-08-19T01:44:27.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.162+0000 7f101b14a740 1 Processor -- start 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101b14a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e7d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101b14a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e7de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101b14a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e7a3f0 0x564bc8ef2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101b14a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10141bd610 con 0x564bc8e7a3f0 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101b14a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f10141bce80 con 0x564bc8e77ea0 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101b14a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f10141ba060 con 0x564bc8e6f210 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101b14a740 10 client.0 _ll_register_callbacks cb 0x7f1004009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101259c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e7de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e7d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e7d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:38214/0 (socket says 172.21.15.103:38214) 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 -- 172.21.15.103:0/3879665701 learned_addr learned my addr 172.21.15.103:0/3879665701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e7a3f0 0x564bc8ef2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 -- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 msgr2=0x564bc8e7de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e7de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 -- 172.21.15.103:0/3879665701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e7a3f0 msgr2=0x564bc8ef2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e7a3f0 0x564bc8ef2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f1009d9b640 1 -- 172.21.15.103:0/3879665701 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc8ef3440 con 0x564bc8e6f210 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.163+0000 7f101259c640 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e7de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f1009d9b640 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e7d950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1004009fd0 tx=0x7f1004009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b96ed783527b979 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f0ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/3879665701 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004003a30 con 0x564bc8e6f210 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/3879665701 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1004023040 con 0x564bc8e6f210 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ffc011440 con 0x564bc8e6f210 2024-08-19T01:44:27.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/3879665701 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1004003bd0 con 0x564bc8e6f210 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f101b14a740 10 client.26812 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564bc8ef4700 con 0x564bc8e6f210 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.164+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/3879665701 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ffc011440 con 0x564bc8e6f210 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/3879665701 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f100400b040 con 0x564bc8e6f210 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f101b14a740 10 client.26812 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f101b14a740 10 client.26812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f101b14a740 20 client.26812 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f101b14a740 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8ef4e80 0x564bc8ef72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f101b14a740 4 client.26812 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bc8ddc400 con 0x564bc8ef4e80 2024-08-19T01:44:27.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.165+0000 7f101259c640 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8ef4e80 0x564bc8ef72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:27.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f0ff9ffb640 10 client.26812 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:27.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101259c640 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8ef4e80 0x564bc8ef72a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4896796efba43824 server_cookie=929c864cf9fa9b30 in_seq=0 out_seq=0 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f0ff9ffb640 1 -- 172.21.15.103:0/3879665701 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564bc8ddc400 con 0x564bc8ef4e80 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f0ff9ffb640 10 client.26812 handle_command_reply: tid=0 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 client.26812 shutdown 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 msgr2=0x564bc8e7d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e7d950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1004009fd0 tx=0x7f1004009a70 comp rx=0 tx=0).stop 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 shutdown_connections 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x564bc8ef4e80 0x564bc8ef72a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bc8e7a3f0 0x564bc8ef2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564bc8e77ea0 0x564bc8e7de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 --2- 172.21.15.103:0/3879665701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564bc8e6f210 0x564bc8e7d950 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 >> 172.21.15.103:0/3879665701 conn(0x564bc8e6b7a0 msgr2=0x564bc8e763e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.166+0000 7f0ff9ffb640 0 client.26812 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.167+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 shutdown_connections 2024-08-19T01:44:27.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.167+0000 7f101b14a740 1 -- 172.21.15.103:0/3879665701 wait complete. 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.167+0000 7f101b14a740 20 client.26812 trim_cache size 0 max 16384 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: "e3392fef-511e-4f6b-96e5-9dbbf598ef2a": { 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "e3392fef-511e-4f6b-96e5-9dbbf598ef2a", 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:44:27.172 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10000a2cd0 msgr2=0x7f10000a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10000a2cd0 0x7f10000a5190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f100c0b7a80 tx=0x7f100c02c040 comp rx=0 tx=0).stop 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 msgr2=0x7f10141ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f10141ab730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f100c005e00 tx=0x7f100c033d60 comp rx=0 tx=0).stop 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 shutdown_connections 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f10000a2cd0 0x7f10000a5190 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f101411ae70 0x7f10141ab730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1014109560 0x7f10141a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 --2- 172.21.15.103:0/3514771549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1014109180 0x7f10141a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:27.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 >> 172.21.15.103:0/3514771549 conn(0x7f101406de70 msgr2=0x7f101411bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:27.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 shutdown_connections 2024-08-19T01:44:27.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:44:27.168+0000 7f1018d4c640 1 -- 172.21.15.103:0/3514771549 wait complete. 2024-08-19T01:44:27.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:27 smithi103 ceph-mon[18436]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 7.50k op/s 2024-08-19T01:44:27.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:27 smithi190 ceph-mon[22032]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 7.50k op/s 2024-08-19T01:44:27.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:27 smithi146 ceph-mon[22098]: pgmap v599: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 7.50k op/s 2024-08-19T01:44:28.044 INFO:tasks.cephfs.filesystem:scrub status for tag:e3392fef-511e-4f6b-96e5-9dbbf598ef2a - {'path': '/', 'tag': 'e3392fef-511e-4f6b-96e5-9dbbf598ef2a', 'options': 'recursive,force'} 2024-08-19T01:44:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 -- 172.21.15.103:0/695911579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e8100ca0 msgr2=0x7fc7e8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 --2- 172.21.15.103:0/695911579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e8100ca0 0x7fc7e8101120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8007380 tx=0x7fc7d802afc0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 -- 172.21.15.103:0/695911579 shutdown_connections 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 --2- 172.21.15.103:0/695911579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 0x7fc7e8107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 --2- 172.21.15.103:0/695911579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e8100ca0 0x7fc7e8101120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 --2- 172.21.15.103:0/695911579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7e80ffab0 0x7fc7e80ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 -- 172.21.15.103:0/695911579 >> 172.21.15.103:0/695911579 conn(0x7fc7e80fb230 msgr2=0x7fc7e80fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.251+0000 7fc7efe77640 1 -- 172.21.15.103:0/695911579 shutdown_connections 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 -- 172.21.15.103:0/695911579 wait complete. 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 Processor -- start 2024-08-19T01:44:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 -- start start 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e80ffab0 0x7fc7e819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7e8100ca0 0x7fc7e819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 0x7fc7e81a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e810ac10 con 0x7fc7e8100ca0 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e810aa90 con 0x7fc7e8101660 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.252+0000 7fc7efe77640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e810ad90 con 0x7fc7e80ffab0 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ed412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7e8100ca0 0x7fc7e819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ee414640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 0x7fc7e81a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7edc13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e80ffab0 0x7fc7e819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ed412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7e8100ca0 0x7fc7e819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32986/0 (socket says 172.21.15.103:32986) 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ee414640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 0x7fc7e81a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39734/0 (socket says 172.21.15.103:39734) 2024-08-19T01:44:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ed412640 1 -- 172.21.15.103:0/721377099 learned_addr learned my addr 172.21.15.103:0/721377099 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ee414640 1 -- 172.21.15.103:0/721377099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e80ffab0 msgr2=0x7fc7e819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ee414640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e80ffab0 0x7fc7e819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ee414640 1 -- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7e8100ca0 msgr2=0x7fc7e819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ee414640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7e8100ca0 0x7fc7e819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.253+0000 7fc7ee414640 1 -- 172.21.15.103:0/721377099 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e81a5cb0 con 0x7fc7e8101660 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.254+0000 7fc7ee414640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 0x7fc7e81a5640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009fd0 tx=0x7fc7e000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.254+0000 7fc7deffd640 1 -- 172.21.15.103:0/721377099 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000ce20 con 0x7fc7e8101660 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.254+0000 7fc7deffd640 1 -- 172.21.15.103:0/721377099 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7e0005020 con 0x7fc7e8101660 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.254+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e81a5fa0 con 0x7fc7e8101660 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.255+0000 7fc7deffd640 1 -- 172.21.15.103:0/721377099 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0010690 con 0x7fc7e8101660 2024-08-19T01:44:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.255+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e81a6460 con 0x7fc7e8101660 2024-08-19T01:44:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.258+0000 7fc7f2c33740 1 Processor -- start 2024-08-19T01:44:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7deffd640 1 -- 172.21.15.103:0/721377099 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc7e00051d0 con 0x7fc7e8101660 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7f2c33740 1 -- start start 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef323a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef328a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef32b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e3ef2869d0 con 0x55e3ef323680 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e3ef22e820 con 0x55e3ef323fc0 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e3ef282990 con 0x55e3ef329050 2024-08-19T01:44:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7edc13640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef328a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7ed412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef323a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7edc13640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef328a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:39750/0 (socket says 172.21.15.103:39750) 2024-08-19T01:44:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.259+0000 7fc7edc13640 1 -- 172.21.15.103:0/1696868590 learned_addr learned my addr 172.21.15.103:0/1696868590 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7edc13640 1 -- 172.21.15.103:0/1696868590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 msgr2=0x55e3ef32b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7edc13640 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef32b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7edc13640 1 -- 172.21.15.103:0/1696868590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 msgr2=0x55e3ef323a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7edc13640 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef323a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7edc13640 1 -- 172.21.15.103:0/1696868590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3eef9a210 con 0x55e3ef323fc0 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7edc13640 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef328a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009c80 tx=0x7fc7e4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a030dec35037f34 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7ed412640 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef323a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7deffd640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7d40a2d30 0x7fc7d40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7d3fff640 1 -- 172.21.15.103:0/1696868590 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4005220 con 0x55e3ef323fc0 2024-08-19T01:44:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.260+0000 7fc7deffd640 1 -- 172.21.15.103:0/721377099 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc7e00cf6d0 con 0x7fc7e8101660 2024-08-19T01:44:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.261+0000 7fc7d3fff640 1 -- 172.21.15.103:0/1696868590 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7e4009e30 con 0x55e3ef323fc0 2024-08-19T01:44:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.261+0000 7fc7d3fff640 1 -- 172.21.15.103:0/1696868590 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40197a0 con 0x55e3ef323fc0 2024-08-19T01:44:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.261+0000 7fc7ed412640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7d40a2d30 0x7fc7d40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 -- 172.21.15.103:0/1696868590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 msgr2=0x55e3ef328a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7ed412640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7d40a2d30 0x7fc7d40a51f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7e819f5a0 tx=0x7fc7d800d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef328a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009c80 tx=0x7fc7e4011810 comp rx=0 tx=0).stop 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 -- 172.21.15.103:0/1696868590 shutdown_connections 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef32b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef328a80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/1696868590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef323a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 -- 172.21.15.103:0/1696868590 >> 172.21.15.103:0/1696868590 conn(0x55e3ef31ecf0 msgr2=0x55e3ef31f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 -- 172.21.15.103:0/1696868590 shutdown_connections 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 -- 172.21.15.103:0/1696868590 wait complete. 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 -- start start 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.262+0000 7fc7f2c33740 1 Processor -- start 2024-08-19T01:44:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323680 0x55e3ef32d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef32dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e3ef32c020 con 0x55e3ef329050 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e3ef242520 con 0x55e3ef323fc0 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55e3ef32c1a0 con 0x55e3ef323680 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7edc13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323680 0x55e3ef32d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7edc13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323680 0x55e3ef32d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:38252/0 (socket says 172.21.15.103:38252) 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7edc13640 1 -- 172.21.15.103:0/291949987 learned_addr learned my addr 172.21.15.103:0/291949987 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.263+0000 7fc7ed412640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef32dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7f2c33740 10 client.0 _ll_register_callbacks cb 0x55e3eef99a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7ee414640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7edc13640 1 -- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 msgr2=0x55e3ef32dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7edc13640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef32dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7edc13640 1 -- 172.21.15.103:0/291949987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 msgr2=0x55e3ef334d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7edc13640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7edc13640 1 -- 172.21.15.103:0/291949987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ef066620 con 0x55e3ef323680 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7ee414640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7edc13640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323680 0x55e3ef32d800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40057c0 tx=0x7fc7e40048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d0d53b657b147c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7d17fa640 1 -- 172.21.15.103:0/291949987 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4029070 con 0x55e3ef323680 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7d17fa640 1 -- 172.21.15.103:0/291949987 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc7e40194a0 con 0x55e3ef323680 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.264+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e4009e30 con 0x55e3ef323680 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.265+0000 7fc7d17fa640 1 -- 172.21.15.103:0/291949987 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc7e4035050 con 0x55e3ef323680 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.265+0000 7fc7f2c33740 10 client.26821 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.265+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e3ef337b50 con 0x55e3ef323680 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.265+0000 7fc7d17fa640 1 -- 172.21.15.103:0/291949987 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7e4009870 con 0x55e3ef323680 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.265+0000 7fc7d17fa640 1 -- 172.21.15.103:0/291949987 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc7e4023850 con 0x55e3ef323680 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.266+0000 7fc7f2c33740 10 client.26821 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.266+0000 7fc7f2c33740 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.266+0000 7fc7f2c33740 20 client.26821 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.266+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef339620 0x55e3ef33ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.266+0000 7fc7f2c33740 4 client.26821 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.266+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc7e4009e30 con 0x55e3ef339620 2024-08-19T01:44:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.266+0000 7fc7ed412640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef339620 0x55e3ef33ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.267+0000 7fc7d17fa640 10 client.26821 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.267+0000 7fc7ed412640 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef339620 0x55e3ef33ba80 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ba1ecdcf92b7b3 server_cookie=933f36b7e9485bc5 in_seq=0 out_seq=0 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7d17fa640 1 -- 172.21.15.103:0/291949987 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc7e4009e30 con 0x55e3ef339620 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7d17fa640 10 client.26821 handle_command_reply: tid=0 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 client.26821 shutdown 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323680 msgr2=0x55e3ef32d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323680 0x55e3ef32d800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40057c0 tx=0x7fc7e40048f0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 shutdown_connections 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef339620 0x55e3ef33ba80 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef32dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/291949987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323680 0x55e3ef32d800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 >> 172.21.15.103:0/291949987 conn(0x55e3ef31ecf0 msgr2=0x55e3ef320500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.268+0000 7fc7d17fa640 0 client.26821 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.269+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 shutdown_connections 2024-08-19T01:44:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.269+0000 7fc7f2c33740 1 -- 172.21.15.103:0/291949987 wait complete. 2024-08-19T01:44:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.269+0000 7fc7f2c33740 20 client.26821 trim_cache size 0 max 16384 2024-08-19T01:44:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 Processor -- start 2024-08-19T01:44:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 -- start start 2024-08-19T01:44:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323680 0x55e3ef32dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323fc0 0x55e3ef32c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81af1e0 con 0x55e3ef329050 2024-08-19T01:44:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81b0de0 con 0x55e3ef323680 2024-08-19T01:44:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.295+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81af8b0 con 0x55e3ef323fc0 2024-08-19T01:44:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323fc0 0x55e3ef32c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ee414640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323fc0 0x55e3ef32c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:38262/0 (socket says 172.21.15.103:38262) 2024-08-19T01:44:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 -- 172.21.15.103:0/984275146 learned_addr learned my addr 172.21.15.103:0/984275146 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 -- 172.21.15.103:0/984275146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323680 msgr2=0x55e3ef32dcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:44:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 --2- 172.21.15.103:0/984275146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323680 0x55e3ef32dcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 -- 172.21.15.103:0/984275146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 msgr2=0x55e3ef334ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 --2- 172.21.15.103:0/984275146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.296+0000 7fc7ed412640 1 -- 172.21.15.103:0/984275146 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ef324c00 con 0x55e3ef323fc0 2024-08-19T01:44:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.297+0000 7fc7ed412640 1 --2- 172.21.15.103:0/984275146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323fc0 0x55e3ef32c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc7d802b5b0 tx=0x7fc7d803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec0eb72c44003afd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.297+0000 7fc7d37fe640 1 -- 172.21.15.103:0/984275146 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d802b900 con 0x55e3ef323fc0 2024-08-19T01:44:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.297+0000 7fc7d37fe640 1 -- 172.21.15.103:0/984275146 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7d804e070 con 0x55e3ef323fc0 2024-08-19T01:44:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.297+0000 7fc7d37fe640 1 -- 172.21.15.103:0/984275146 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800a2d0 con 0x55e3ef323fc0 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 -- 172.21.15.103:0/984275146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323fc0 msgr2=0x55e3ef32c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/984275146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323fc0 0x55e3ef32c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc7d802b5b0 tx=0x7fc7d803f000 comp rx=0 tx=0).stop 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 -- 172.21.15.103:0/984275146 shutdown_connections 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/984275146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef329050 0x55e3ef334ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/984275146 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef323fc0 0x55e3ef32c5a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/984275146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323680 0x55e3ef32dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 -- 172.21.15.103:0/984275146 >> 172.21.15.103:0/984275146 conn(0x55e3ef31ecf0 msgr2=0x55e3ef32d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 -- 172.21.15.103:0/984275146 shutdown_connections 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.298+0000 7fc7f2c33740 1 -- 172.21.15.103:0/984275146 wait complete. 2024-08-19T01:44:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.299+0000 7fc7f2c33740 1 -- start start 2024-08-19T01:44:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.299+0000 7fc7f2c33740 1 Processor -- start 2024-08-19T01:44:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef331a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef331f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7f2c33740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef3a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81b0710 con 0x55e3ef323680 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81aff80 con 0x55e3ef323fc0 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7f2c33740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81ad1c0 con 0x55e3ef329050 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7f2c33740 10 client.0 _ll_register_callbacks cb 0x7fc7e4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ee414640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef3a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ed412640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef331f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ee414640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef3a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:38274/0 (socket says 172.21.15.103:38274) 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ee414640 1 -- 172.21.15.103:0/2766520373 learned_addr learned my addr 172.21.15.103:0/2766520373 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7edc13640 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef331a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ed412640 1 -- 172.21.15.103:0/2766520373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 msgr2=0x55e3ef3a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ed412640 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef3a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ed412640 1 -- 172.21.15.103:0/2766520373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 msgr2=0x55e3ef331a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ed412640 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef331a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.300+0000 7fc7ed412640 1 -- 172.21.15.103:0/2766520373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ef334fc0 con 0x55e3ef323fc0 2024-08-19T01:44:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.301+0000 7fc7ed412640 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef331f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800d090 tx=0x7fc7d802afc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5821e6cad46718 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.301+0000 7fc7d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:44:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.301+0000 7fc7d17fa640 1 -- 172.21.15.103:0/2766520373 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8061060 con 0x55e3ef323fc0 2024-08-19T01:44:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.301+0000 7fc7d17fa640 1 -- 172.21.15.103:0/2766520373 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc7d802bd10 con 0x55e3ef323fc0 2024-08-19T01:44:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.301+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d804e070 con 0x55e3ef323fc0 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.302+0000 7fc7d17fa640 1 -- 172.21.15.103:0/2766520373 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc7d800a0d0 con 0x55e3ef323fc0 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.302+0000 7fc7f2c33740 10 client.26882 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.302+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e3ef3a3840 con 0x55e3ef323fc0 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.302+0000 7fc7d17fa640 1 -- 172.21.15.103:0/2766520373 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7d804e070 con 0x55e3ef323fc0 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7d17fa640 1 -- 172.21.15.103:0/2766520373 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc7d8004ca0 con 0x55e3ef323fc0 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7f2c33740 10 client.26882 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7f2c33740 10 client.26882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7f2c33740 20 client.26882 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef3a41d0 0x55e3ef3a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7f2c33740 4 client.26882 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:44:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3ef294100 con 0x55e3ef3a41d0 2024-08-19T01:44:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.303+0000 7fc7edc13640 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef3a41d0 0x55e3ef3a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.304+0000 7fc7d17fa640 10 client.26882 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.304+0000 7fc7edc13640 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef3a41d0 0x55e3ef3a65f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dddfe4c3b4d07e7 server_cookie=4c901a155f01dfd9 in_seq=0 out_seq=0 2024-08-19T01:44:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.324+0000 7fc7d17fa640 1 -- 172.21.15.103:0/2766520373 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3ef294100 con 0x55e3ef3a41d0 2024-08-19T01:44:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.324+0000 7fc7d17fa640 10 client.26882 handle_command_reply: tid=0 2024-08-19T01:44:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.324+0000 7fc7f2c33740 1 client.26882 shutdown 2024-08-19T01:44:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 msgr2=0x55e3ef331f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef331f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800d090 tx=0x7fc7d802afc0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 shutdown_connections 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55e3ef3a41d0 0x55e3ef3a65f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3ef329050 0x55e3ef3a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7d17fa640 0 client.26882 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3ef323fc0 0x55e3ef331f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 --2- 172.21.15.103:0/2766520373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3ef323680 0x55e3ef331a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 >> 172.21.15.103:0/2766520373 conn(0x55e3ef31ecf0 msgr2=0x55e3ef31d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.325+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 shutdown_connections 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7f2c33740 1 -- 172.21.15.103:0/2766520373 wait complete. 2024-08-19T01:44:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7f2c33740 20 client.26882 trim_cache size 0 max 16384 2024-08-19T01:44:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:44:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:44:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:44:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7d40a2d30 msgr2=0x7fc7d40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7d40a2d30 0x7fc7d40a51f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7e819f5a0 tx=0x7fc7d800d000 comp rx=0 tx=0).stop 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 msgr2=0x7fc7e81a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 0x7fc7e81a5640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009fd0 tx=0x7fc7e000ef10 comp rx=0 tx=0).stop 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 shutdown_connections 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc7d40a2d30 0x7fc7d40a51f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7e8101660 0x7fc7e81a5640 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7e8100ca0 0x7fc7e819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 --2- 172.21.15.103:0/721377099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7e80ffab0 0x7fc7e819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.326+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 >> 172.21.15.103:0/721377099 conn(0x7fc7e80fb230 msgr2=0x7fc7e810dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.327+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 shutdown_connections 2024-08-19T01:44:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:28.327+0000 7fc7efe77640 1 -- 172.21.15.103:0/721377099 wait complete. 2024-08-19T01:44:29.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:29 smithi103 ceph-mon[18436]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:29 smithi190 ceph-mon[22032]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:29.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:29 smithi146 ceph-mon[22098]: pgmap v600: 97 pgs: 97 active+clean; 13 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:31.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:31 smithi103 ceph-mon[18436]: pgmap v601: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:31 smithi190 ceph-mon[22032]: pgmap v601: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:31.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:31 smithi146 ceph-mon[22098]: pgmap v601: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:33.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:33 smithi103 ceph-mon[18436]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 8.49k op/s 2024-08-19T01:44:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:33 smithi190 ceph-mon[22032]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 8.49k op/s 2024-08-19T01:44:33.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:33 smithi146 ceph-mon[22098]: pgmap v602: 97 pgs: 97 active+clean; 13 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 33 MiB/s wr, 8.49k op/s 2024-08-19T01:44:35.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:35 smithi103 ceph-mon[18436]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:35.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:35 smithi190 ceph-mon[22032]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:35.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:35 smithi146 ceph-mon[22098]: pgmap v603: 97 pgs: 97 active+clean; 13 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:36.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:36 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:44:36.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:36 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:44:36.950 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:36 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:44:37.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:37 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:44:37.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:37 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:44:37.487 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:37 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:44:37.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:37 smithi103 ceph-mon[18436]: scrub summary: idle 2024-08-19T01:44:37.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:37 smithi103 ceph-mon[18436]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 7.69k op/s 2024-08-19T01:44:37.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:37 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:44:37.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:37 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:44:37.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:37 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:44:37.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:37 smithi190 ceph-mon[22032]: scrub summary: idle 2024-08-19T01:44:37.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:37 smithi190 ceph-mon[22032]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 7.69k op/s 2024-08-19T01:44:37.950 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:37 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:44:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:37 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:44:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:37 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:44:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:37 smithi146 ceph-mon[22098]: scrub summary: idle 2024-08-19T01:44:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:37 smithi146 ceph-mon[22098]: pgmap v604: 97 pgs: 97 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 7.69k op/s 2024-08-19T01:44:38.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:38 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:44:38.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:38 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:44:38.950 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:38 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:44:39.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:39 smithi103 ceph-mon[18436]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 7.27k op/s 2024-08-19T01:44:39.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:44:39.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:44:39.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:39 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:44:39.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:39 smithi190 ceph-mon[22032]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 7.27k op/s 2024-08-19T01:44:39.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:44:39.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:44:39.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:39 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:44:39.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:39 smithi146 ceph-mon[22098]: pgmap v605: 97 pgs: 97 active+clean; 13 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 7.27k op/s 2024-08-19T01:44:39.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:44:39.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:44:39.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:39 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:44:41.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:41 smithi103 ceph-mon[18436]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 MiB/s wr, 7.31k op/s 2024-08-19T01:44:41.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:41 smithi190 ceph-mon[22032]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 MiB/s wr, 7.31k op/s 2024-08-19T01:44:41.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:41 smithi146 ceph-mon[22098]: pgmap v606: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 MiB/s wr, 7.31k op/s 2024-08-19T01:44:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.477+0000 7fde771a1640 1 -- 172.21.15.103:0/2694228013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70109960 msgr2=0x7fde70110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.477+0000 7fde771a1640 1 --2- 172.21.15.103:0/2694228013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70109960 0x7fde70110280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde68009be0 tx=0x7fde6802a6c0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.478+0000 7fde771a1640 1 -- 172.21.15.103:0/2694228013 shutdown_connections 2024-08-19T01:44:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.478+0000 7fde771a1640 1 --2- 172.21.15.103:0/2694228013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70109960 0x7fde70110280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.478+0000 7fde771a1640 1 --2- 172.21.15.103:0/2694228013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70108fa0 0x7fde70109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.478+0000 7fde771a1640 1 --2- 172.21.15.103:0/2694228013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70108140 0x7fde70108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.478+0000 7fde771a1640 1 -- 172.21.15.103:0/2694228013 >> 172.21.15.103:0/2694228013 conn(0x7fde7006b400 msgr2=0x7fde70079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.478+0000 7fde771a1640 1 -- 172.21.15.103:0/2694228013 shutdown_connections 2024-08-19T01:44:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.478+0000 7fde771a1640 1 -- 172.21.15.103:0/2694228013 wait complete. 2024-08-19T01:44:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 Processor -- start 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 -- start start 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70108140 0x7fde7019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70108fa0 0x7fde7019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70109960 0x7fde701a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde701139a0 con 0x7fde70108140 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fde70113820 con 0x7fde70109960 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.479+0000 7fde771a1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde70113b20 con 0x7fde70108fa0 2024-08-19T01:44:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde74f3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70108140 0x7fde7019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70109960 0x7fde701a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70109960 0x7fde701a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:51472/0 (socket says 172.21.15.103:51472) 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 -- 172.21.15.103:0/1064441802 learned_addr learned my addr 172.21.15.103:0/1064441802 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde6ffff640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70108fa0 0x7fde7019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 -- 172.21.15.103:0/1064441802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70108fa0 msgr2=0x7fde7019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70108fa0 0x7fde7019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 -- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70108140 msgr2=0x7fde7019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70108140 0x7fde7019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde7573e640 1 -- 172.21.15.103:0/1064441802 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde701a5ba0 con 0x7fde70109960 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde6ffff640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70108fa0 0x7fde7019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.480+0000 7fde74f3d640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70108140 0x7fde7019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.481+0000 7fde7573e640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70109960 0x7fde701a5530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde68034040 tx=0x7fde68030f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.481+0000 7fde6dffb640 1 -- 172.21.15.103:0/1064441802 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68041020 con 0x7fde70109960 2024-08-19T01:44:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.481+0000 7fde6dffb640 1 -- 172.21.15.103:0/1064441802 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde68037040 con 0x7fde70109960 2024-08-19T01:44:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.481+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde701a5e30 con 0x7fde70109960 2024-08-19T01:44:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.481+0000 7fde6dffb640 1 -- 172.21.15.103:0/1064441802 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68036070 con 0x7fde70109960 2024-08-19T01:44:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.481+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde701a6230 con 0x7fde70109960 2024-08-19T01:44:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.485+0000 7fde6dffb640 1 -- 172.21.15.103:0/1064441802 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fde68044020 con 0x7fde70109960 2024-08-19T01:44:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.487+0000 7fde7a03a740 1 Processor -- start 2024-08-19T01:44:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.487+0000 7fde7a03a740 1 -- start start 2024-08-19T01:44:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.487+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83680 0x55621db83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db83fc0 0x55621db88a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde6dffb640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fde600a2d30 0x7fde600a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde7a03a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55621dae69d0 con 0x55621db83680 2024-08-19T01:44:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde7a03a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55621da8e820 con 0x55621db83fc0 2024-08-19T01:44:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde7a03a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55621dae2990 con 0x55621db89050 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83680 0x55621db83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83680 0x55621db83a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34364/0 (socket says 172.21.15.103:34364) 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 -- 172.21.15.103:0/4184962402 learned_addr learned my addr 172.21.15.103:0/4184962402 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde7573e640 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 -- 172.21.15.103:0/4184962402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 msgr2=0x55621db8b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde6dffb640 1 -- 172.21.15.103:0/1064441802 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fde680efca0 con 0x7fde70109960 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 -- 172.21.15.103:0/4184962402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db83fc0 msgr2=0x55621db88a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db83fc0 0x55621db88a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde74f3d640 1 -- 172.21.15.103:0/4184962402 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621d7fa210 con 0x55621db83680 2024-08-19T01:44:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde7573e640 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.488+0000 7fde6ffff640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fde600a2d30 0x7fde600a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.491 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.489+0000 7fde74f3d640 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83680 0x55621db83a80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fde64011510 tx=0x7fde640119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5c89198aeb7588 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.489+0000 7fde5affd640 1 -- 172.21.15.103:0/4184962402 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6402e020 con 0x55621db83680 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.489+0000 7fde5affd640 1 -- 172.21.15.103:0/4184962402 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde64004220 con 0x55621db83680 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.489+0000 7fde5affd640 1 -- 172.21.15.103:0/4184962402 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6400f8b0 con 0x55621db83680 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.491+0000 7fde7a03a740 1 -- 172.21.15.103:0/4184962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83680 msgr2=0x55621db83a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.491+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83680 0x55621db83a80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fde64011510 tx=0x7fde640119d0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.491+0000 7fde7a03a740 1 -- 172.21.15.103:0/4184962402 shutdown_connections 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.491+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.491+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db83fc0 0x55621db88a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.491+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4184962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83680 0x55621db83a80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.493 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.491+0000 7fde7a03a740 1 -- 172.21.15.103:0/4184962402 >> 172.21.15.103:0/4184962402 conn(0x55621db7ecf0 msgr2=0x55621db7f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.492+0000 7fde6ffff640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fde600a2d30 0x7fde600a51f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55621db849f0 tx=0x7fde5c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.493+0000 7fde7a03a740 1 -- 172.21.15.103:0/4184962402 shutdown_connections 2024-08-19T01:44:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.493+0000 7fde7a03a740 1 -- 172.21.15.103:0/4184962402 wait complete. 2024-08-19T01:44:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.493+0000 7fde7a03a740 1 -- start start 2024-08-19T01:44:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.493+0000 7fde7a03a740 1 Processor -- start 2024-08-19T01:44:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621db8da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7a03a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55621db8c020 con 0x55621db83fc0 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7a03a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55621daa2520 con 0x55621db92ff0 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7a03a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55621db8c1a0 con 0x55621db89050 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7a03a740 10 client.0 _ll_register_callbacks cb 0x55621d7f9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:39594/0 (socket says 172.21.15.103:39594) 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde6ffff640 1 -- 172.21.15.103:0/3427013258 learned_addr learned my addr 172.21.15.103:0/3427013258 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde7573e640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.494+0000 7fde74f3d640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621db8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde6ffff640 1 -- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 msgr2=0x55621db953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde6ffff640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde6ffff640 1 -- 172.21.15.103:0/3427013258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 msgr2=0x55621db8da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde6ffff640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621db8da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde6ffff640 1 -- 172.21.15.103:0/3427013258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621d8c6620 con 0x55621db89050 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde74f3d640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621db8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde7573e640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde6ffff640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8dfc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde5c012900 tx=0x7fde5c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9affa0421ce8c542 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde4ffff640 1 -- 172.21.15.103:0/3427013258 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00fd60 con 0x55621db89050 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde4ffff640 1 -- 172.21.15.103:0/3427013258 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fde5c01f420 con 0x55621db89050 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.495+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde64004220 con 0x55621db89050 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.496+0000 7fde4ffff640 1 -- 172.21.15.103:0/3427013258 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fde5c01f620 con 0x55621db89050 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.496+0000 7fde7a03a740 10 client.26839 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.496+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55621db982a0 con 0x55621db89050 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.496+0000 7fde4ffff640 1 -- 172.21.15.103:0/3427013258 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde64004220 con 0x55621db89050 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde4ffff640 1 -- 172.21.15.103:0/3427013258 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fde5c0277d0 con 0x55621db89050 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde7a03a740 10 client.26839 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde7a03a740 10 client.26839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde7a03a740 20 client.26839 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde7a03a740 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621db9c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde7a03a740 4 client.26839 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55621daf4100 con 0x55621db99dd0 2024-08-19T01:44:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.497+0000 7fde74f3d640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621db9c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.498+0000 7fde74f3d640 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621db9c250 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d77ec2c02857aa8 server_cookie=285458e551df8ddc in_seq=0 out_seq=0 2024-08-19T01:44:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.498+0000 7fde4ffff640 10 client.26839 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde4ffff640 1 -- 172.21.15.103:0/3427013258 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55621daf4100 con 0x55621db99dd0 2024-08-19T01:44:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde4ffff640 10 client.26839 handle_command_reply: tid=0 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 client.26839 shutdown 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 msgr2=0x55621db8dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8dfc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde5c012900 tx=0x7fde5c004730 comp rx=0 tx=0).stop 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 shutdown_connections 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621db9c250 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde4ffff640 0 client.26839 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db89050 0x55621db8dfc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 --2- 172.21.15.103:0/3427013258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621db8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.499+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 >> 172.21.15.103:0/3427013258 conn(0x55621db7ecf0 msgr2=0x55621db80500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.500+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 shutdown_connections 2024-08-19T01:44:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.500+0000 7fde7a03a740 1 -- 172.21.15.103:0/3427013258 wait complete. 2024-08-19T01:44:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.500+0000 7fde7a03a740 20 client.26839 trim_cache size 0 max 16384 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.522+0000 7fde7a03a740 1 Processor -- start 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7a03a740 1 -- start start 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db83fc0 0x55621db8bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db89050 0x55621db8c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db8da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7a03a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde701a6900 con 0x55621db89050 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7a03a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fde701a8500 con 0x55621db92ff0 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7a03a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde701a6fd0 con 0x55621db83fc0 2024-08-19T01:44:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db89050 0x55621db8c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7573e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db89050 0x55621db8c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34376/0 (socket says 172.21.15.103:34376) 2024-08-19T01:44:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde74f3d640 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db83fc0 0x55621db8bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde6ffff640 1 -- 172.21.15.103:0/4199016101 learned_addr learned my addr 172.21.15.103:0/4199016101 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.523+0000 7fde7573e640 1 -- 172.21.15.103:0/4199016101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db83fc0 msgr2=0x55621db8bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde7573e640 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db83fc0 0x55621db8bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde7573e640 1 -- 172.21.15.103:0/4199016101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db89050 msgr2=0x55621db8c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde7573e640 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db89050 0x55621db8c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde7573e640 1 -- 172.21.15.103:0/4199016101 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621db8d5c0 con 0x55621db92ff0 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde7573e640 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db8da80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde680bc1d0 tx=0x7fde680ed640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b22e4a1331725dc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde5a7fc640 1 -- 172.21.15.103:0/4199016101 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680bc010 con 0x55621db92ff0 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde5a7fc640 1 -- 172.21.15.103:0/4199016101 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde68004690 con 0x55621db92ff0 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.524+0000 7fde5a7fc640 1 -- 172.21.15.103:0/4199016101 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680c0880 con 0x55621db92ff0 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.525+0000 7fde7a03a740 1 -- 172.21.15.103:0/4199016101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 msgr2=0x55621db8da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.525+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db8da80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fde680bc1d0 tx=0x7fde680ed640 comp rx=0 tx=0).stop 2024-08-19T01:44:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.526+0000 7fde7a03a740 1 -- 172.21.15.103:0/4199016101 shutdown_connections 2024-08-19T01:44:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.526+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db92ff0 0x55621db8da80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.526+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db89050 0x55621db8c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.526+0000 7fde7a03a740 1 --2- 172.21.15.103:0/4199016101 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db83fc0 0x55621db8bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.526+0000 7fde7a03a740 1 -- 172.21.15.103:0/4199016101 >> 172.21.15.103:0/4199016101 conn(0x55621db7ecf0 msgr2=0x55621db853e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.526+0000 7fde7a03a740 1 -- 172.21.15.103:0/4199016101 shutdown_connections 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.527+0000 7fde7a03a740 1 -- 172.21.15.103:0/4199016101 wait complete. 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.527+0000 7fde7a03a740 1 -- start start 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.527+0000 7fde7a03a740 1 Processor -- start 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.527+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621dbab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.527+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db89050 0x55621dbab600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.527+0000 7fde7a03a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db92ff0 0x55621dc029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7a03a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde701a7e30 con 0x55621db83fc0 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7a03a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fde701a76a0 con 0x55621db89050 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7a03a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fde70072570 con 0x55621db92ff0 2024-08-19T01:44:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7a03a740 10 client.0 _ll_register_callbacks cb 0x7fde5c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde74f3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621dbab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7573e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db92ff0 0x55621dc029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde74f3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621dbab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34388/0 (socket says 172.21.15.103:34388) 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db89050 0x55621dbab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde74f3d640 1 -- 172.21.15.103:0/1355966746 learned_addr learned my addr 172.21.15.103:0/1355966746 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7573e640 1 -- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db89050 msgr2=0x55621dbab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7573e640 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db89050 0x55621dbab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7573e640 1 -- 172.21.15.103:0/1355966746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 msgr2=0x55621dbab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7573e640 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621dbab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7573e640 1 -- 172.21.15.103:0/1355966746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621dc02ee0 con 0x55621db92ff0 2024-08-19T01:44:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde6ffff640 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db89050 0x55621dbab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde7573e640 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db92ff0 0x55621dc029a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55621daa25a0 tx=0x7fde68009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8592b2daadefd737 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde4ffff640 1 -- 172.21.15.103:0/1355966746 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680c5140 con 0x55621db92ff0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.528+0000 7fde4ffff640 1 -- 172.21.15.103:0/1355966746 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fde680b9b20 con 0x55621db92ff0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55621daee450 con 0x55621db92ff0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde4ffff640 1 -- 172.21.15.103:0/1355966746 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fde680c0880 con 0x55621db92ff0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 10 client.26845 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55621dc04060 con 0x55621db92ff0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde4ffff640 1 -- 172.21.15.103:0/1355966746 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde680d26a0 con 0x55621db92ff0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde4ffff640 1 -- 172.21.15.103:0/1355966746 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fde680c4cb0 con 0x55621db92ff0 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 10 client.26845 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 10 client.26845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 20 client.26845 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621dc069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 4 client.26845 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:44:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.529+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55621daee450 con 0x55621db99dd0 2024-08-19T01:44:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.530+0000 7fde74f3d640 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621dc069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.530+0000 7fde4ffff640 10 client.26845 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.530+0000 7fde74f3d640 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621dc069a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbce1eae03854ed server_cookie=61d04af1dc92f82 in_seq=0 out_seq=0 2024-08-19T01:44:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.541+0000 7fde4ffff640 1 -- 172.21.15.103:0/1355966746 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55621daee450 con 0x55621db99dd0 2024-08-19T01:44:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.541+0000 7fde4ffff640 10 client.26845 handle_command_reply: tid=0 2024-08-19T01:44:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.541+0000 7fde7a03a740 1 client.26845 shutdown 2024-08-19T01:44:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.541+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db92ff0 msgr2=0x55621dc029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.541+0000 7fde7a03a740 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db92ff0 0x55621dc029a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55621daa25a0 tx=0x7fde68009500 comp rx=0 tx=0).stop 2024-08-19T01:44:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.541+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 shutdown_connections 2024-08-19T01:44:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.542+0000 7fde7a03a740 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55621db99dd0 0x55621dc069a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.542+0000 7fde4ffff640 0 client.26845 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.542+0000 7fde7a03a740 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55621db92ff0 0x55621dc029a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.542+0000 7fde7a03a740 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55621db89050 0x55621dbab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.542+0000 7fde7a03a740 1 --2- 172.21.15.103:0/1355966746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55621db83fc0 0x55621dbab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.542+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 >> 172.21.15.103:0/1355966746 conn(0x55621db7ecf0 msgr2=0x55621db8da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.543+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 shutdown_connections 2024-08-19T01:44:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.543+0000 7fde7a03a740 1 -- 172.21.15.103:0/1355966746 wait complete. 2024-08-19T01:44:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.543+0000 7fde7a03a740 20 client.26845 trim_cache size 0 max 16384 2024-08-19T01:44:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:44:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:44:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:44:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fde600a2d30 msgr2=0x7fde600a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fde600a2d30 0x7fde600a51f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55621db849f0 tx=0x7fde5c005f90 comp rx=0 tx=0).stop 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70109960 msgr2=0x7fde701a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70109960 0x7fde701a5530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde68034040 tx=0x7fde68030f90 comp rx=0 tx=0).stop 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 shutdown_connections 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fde600a2d30 0x7fde600a51f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde70109960 0x7fde701a5530 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70108fa0 0x7fde7019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 --2- 172.21.15.103:0/1064441802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70108140 0x7fde7019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 >> 172.21.15.103:0/1064441802 conn(0x7fde7006b400 msgr2=0x7fde7010c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.544+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 shutdown_connections 2024-08-19T01:44:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:41.545+0000 7fde771a1640 1 -- 172.21.15.103:0/1064441802 wait complete. 2024-08-19T01:44:43.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:43 smithi103 ceph-mon[18436]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 8.26k op/s 2024-08-19T01:44:43.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:43 smithi190 ceph-mon[22032]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 8.26k op/s 2024-08-19T01:44:43.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:43 smithi146 ceph-mon[22098]: pgmap v607: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 8.26k op/s 2024-08-19T01:44:45.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:45 smithi103 ceph-mon[18436]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 7.08k op/s 2024-08-19T01:44:45.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:45 smithi190 ceph-mon[22032]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 7.08k op/s 2024-08-19T01:44:45.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:45 smithi146 ceph-mon[22098]: pgmap v608: 97 pgs: 97 active+clean; 13 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 7.08k op/s 2024-08-19T01:44:47.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:47 smithi103 ceph-mon[18436]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:47.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:47 smithi190 ceph-mon[22032]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:47.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:47 smithi146 ceph-mon[22098]: pgmap v609: 97 pgs: 97 active+clean; 13 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 7.32k op/s 2024-08-19T01:44:49.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:49 smithi103 ceph-mon[18436]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 7.08k op/s 2024-08-19T01:44:49.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:49 smithi190 ceph-mon[22032]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 7.08k op/s 2024-08-19T01:44:49.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:49 smithi146 ceph-mon[22098]: pgmap v610: 97 pgs: 97 active+clean; 13 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 7.08k op/s 2024-08-19T01:44:51.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:51 smithi103 ceph-mon[18436]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 7.14k op/s 2024-08-19T01:44:51.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:51 smithi190 ceph-mon[22032]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 7.14k op/s 2024-08-19T01:44:51.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:51 smithi146 ceph-mon[22098]: pgmap v611: 97 pgs: 97 active+clean; 13 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 7.14k op/s 2024-08-19T01:44:53.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:53 smithi103 ceph-mon[18436]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 MiB/s wr, 8.13k op/s 2024-08-19T01:44:53.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:53 smithi190 ceph-mon[22032]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 MiB/s wr, 8.13k op/s 2024-08-19T01:44:53.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:53 smithi146 ceph-mon[22098]: pgmap v612: 97 pgs: 97 active+clean; 13 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 32 MiB/s wr, 8.13k op/s 2024-08-19T01:44:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.689+0000 7ff01dc0c640 1 -- 172.21.15.103:0/4091629321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0180730a0 msgr2=0x7ff018073520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.689+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/4091629321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0180730a0 0x7ff018073520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff004009930 tx=0x7ff00402a420 comp rx=0 tx=0).stop 2024-08-19T01:44:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.689+0000 7ff01dc0c640 1 -- 172.21.15.103:0/4091629321 shutdown_connections 2024-08-19T01:44:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.689+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/4091629321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff018073a60 0x7ff018112480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.689+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/4091629321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff0180730a0 0x7ff018073520 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff004009930 tx=0x7ff00402a420 comp rx=0 tx=0).stop 2024-08-19T01:44:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.689+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/4091629321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff018109190 0x7ff018109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.689+0000 7ff01dc0c640 1 -- 172.21.15.103:0/4091629321 >> 172.21.15.103:0/4091629321 conn(0x7ff01806ea40 msgr2=0x7ff018070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.693+0000 7ff01dc0c640 1 -- 172.21.15.103:0/4091629321 shutdown_connections 2024-08-19T01:44:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.693+0000 7ff01dc0c640 1 -- 172.21.15.103:0/4091629321 wait complete. 2024-08-19T01:44:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.694+0000 7ff01dc0c640 1 Processor -- start 2024-08-19T01:44:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01dc0c640 1 -- start start 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01dc0c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff018073a60 0x7ff0181a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01dc0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff018109190 0x7ff0181a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01dc0c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0181a7b00 0x7ff0181a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01cc0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff018073a60 0x7ff0181a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01cc0a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff018073a60 0x7ff0181a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:50114/0 (socket says 172.21.15.103:50114) 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01dc0c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff018115a00 con 0x7ff018109190 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01dc0c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff018115880 con 0x7ff018073a60 2024-08-19T01:44:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01dc0c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff018115b80 con 0x7ff0181a7b00 2024-08-19T01:44:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01cc0a640 1 -- 172.21.15.103:0/659631565 learned_addr learned my addr 172.21.15.103:0/659631565 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff017fff640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff018109190 0x7ff0181a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.695+0000 7ff01d40b640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0181a7b00 0x7ff0181a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01cc0a640 1 -- 172.21.15.103:0/659631565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0181a7b00 msgr2=0x7ff0181a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0181a7b00 0x7ff0181a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01cc0a640 1 -- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff018109190 msgr2=0x7ff0181a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff018109190 0x7ff0181a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01cc0a640 1 -- 172.21.15.103:0/659631565 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0181aa560 con 0x7ff018073a60 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff018073a60 0x7ff0181a2560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00d950 tx=0x7ff00c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff015ffb640 1 -- 172.21.15.103:0/659631565 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c017070 con 0x7ff018073a60 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0181aa850 con 0x7ff018073a60 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff015ffb640 1 -- 172.21.15.103:0/659631565 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00c009c30 con 0x7ff018073a60 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff015ffb640 1 -- 172.21.15.103:0/659631565 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00f620 con 0x7ff018073a60 2024-08-19T01:44:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.696+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0181aad70 con 0x7ff018073a60 2024-08-19T01:44:54.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.699+0000 7ff015ffb640 1 -- 172.21.15.103:0/659631565 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff00c0109a0 con 0x7ff018073a60 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 Processor -- start 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 -- start start 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb020 0x55ac09bcb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bd0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bd2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ac09b259d0 con 0x55ac09bcb020 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ac09acd820 con 0x55ac09bd09f0 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff021c2d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ac09b21990 con 0x55ac09bcb960 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff015ffb640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0080a2b80 0x7ff0080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff01cc0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bd0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff017fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bd2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff01cc0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bd0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58012/0 (socket says 172.21.15.103:58012) 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff01cc0a640 1 -- 172.21.15.103:0/637587829 learned_addr learned my addr 172.21.15.103:0/637587829 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff01d40b640 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb020 0x55ac09bcb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff01d40b640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0080a2b80 0x7ff0080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.700+0000 7ff015ffb640 1 -- 172.21.15.103:0/659631565 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff00c0ceb40 con 0x7ff018073a60 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff01cc0a640 1 -- 172.21.15.103:0/637587829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 msgr2=0x55ac09bd2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bd2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff01cc0a640 1 -- 172.21.15.103:0/637587829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb020 msgr2=0x55ac09bcb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb020 0x55ac09bcb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff01cc0a640 1 -- 172.21.15.103:0/637587829 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac09839210 con 0x55ac09bcb960 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bd0420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff00c097890 tx=0x7ff00c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=687a04a4a3a7bbd server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff01d40b640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0080a2b80 0x7ff0080a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff010009fd0 tx=0x7ff0100096a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff002ffd640 1 -- 172.21.15.103:0/637587829 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c020050 con 0x55ac09bcb960 2024-08-19T01:44:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff002ffd640 1 -- 172.21.15.103:0/637587829 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00c09b480 con 0x55ac09bcb960 2024-08-19T01:44:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.701+0000 7ff002ffd640 1 -- 172.21.15.103:0/637587829 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c09abc0 con 0x55ac09bcb960 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 -- 172.21.15.103:0/637587829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 msgr2=0x55ac09bd0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bd0420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff00c097890 tx=0x7ff00c005880 comp rx=0 tx=0).stop 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 -- 172.21.15.103:0/637587829 shutdown_connections 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bd2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bd0420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 --2- 172.21.15.103:0/637587829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb020 0x55ac09bcb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 -- 172.21.15.103:0/637587829 >> 172.21.15.103:0/637587829 conn(0x55ac09bc6690 msgr2=0x55ac09bc6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 -- 172.21.15.103:0/637587829 shutdown_connections 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.702+0000 7ff021c2d740 1 -- 172.21.15.103:0/637587829 wait complete. 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.703+0000 7ff021c2d740 1 -- start start 2024-08-19T01:44:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.703+0000 7ff021c2d740 1 Processor -- start 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.703+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.703+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb960 0x55ac09bd56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bdc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff021c2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ac09bd3960 con 0x55ac09bcb960 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff021c2d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ac09ae1520 con 0x55ac09bd09f0 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff021c2d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ac09bd3ae0 con 0x55ac09bcb020 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58026/0 (socket says 172.21.15.103:58026) 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 -- 172.21.15.103:0/1007271659 learned_addr learned my addr 172.21.15.103:0/1007271659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff017fff640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb960 0x55ac09bd56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff021c2d740 10 client.0 _ll_register_callbacks cb 0x55ac09838a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01d40b640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bdc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 -- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 msgr2=0x55ac09bdc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bdc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 -- 172.21.15.103:0/1007271659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb960 msgr2=0x55ac09bd56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb960 0x55ac09bd56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 -- 172.21.15.103:0/1007271659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac09905620 con 0x55ac09bcb020 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01d40b640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bdc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff017fff640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb960 0x55ac09bd56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.704+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd5170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff00c09b410 tx=0x7ff00c005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1afc8b5b0acc8 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7feff7fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7feff7fff640 1 -- 172.21.15.103:0/1007271659 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c09ab40 con 0x55ac09bcb020 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7feff7fff640 1 -- 172.21.15.103:0/1007271659 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff00c0b4160 con 0x55ac09bcb020 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff00c09b480 con 0x55ac09bcb020 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7feff7fff640 1 -- 172.21.15.103:0/1007271659 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff00c020020 con 0x55ac09bcb020 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7ff021c2d740 10 client.26857 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac09bd3960 con 0x55ac09bcb020 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.705+0000 7feff7fff640 1 -- 172.21.15.103:0/1007271659 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00c09b480 con 0x55ac09bcb020 2024-08-19T01:44:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7feff7fff640 1 -- 172.21.15.103:0/1007271659 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff00c07b080 con 0x55ac09bcb020 2024-08-19T01:44:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7ff021c2d740 10 client.26857 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7ff021c2d740 10 client.26857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7ff021c2d740 20 client.26857 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09be0620 0x55ac09be2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7ff021c2d740 4 client.26857 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:44:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac09b33100 con 0x55ac09be0620 2024-08-19T01:44:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.706+0000 7ff017fff640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09be0620 0x55ac09be2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.707+0000 7feff7fff640 10 client.26857 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.707+0000 7ff017fff640 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09be0620 0x55ac09be2a10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e77d2817f96ece server_cookie=dfef1ff4987d1203 in_seq=0 out_seq=0 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.707+0000 7feff7fff640 1 -- 172.21.15.103:0/1007271659 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac09b33100 con 0x55ac09be0620 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.707+0000 7feff7fff640 10 client.26857 handle_command_reply: tid=0 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.707+0000 7ff021c2d740 1 client.26857 shutdown 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 msgr2=0x55ac09bd5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd5170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff00c09b410 tx=0x7ff00c005d30 comp rx=0 tx=0).stop 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 shutdown_connections 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09be0620 0x55ac09be2a10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bd09f0 0x55ac09bdc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7feff7fff640 0 client.26857 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bcb960 0x55ac09bd56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1007271659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd5170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 >> 172.21.15.103:0/1007271659 conn(0x55ac09bc6690 msgr2=0x55ac09bc7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 shutdown_connections 2024-08-19T01:44:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 1 -- 172.21.15.103:0/1007271659 wait complete. 2024-08-19T01:44:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.708+0000 7ff021c2d740 20 client.26857 trim_cache size 0 max 16384 2024-08-19T01:44:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 Processor -- start 2024-08-19T01:44:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 -- start start 2024-08-19T01:44:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb020 0x55ac09bd5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bdc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bc7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181af660 con 0x55ac09bd09f0 2024-08-19T01:44:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b1260 con 0x55ac09bcb020 2024-08-19T01:44:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.738+0000 7ff021c2d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181afd30 con 0x55ac09bcb960 2024-08-19T01:44:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bdc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bdc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:58040/0 (socket says 172.21.15.103:58040) 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 -- 172.21.15.103:0/1939153497 learned_addr learned my addr 172.21.15.103:0/1939153497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb020 0x55ac09bd5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff01d40b640 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bc7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 -- 172.21.15.103:0/1939153497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb020 msgr2=0x55ac09bd5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb020 0x55ac09bd5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 -- 172.21.15.103:0/1939153497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 msgr2=0x55ac09bc7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bc7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 -- 172.21.15.103:0/1939153497 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac09bc64b0 con 0x55ac09bcb960 2024-08-19T01:44:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb020 0x55ac09bd5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.739+0000 7ff017fff640 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bdc6b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff00402aa10 tx=0x7ff004005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3431485a4e70428 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.740+0000 7ff0027fc640 1 -- 172.21.15.103:0/1939153497 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004035040 con 0x55ac09bcb960 2024-08-19T01:44:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.740+0000 7ff0027fc640 1 -- 172.21.15.103:0/1939153497 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff004031c30 con 0x55ac09bcb960 2024-08-19T01:44:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.740+0000 7ff0027fc640 1 -- 172.21.15.103:0/1939153497 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004003d80 con 0x55ac09bcb960 2024-08-19T01:44:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.740+0000 7ff021c2d740 1 -- 172.21.15.103:0/1939153497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 msgr2=0x55ac09bdc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.740+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bdc6b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff00402aa10 tx=0x7ff004005a20 comp rx=0 tx=0).stop 2024-08-19T01:44:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 -- 172.21.15.103:0/1939153497 shutdown_connections 2024-08-19T01:44:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bc7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb960 0x55ac09bdc6b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1939153497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb020 0x55ac09bd5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 -- 172.21.15.103:0/1939153497 >> 172.21.15.103:0/1939153497 conn(0x55ac09bc6690 msgr2=0x55ac09bbcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 -- 172.21.15.103:0/1939153497 shutdown_connections 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 -- 172.21.15.103:0/1939153497 wait complete. 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 -- start start 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.741+0000 7ff021c2d740 1 Processor -- start 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb960 0x55ac09bda070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff021c2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bdb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff021c2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b0b90 con 0x55ac09bd09f0 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff021c2d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181b0400 con 0x55ac09bcb960 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff021c2d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181ad660 con 0x55ac09bcb020 2024-08-19T01:44:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff021c2d740 10 client.0 _ll_register_callbacks cb 0x7ff0100123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:44:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff01cc0a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff01d40b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bdb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff01d40b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bdb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43946/0 (socket says 172.21.15.103:43946) 2024-08-19T01:44:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff01d40b640 1 -- 172.21.15.103:0/1185906122 learned_addr learned my addr 172.21.15.103:0/1185906122 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:44:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.742+0000 7ff017fff640 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb960 0x55ac09bda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.743+0000 7ff01d40b640 1 -- 172.21.15.103:0/1185906122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 msgr2=0x55ac09bd9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.743+0000 7ff01d40b640 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.743+0000 7ff01d40b640 1 -- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb960 msgr2=0x55ac09bda070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.743+0000 7ff01d40b640 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb960 0x55ac09bda070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.743+0000 7ff01d40b640 1 -- 172.21.15.103:0/1185906122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac09bdb690 con 0x55ac09bd09f0 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.743+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7ff01d40b640 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bdb080 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff010009ef0 tx=0x7ff010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368a4532056151b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:44:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7feff7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7feff7fff640 1 -- 172.21.15.103:0/1185906122 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100043c0 con 0x55ac09bd09f0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7feff7fff640 1 -- 172.21.15.103:0/1185906122 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff010004560 con 0x55ac09bd09f0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff004031c30 con 0x55ac09bd09f0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7feff7fff640 1 -- 172.21.15.103:0/1185906122 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff010004760 con 0x55ac09bd09f0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7ff021c2d740 10 client.17670 fetch_fsmap learned FSMap version 39 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.744+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac09bdca10 con 0x55ac09bd09f0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7feff7fff640 1 -- 172.21.15.103:0/1185906122 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff004031c30 con 0x55ac09bd09f0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7feff7fff640 1 -- 172.21.15.103:0/1185906122 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff010029020 con 0x55ac09bd09f0 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7ff021c2d740 10 client.17670 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7ff021c2d740 10 client.17670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:44:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7ff021c2d740 20 client.17670 populate_metadata read hostname 'smithi103' 2024-08-19T01:44:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09c4ba60 0x55ac09c4de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:44:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7ff021c2d740 4 client.17670 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:44:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac09b2d450 con 0x55ac09c4ba60 2024-08-19T01:44:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.745+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09c4ba60 0x55ac09c4de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:44:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.746+0000 7ff01cc0a640 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09c4ba60 0x55ac09c4de80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce13a88ea0f30890 server_cookie=6571f68424187e75 in_seq=0 out_seq=0 2024-08-19T01:44:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.746+0000 7feff7fff640 10 client.17670 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7feff7fff640 1 -- 172.21.15.103:0/1185906122 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac09b2d450 con 0x55ac09c4ba60 2024-08-19T01:44:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7feff7fff640 10 client.17670 handle_command_reply: tid=0 2024-08-19T01:44:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 client.17670 shutdown 2024-08-19T01:44:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 msgr2=0x55ac09bdb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bdb080 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff010009ef0 tx=0x7ff010004a40 comp rx=0 tx=0).stop 2024-08-19T01:44:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 shutdown_connections 2024-08-19T01:44:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ac09c4ba60 0x55ac09c4de80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac09bd09f0 0x55ac09bdb080 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ac09bcb960 0x55ac09bda070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 --2- 172.21.15.103:0/1185906122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac09bcb020 0x55ac09bd9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 >> 172.21.15.103:0/1185906122 conn(0x55ac09bc6690 msgr2=0x55ac09bbcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7feff7fff640 0 client.17670 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 shutdown_connections 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 1 -- 172.21.15.103:0/1185906122 wait complete. 2024-08-19T01:44:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.758+0000 7ff021c2d740 20 client.17670 trim_cache size 0 max 16384 2024-08-19T01:44:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:44:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:44:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:44:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0080a2b80 msgr2=0x7ff0080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0080a2b80 0x7ff0080a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff010009fd0 tx=0x7ff0100096a0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff018073a60 msgr2=0x7ff0181a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff018073a60 0x7ff0181a2560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00d950 tx=0x7ff00c00de10 comp rx=0 tx=0).stop 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 shutdown_connections 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff0080a2b80 0x7ff0080a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0181a7b00 0x7ff0181a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff018109190 0x7ff0181a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 --2- 172.21.15.103:0/659631565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff018073a60 0x7ff0181a2560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:44:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 >> 172.21.15.103:0/659631565 conn(0x7ff01806ea40 msgr2=0x7ff01806f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:44:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.760+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 shutdown_connections 2024-08-19T01:44:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:44:54.761+0000 7ff01dc0c640 1 -- 172.21.15.103:0/659631565 wait complete. 2024-08-19T01:44:55.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:55 smithi103 ceph-mon[18436]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 7.19k op/s 2024-08-19T01:44:55.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:55 smithi190 ceph-mon[22032]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 7.19k op/s 2024-08-19T01:44:55.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:55 smithi146 ceph-mon[22098]: pgmap v613: 97 pgs: 97 active+clean; 13 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 7.19k op/s 2024-08-19T01:44:57.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:57 smithi103 ceph-mon[18436]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 7.27k op/s 2024-08-19T01:44:57.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:57 smithi190 ceph-mon[22032]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 7.27k op/s 2024-08-19T01:44:57.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:57 smithi146 ceph-mon[22098]: pgmap v614: 97 pgs: 97 active+clean; 13 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 7.27k op/s 2024-08-19T01:44:58.044 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-19T01:44:58.571 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:44:59.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:44:59 smithi103 ceph-mon[18436]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 7.10k op/s 2024-08-19T01:44:59.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:44:59 smithi190 ceph-mon[22032]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 7.10k op/s 2024-08-19T01:44:59.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:44:59 smithi146 ceph-mon[22098]: pgmap v615: 97 pgs: 97 active+clean; 13 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 7.10k op/s 2024-08-19T01:45:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:01 smithi103 ceph-mon[18436]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 28 MiB/s wr, 7.16k op/s 2024-08-19T01:45:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:01 smithi190 ceph-mon[22032]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 28 MiB/s wr, 7.16k op/s 2024-08-19T01:45:01.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:01 smithi146 ceph-mon[22098]: pgmap v616: 97 pgs: 97 active+clean; 13 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 28 MiB/s wr, 7.16k op/s 2024-08-19T01:45:03.294 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:03 smithi103 ceph-mon[18436]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 7.99k op/s 2024-08-19T01:45:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:03 smithi190 ceph-mon[22032]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 7.99k op/s 2024-08-19T01:45:03.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:03 smithi146 ceph-mon[22098]: pgmap v617: 97 pgs: 97 active+clean; 13 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 7.99k op/s 2024-08-19T01:45:04.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.235+0000 7f71f715f640 1 -- 172.21.15.103:0/1331506594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f010d250 msgr2=0x7f71f0114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.235+0000 7f71f715f640 1 --2- 172.21.15.103:0/1331506594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f010d250 0x7f71f0114dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009be0 tx=0x7f71e802a6c0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.236+0000 7f71f715f640 1 -- 172.21.15.103:0/1331506594 shutdown_connections 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.236+0000 7f71f715f640 1 --2- 172.21.15.103:0/1331506594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f010d250 0x7f71f0114dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.236+0000 7f71f715f640 1 --2- 172.21.15.103:0/1331506594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71f0108ad0 0x7f71f010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.236+0000 7f71f715f640 1 --2- 172.21.15.103:0/1331506594 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108120 0x7f71f0108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.236+0000 7f71f715f640 1 -- 172.21.15.103:0/1331506594 >> 172.21.15.103:0/1331506594 conn(0x7f71f0077820 msgr2=0x7f71f0079c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.236+0000 7f71f715f640 1 -- 172.21.15.103:0/1331506594 shutdown_connections 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.236+0000 7f71f715f640 1 -- 172.21.15.103:0/1331506594 wait complete. 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 Processor -- start 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 -- start start 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71f0108120 0x7f71f0073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108ad0 0x7f71f006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f010d250 0x7f71f006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0118130 con 0x7f71f0108120 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0117fb0 con 0x7f71f0108ad0 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71f715f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01182b0 con 0x7f71f010d250 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71effff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108ad0 0x7f71f006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71effff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108ad0 0x7f71f006c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:33324/0 (socket says 172.21.15.103:33324) 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.237+0000 7f71effff640 1 -- 172.21.15.103:0/3645064292 learned_addr learned my addr 172.21.15.103:0/3645064292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.238+0000 7f71effff640 1 -- 172.21.15.103:0/3645064292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f010d250 msgr2=0x7f71f006cd00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.238+0000 7f71effff640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f010d250 0x7f71f006cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.238+0000 7f71effff640 1 -- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71f0108120 msgr2=0x7f71f0073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.238+0000 7f71effff640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71f0108120 0x7f71f0073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.238+0000 7f71effff640 1 -- 172.21.15.103:0/3645064292 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f006d2c0 con 0x7f71f0108ad0 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.239+0000 7f71effff640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108ad0 0x7f71f006c7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e9f0 tx=0x7f71dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:04.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.239+0000 7f71edffb640 1 -- 172.21.15.103:0/3645064292 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00ccc0 con 0x7f71f0108ad0 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.239+0000 7f71edffb640 1 -- 172.21.15.103:0/3645064292 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71dc0108e0 con 0x7f71f0108ad0 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.239+0000 7f71edffb640 1 -- 172.21.15.103:0/3645064292 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc018880 con 0x7f71f0108ad0 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.239+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f006d5b0 con 0x7f71f0108ad0 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.240+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f00714c0 con 0x7f71f0108ad0 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.243+0000 7f71edffb640 1 -- 172.21.15.103:0/3645064292 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f71dc010430 con 0x7f71f0108ad0 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.243+0000 7f71f955d740 1 Processor -- start 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f955d740 1 -- start start 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f955d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f955d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f955d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f955d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563ca5ee7980 con 0x563ca5f82b30 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f955d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563ca5e7fa70 con 0x563ca5f82210 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f955d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563ca5ee3940 con 0x563ca5f8d3f0 2024-08-19T01:45:04.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71edffb640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f71e00a2cd0 0x7f71e00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f56fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71effff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38242/0 (socket says 172.21.15.103:38242) 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f56fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f825f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:33336/0 (socket says 172.21.15.103:33336) 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f4efb640 1 -- 172.21.15.103:0/4273494909 learned_addr learned my addr 172.21.15.103:0/4273494909 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71f56fc640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f71e00a2cd0 0x7f71e00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.244+0000 7f71edffb640 1 -- 172.21.15.103:0/3645064292 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f71dc0cf270 con 0x7f71f0108ad0 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.245+0000 7f71f56fc640 1 -- 172.21.15.103:0/4273494909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 msgr2=0x563ca5f8f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.245+0000 7f71f56fc640 1 --2- 172.21.15.103:0/4273494909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.245+0000 7f71f56fc640 1 -- 172.21.15.103:0/4273494909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 msgr2=0x563ca5f8ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.245+0000 7f71f56fc640 1 --2- 172.21.15.103:0/4273494909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.245+0000 7f71f56fc640 1 -- 172.21.15.103:0/4273494909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca5ee26e0 con 0x563ca5f82210 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.245+0000 7f71f56fc640 1 --2- 172.21.15.103:0/4273494909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f825f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009b30 tx=0x7f71e8030880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5878906444be8be6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.246+0000 7f71daffd640 1 -- 172.21.15.103:0/4273494909 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e804d070 con 0x563ca5f82210 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.246+0000 7f71daffd640 1 -- 172.21.15.103:0/4273494909 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e8031bb0 con 0x563ca5f82210 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.246+0000 7f71daffd640 1 -- 172.21.15.103:0/4273494909 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8036040 con 0x563ca5f82210 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f955d740 1 -- 172.21.15.103:0/4273494909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 msgr2=0x563ca5f825f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f955d740 1 --2- 172.21.15.103:0/4273494909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f825f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009b30 tx=0x7f71e8030880 comp rx=0 tx=0).stop 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f56fc640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f71e00a2cd0 0x7f71e00a5190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f71e8037000 tx=0x7f71e8044000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f955d740 1 -- 172.21.15.103:0/4273494909 shutdown_connections 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f955d740 1 --2- 172.21.15.103:0/4273494909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f955d740 1 --2- 172.21.15.103:0/4273494909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f955d740 1 --2- 172.21.15.103:0/4273494909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f825f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.247+0000 7f71f955d740 1 -- 172.21.15.103:0/4273494909 >> 172.21.15.103:0/4273494909 conn(0x563ca5f7e7a0 msgr2=0x563ca5f7eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.248+0000 7f71f955d740 1 -- 172.21.15.103:0/4273494909 shutdown_connections 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.248+0000 7f71f955d740 1 -- 172.21.15.103:0/4273494909 wait complete. 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.248+0000 7f71f955d740 1 -- start start 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.248+0000 7f71f955d740 1 Processor -- start 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f955d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f8a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f955d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f955d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f955d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563ca5f8ffc0 con 0x563ca5f82b30 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f955d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563ca5ea2aa0 con 0x563ca5f82210 2024-08-19T01:45:04.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f955d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563ca5f90140 con 0x563ca5f8d3f0 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f955d740 10 client.0 _ll_register_callbacks cb 0x563ca5e91e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f8a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f8a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:33346/0 (socket says 172.21.15.103:33346) 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f4efb640 1 -- 172.21.15.103:0/745097879 learned_addr learned my addr 172.21.15.103:0/745097879 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71f56fc640 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.249+0000 7f71effff640 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71f4efb640 1 -- 172.21.15.103:0/745097879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 msgr2=0x563ca5f8ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71f4efb640 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71f4efb640 1 -- 172.21.15.103:0/745097879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 msgr2=0x563ca5f8a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71f4efb640 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71f4efb640 1 -- 172.21.15.103:0/745097879 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca5e86650 con 0x563ca5f82210 2024-08-19T01:45:04.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71f4efb640 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f8a3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f71e40116f0 tx=0x7f71e4011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a214bc71769262a server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71cffff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71cffff640 1 -- 172.21.15.103:0/745097879 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e401e060 con 0x563ca5f82210 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e8031bb0 con 0x563ca5f82210 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.250+0000 7f71cffff640 1 -- 172.21.15.103:0/745097879 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f71e40035f0 con 0x563ca5f82210 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.251+0000 7f71cffff640 1 -- 172.21.15.103:0/745097879 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71e40037f0 con 0x563ca5f82210 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.251+0000 7f71f955d740 10 client.26924 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.251+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563ca5c593c0 con 0x563ca5f82210 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.251+0000 7f71cffff640 1 -- 172.21.15.103:0/745097879 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4003e40 con 0x563ca5f82210 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71cffff640 1 -- 172.21.15.103:0/745097879 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f71e400fcb0 con 0x563ca5f82210 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71f955d740 10 client.26924 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71f955d740 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71f955d740 20 client.26924 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71f955d740 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca5f9d380 0x563ca5f9f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71f955d740 4 client.26924 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f71e8031bb0 con 0x563ca5f9d380 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.252+0000 7f71effff640 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca5f9d380 0x563ca5f9f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.253+0000 7f71effff640 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca5f9d380 0x563ca5f9f820 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a55f8d89b67724 server_cookie=8608e0cf499d3ddd in_seq=0 out_seq=0 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.253+0000 7f71cffff640 10 client.26924 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.253+0000 7f71cffff640 1 -- 172.21.15.103:0/745097879 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f71e8031bb0 con 0x563ca5f9d380 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.253+0000 7f71cffff640 10 client.26924 handle_command_reply: tid=0 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 client.26924 shutdown 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 msgr2=0x563ca5f8a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f8a3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f71e40116f0 tx=0x7f71e4011af0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 shutdown_connections 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca5f9d380 0x563ca5f9f820 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f8ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f8a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 --2- 172.21.15.103:0/745097879 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f82210 0x563ca5f8a3b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 >> 172.21.15.103:0/745097879 conn(0x563ca5f7e7a0 msgr2=0x563ca5f844d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.254+0000 7f71cffff640 0 client.26924 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.255+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 shutdown_connections 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.255+0000 7f71f955d740 1 -- 172.21.15.103:0/745097879 wait complete. 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.255+0000 7f71f955d740 20 client.26924 trim_cache size 0 max 16384 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.286+0000 7f71f955d740 1 Processor -- start 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f955d740 1 -- start start 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f955d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f84cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f955d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8d3f0 0x563ca5f88c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f955d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8c210 0x563ca5fa50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f955d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01aafe0 con 0x563ca5f82b30 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f955d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01acbe0 con 0x563ca5f8d3f0 2024-08-19T01:45:04.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f955d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01ab6b0 con 0x563ca5f8c210 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f84cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f84cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38270/0 (socket says 172.21.15.103:38270) 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f4efb640 1 -- 172.21.15.103:0/2995737440 learned_addr learned my addr 172.21.15.103:0/2995737440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71effff640 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8d3f0 0x563ca5f88c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.287+0000 7f71f56fc640 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8c210 0x563ca5fa50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71f4efb640 1 -- 172.21.15.103:0/2995737440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8c210 msgr2=0x563ca5fa50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71f4efb640 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8c210 0x563ca5fa50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71f4efb640 1 -- 172.21.15.103:0/2995737440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8d3f0 msgr2=0x563ca5f88c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71f4efb640 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8d3f0 0x563ca5f88c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71f4efb640 1 -- 172.21.15.103:0/2995737440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca5f89360 con 0x563ca5f82b30 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71effff640 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8d3f0 0x563ca5f88c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71f4efb640 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f84cc0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f71e4019480 tx=0x7f71e4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1479fbf0d72f7d52 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71da7fc640 1 -- 172.21.15.103:0/2995737440 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4031020 con 0x563ca5f82b30 2024-08-19T01:45:04.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71da7fc640 1 -- 172.21.15.103:0/2995737440 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4003e40 con 0x563ca5f82b30 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.288+0000 7f71da7fc640 1 -- 172.21.15.103:0/2995737440 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4029040 con 0x563ca5f82b30 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 -- 172.21.15.103:0/2995737440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 msgr2=0x563ca5f84cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f84cc0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f71e4019480 tx=0x7f71e4001e50 comp rx=0 tx=0).stop 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 -- 172.21.15.103:0/2995737440 shutdown_connections 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8c210 0x563ca5fa50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8d3f0 0x563ca5f88c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 --2- 172.21.15.103:0/2995737440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f84cc0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 -- 172.21.15.103:0/2995737440 >> 172.21.15.103:0/2995737440 conn(0x563ca5f7e7a0 msgr2=0x563ca5f8bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 -- 172.21.15.103:0/2995737440 shutdown_connections 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.289+0000 7f71f955d740 1 -- 172.21.15.103:0/2995737440 wait complete. 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 -- start start 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 Processor -- start 2024-08-19T01:45:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8c210 0x563ca5f915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f91b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01ac510 con 0x563ca5f82b30 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01abd80 con 0x563ca5f8c210 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.290+0000 7f71f955d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01a8fc0 con 0x563ca5f8d3f0 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f955d740 10 client.0 _ll_register_callbacks cb 0x7f71e4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f91090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38280/0 (socket says 172.21.15.103:38280) 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 -- 172.21.15.103:0/14754929 learned_addr learned my addr 172.21.15.103:0/14754929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f56fc640 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f91b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71effff640 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8c210 0x563ca5f915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 -- 172.21.15.103:0/14754929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 msgr2=0x563ca5f91b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f91b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 -- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8c210 msgr2=0x563ca5f915d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8c210 0x563ca5f915d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 -- 172.21.15.103:0/14754929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca60070c0 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71f4efb640 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f91090 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f71e4013010 tx=0x7f71e4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fd437eb4c03a64 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.291+0000 7f71cffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71cffff640 1 -- 172.21.15.103:0/14754929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4029070 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71cffff640 1 -- 172.21.15.103:0/14754929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f71e4009df0 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e4003e40 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71cffff640 1 -- 172.21.15.103:0/14754929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71e4031040 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71f955d740 10 client.17685 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563ca6008310 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71cffff640 1 -- 172.21.15.103:0/14754929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4003e40 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71cffff640 1 -- 172.21.15.103:0/14754929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f71e403d030 con 0x563ca5f82b30 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.292+0000 7f71f955d740 10 client.17685 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71f955d740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71f955d740 20 client.17685 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71f955d740 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca60089c0 0x563ca600adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71f955d740 4 client.17685 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ca5ef4ef0 con 0x563ca60089c0 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71effff640 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca60089c0 0x563ca600adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71effff640 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca60089c0 0x563ca600adb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fac401a2c9a346e server_cookie=a0547eb89547bc7b in_seq=0 out_seq=0 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.293+0000 7f71cffff640 10 client.17685 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71cffff640 1 -- 172.21.15.103:0/14754929 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ca5ef4ef0 con 0x563ca60089c0 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71cffff640 10 client.17685 handle_command_reply: tid=0 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 client.17685 shutdown 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 msgr2=0x563ca5f91090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f91090 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f71e4013010 tx=0x7f71e4005c50 comp rx=0 tx=0).stop 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 shutdown_connections 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x563ca60089c0 0x563ca600adb0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563ca5f8d3f0 0x563ca5f91b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563ca5f8c210 0x563ca5f915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71cffff640 0 client.17685 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 --2- 172.21.15.103:0/14754929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563ca5f82b30 0x563ca5f91090 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 >> 172.21.15.103:0/14754929 conn(0x563ca5f7e7a0 msgr2=0x563ca5f85630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 shutdown_connections 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 1 -- 172.21.15.103:0/14754929 wait complete. 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.294+0000 7f71f955d740 20 client.17685 trim_cache size 0 max 16384 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:45:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f71e00a2cd0 msgr2=0x7f71e00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f71e00a2cd0 0x7f71e00a5190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f71e8037000 tx=0x7f71e8044000 comp rx=0 tx=0).stop 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108ad0 msgr2=0x7f71f006c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108ad0 0x7f71f006c7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e9f0 tx=0x7f71dc00eeb0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 shutdown_connections 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f71e00a2cd0 0x7f71e00a5190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f010d250 0x7f71f006cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71f0108ad0 0x7f71f006c7c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 --2- 172.21.15.103:0/3645064292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71f0108120 0x7f71f0073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.295+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 >> 172.21.15.103:0/3645064292 conn(0x7f71f0077820 msgr2=0x7f71f0078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:04.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.296+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 shutdown_connections 2024-08-19T01:45:04.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:03.296+0000 7f71f715f640 1 -- 172.21.15.103:0/3645064292 wait complete. 2024-08-19T01:45:05.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:05 smithi103 ceph-mon[18436]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 6.87k op/s 2024-08-19T01:45:05.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:05 smithi190 ceph-mon[22032]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 6.87k op/s 2024-08-19T01:45:05.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:05 smithi146 ceph-mon[22098]: pgmap v618: 97 pgs: 97 active+clean; 13 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 27 MiB/s wr, 6.87k op/s 2024-08-19T01:45:07.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:07 smithi103 ceph-mon[18436]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 6.73k op/s 2024-08-19T01:45:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:07 smithi190 ceph-mon[22032]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 6.73k op/s 2024-08-19T01:45:07.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:07 smithi146 ceph-mon[22098]: pgmap v619: 97 pgs: 97 active+clean; 13 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 6.73k op/s 2024-08-19T01:45:07.903 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-19T01:45:07.903 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:45:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc9327fc640 1 -- 172.21.15.103:0/3205884623 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924035020 con 0x7fc934109870 2024-08-19T01:45:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc938870640 1 -- 172.21.15.103:0/3205884623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934109870 msgr2=0x7fc934110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc938870640 1 --2- 172.21.15.103:0/3205884623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934109870 0x7fc934110100 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92402a410 comp rx=0 tx=0).stop 2024-08-19T01:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc938870640 1 -- 172.21.15.103:0/3205884623 shutdown_connections 2024-08-19T01:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc938870640 1 --2- 172.21.15.103:0/3205884623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934109870 0x7fc934110100 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc938870640 1 --2- 172.21.15.103:0/3205884623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc934108eb0 0x7fc934109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc938870640 1 --2- 172.21.15.103:0/3205884623 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9341081a0 0x7fc9341085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.952+0000 7fc938870640 1 -- 172.21.15.103:0/3205884623 >> 172.21.15.103:0/3205884623 conn(0x7fc9340771a0 msgr2=0x7fc9340795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.953+0000 7fc938870640 1 -- 172.21.15.103:0/3205884623 shutdown_connections 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.953+0000 7fc938870640 1 -- 172.21.15.103:0/3205884623 wait complete. 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.954+0000 7fc938870640 1 Processor -- start 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.954+0000 7fc938870640 1 -- start start 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.954+0000 7fc938870640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9341081a0 0x7fc93407f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc938870640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934108eb0 0x7fc93407fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9341081a0 0x7fc93407f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9341081a0 0x7fc93407f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43200/0 (socket says 172.21.15.103:43200) 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc9337fe640 1 -- 172.21.15.103:0/365843352 learned_addr learned my addr 172.21.15.103:0/365843352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc932ffd640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934108eb0 0x7fc93407fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc938870640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc934109870 0x7fc9341adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc934113820 con 0x7fc934108eb0 2024-08-19T01:45:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341136a0 con 0x7fc934109870 2024-08-19T01:45:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.955+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341139a0 con 0x7fc9341081a0 2024-08-19T01:45:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc932ffd640 1 -- 172.21.15.103:0/365843352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9341081a0 msgr2=0x7fc93407f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc932ffd640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9341081a0 0x7fc93407f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc932ffd640 1 -- 172.21.15.103:0/365843352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc934109870 msgr2=0x7fc9341adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc933fff640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc934109870 0x7fc9341adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc932ffd640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc934109870 0x7fc9341adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc932ffd640 1 -- 172.21.15.103:0/365843352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc934082d20 con 0x7fc934108eb0 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc933fff640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc934109870 0x7fc9341adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc9337fe640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9341081a0 0x7fc93407f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.956+0000 7fc932ffd640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934108eb0 0x7fc93407fbf0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c970 tx=0x7fc91c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.957+0000 7fc930ff9640 1 -- 172.21.15.103:0/365843352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c007bb0 con 0x7fc934108eb0 2024-08-19T01:45:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.957+0000 7fc930ff9640 1 -- 172.21.15.103:0/365843352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc91c007d50 con 0x7fc934108eb0 2024-08-19T01:45:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.958+0000 7fc930ff9640 1 -- 172.21.15.103:0/365843352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c018820 con 0x7fc934108eb0 2024-08-19T01:45:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.958+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9341ae530 con 0x7fc934108eb0 2024-08-19T01:45:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.958+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341ae9f0 con 0x7fc934108eb0 2024-08-19T01:45:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.961+0000 7fc93c88b740 1 Processor -- start 2024-08-19T01:45:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.962+0000 7fc93c88b740 1 -- start start 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.962+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d931810 0x56390d931c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.962+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d932150 0x56390d936c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.962+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d9371e0 0x56390d9396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.962+0000 7fc93c88b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56390d8949d0 con 0x56390d931810 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.962+0000 7fc93c88b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56390d83c820 con 0x56390d9371e0 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.962+0000 7fc93c88b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56390d890990 con 0x56390d932150 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d932150 0x56390d936c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d932150 0x56390d936c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43212/0 (socket says 172.21.15.103:43212) 2024-08-19T01:45:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 -- 172.21.15.103:0/846332779 learned_addr learned my addr 172.21.15.103:0/846332779 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc933fff640 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d931810 0x56390d931c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc932ffd640 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d9371e0 0x56390d9396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 -- 172.21.15.103:0/846332779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d9371e0 msgr2=0x56390d9396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d9371e0 0x56390d9396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 -- 172.21.15.103:0/846332779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d931810 msgr2=0x56390d931c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d931810 0x56390d931c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.963+0000 7fc9337fe640 1 -- 172.21.15.103:0/846332779 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56390d5a8210 con 0x56390d932150 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.964+0000 7fc933fff640 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d931810 0x56390d931c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.964+0000 7fc9337fe640 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d932150 0x56390d936c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e880 tx=0x7fc92c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c075f101fc7ad25 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.964+0000 7fc930ff9640 1 -- 172.21.15.103:0/365843352 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc91c0189c0 con 0x7fc934108eb0 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.965+0000 7fc929ffb640 1 -- 172.21.15.103:0/846332779 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c004d20 con 0x56390d932150 2024-08-19T01:45:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.965+0000 7fc929ffb640 1 -- 172.21.15.103:0/846332779 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92c003970 con 0x56390d932150 2024-08-19T01:45:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.966+0000 7fc93c88b740 1 -- 172.21.15.103:0/846332779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d932150 msgr2=0x56390d936c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.966+0000 7fc93c88b740 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d932150 0x56390d936c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e880 tx=0x7fc92c00ed40 comp rx=0 tx=0).stop 2024-08-19T01:45:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc93c88b740 1 -- 172.21.15.103:0/846332779 shutdown_connections 2024-08-19T01:45:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc93c88b740 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d9371e0 0x56390d9396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc93c88b740 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d932150 0x56390d936c10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc93c88b740 1 --2- 172.21.15.103:0/846332779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d931810 0x56390d931c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc93c88b740 1 -- 172.21.15.103:0/846332779 >> 172.21.15.103:0/846332779 conn(0x56390d92ce80 msgr2=0x56390d92d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc930ff9640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc9200a2d50 0x7fc9200a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc93c88b740 1 -- 172.21.15.103:0/846332779 shutdown_connections 2024-08-19T01:45:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.967+0000 7fc93c88b740 1 -- 172.21.15.103:0/846332779 wait complete. 2024-08-19T01:45:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.968+0000 7fc93c88b740 1 -- start start 2024-08-19T01:45:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.968+0000 7fc93c88b740 1 Processor -- start 2024-08-19T01:45:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.968+0000 7fc9337fe640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc9200a2d50 0x7fc9200a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.968+0000 7fc930ff9640 1 -- 172.21.15.103:0/365843352 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc91c0cefc0 con 0x7fc934108eb0 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.968+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d931810 0x56390d93b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 0x56390d93bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d942ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc93c88b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56390d93a200 con 0x56390d932150 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc93c88b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56390d850520 con 0x56390d931810 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc93c88b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56390d93a380 con 0x56390d9371e0 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc93c88b740 10 client.0 _ll_register_callbacks cb 0x56390d5a7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d931810 0x56390d93b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d942ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d942ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43216/0 (socket says 172.21.15.103:43216) 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 -- 172.21.15.103:0/821382957 learned_addr learned my addr 172.21.15.103:0/821382957 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc932ffd640 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 0x56390d93bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 -- 172.21.15.103:0/821382957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d931810 msgr2=0x56390d93b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d931810 0x56390d93b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 -- 172.21.15.103:0/821382957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 msgr2=0x56390d93bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 0x56390d93bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.969+0000 7fc933fff640 1 -- 172.21.15.103:0/821382957 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56390d674620 con 0x56390d9371e0 2024-08-19T01:45:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.970+0000 7fc9337fe640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc9200a2d50 0x7fc9200a5210 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009f20 tx=0x7fc92c00ef90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.970+0000 7fc933fff640 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d942ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9240042a0 tx=0x7fc9240042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8e59087ab54ce49 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.970+0000 7fc912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:45:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.970+0000 7fc912ffd640 1 -- 172.21.15.103:0/821382957 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924040070 con 0x56390d9371e0 2024-08-19T01:45:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.970+0000 7fc912ffd640 1 -- 172.21.15.103:0/821382957 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9240318d0 con 0x56390d9371e0 2024-08-19T01:45:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.970+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92c003970 con 0x56390d9371e0 2024-08-19T01:45:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.971+0000 7fc912ffd640 1 -- 172.21.15.103:0/821382957 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc924035040 con 0x56390d9371e0 2024-08-19T01:45:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.971+0000 7fc93c88b740 10 client.26887 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.971+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56390d956da0 con 0x56390d9371e0 2024-08-19T01:45:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc912ffd640 1 -- 172.21.15.103:0/821382957 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92c003970 con 0x56390d9371e0 2024-08-19T01:45:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc912ffd640 1 -- 172.21.15.103:0/821382957 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc92403c440 con 0x56390d9371e0 2024-08-19T01:45:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc93c88b740 10 client.26887 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc93c88b740 10 client.26887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc93c88b740 20 client.26887 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc93c88b740 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d958400 0x56390d95a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc93c88b740 4 client.26887 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56390d8a2100 con 0x56390d958400 2024-08-19T01:45:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.973+0000 7fc9337fe640 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d958400 0x56390d95a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.974+0000 7fc9337fe640 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d958400 0x56390d95a810 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd82bc532e4937 server_cookie=aeab0420008e519b in_seq=0 out_seq=0 2024-08-19T01:45:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.974+0000 7fc912ffd640 10 client.26887 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc912ffd640 1 -- 172.21.15.103:0/821382957 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56390d8a2100 con 0x56390d958400 2024-08-19T01:45:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc912ffd640 10 client.26887 handle_command_reply: tid=0 2024-08-19T01:45:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 client.26887 shutdown 2024-08-19T01:45:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 msgr2=0x56390d942ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d942ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9240042a0 tx=0x7fc9240042d0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 shutdown_connections 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d958400 0x56390d95a810 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d942ee0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 0x56390d93bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 --2- 172.21.15.103:0/821382957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d931810 0x56390d93b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 >> 172.21.15.103:0/821382957 conn(0x56390d92ce80 msgr2=0x56390d92e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.975+0000 7fc912ffd640 0 client.26887 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.976+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 shutdown_connections 2024-08-19T01:45:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.976+0000 7fc93c88b740 1 -- 172.21.15.103:0/821382957 wait complete. 2024-08-19T01:45:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:07.976+0000 7fc93c88b740 20 client.26887 trim_cache size 0 max 16384 2024-08-19T01:45:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.010+0000 7fc93c88b740 1 Processor -- start 2024-08-19T01:45:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.011+0000 7fc93c88b740 1 -- start start 2024-08-19T01:45:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.011+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d931810 0x56390d92b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.011+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d932150 0x56390d942e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.011+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d9371e0 0x56390d92bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d931810 0x56390d92b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc9337fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d931810 0x56390d92b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43218/0 (socket says 172.21.15.103:43218) 2024-08-19T01:45:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d932150 0x56390d942e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d932150 0x56390d942e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:33422/0 (socket says 172.21.15.103:33422) 2024-08-19T01:45:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc9337fe640 1 -- 172.21.15.103:0/4290433900 learned_addr learned my addr 172.21.15.103:0/4290433900 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341aef00 con 0x56390d9371e0 2024-08-19T01:45:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b0b00 con 0x56390d932150 2024-08-19T01:45:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.012+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341af5d0 con 0x56390d931810 2024-08-19T01:45:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.013+0000 7fc933fff640 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d9371e0 0x56390d92bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.013+0000 7fc9337fe640 1 -- 172.21.15.103:0/4290433900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d932150 msgr2=0x56390d942e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.013+0000 7fc9337fe640 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d932150 0x56390d942e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.013+0000 7fc9337fe640 1 -- 172.21.15.103:0/4290433900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d9371e0 msgr2=0x56390d92bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.013+0000 7fc9337fe640 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d9371e0 0x56390d92bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.013+0000 7fc9337fe640 1 -- 172.21.15.103:0/4290433900 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56390d92cca0 con 0x56390d931810 2024-08-19T01:45:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.014+0000 7fc9337fe640 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d931810 0x56390d92b730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56390d932b80 tx=0x7fc92c029000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d941962b607022 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.014+0000 7fc9297fa640 1 -- 172.21.15.103:0/4290433900 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c003970 con 0x56390d931810 2024-08-19T01:45:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.014+0000 7fc9297fa640 1 -- 172.21.15.103:0/4290433900 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92c01f040 con 0x56390d931810 2024-08-19T01:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.015+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d931810 msgr2=0x56390d92b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.015+0000 7fc93c88b740 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d931810 0x56390d92b730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56390d932b80 tx=0x7fc92c029000 comp rx=0 tx=0).stop 2024-08-19T01:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.015+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 shutdown_connections 2024-08-19T01:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.015+0000 7fc93c88b740 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d9371e0 0x56390d92bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.015+0000 7fc93c88b740 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d932150 0x56390d942e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.015+0000 7fc93c88b740 1 --2- 172.21.15.103:0/4290433900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d931810 0x56390d92b730 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.015+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 >> 172.21.15.103:0/4290433900 conn(0x56390d92ce80 msgr2=0x56390d93b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.016+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 shutdown_connections 2024-08-19T01:45:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.016+0000 7fc93c88b740 1 -- 172.21.15.103:0/4290433900 wait complete. 2024-08-19T01:45:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.016+0000 7fc93c88b740 1 -- start start 2024-08-19T01:45:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.016+0000 7fc93c88b740 1 Processor -- start 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 0x56390d940460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d9409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc93c88b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d941960 0x56390d941dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc93c88b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b0430 con 0x56390d932150 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc93c88b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341afca0 con 0x56390d941960 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc93c88b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9340722d0 con 0x56390d9371e0 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc93c88b740 10 client.0 _ll_register_callbacks cb 0x7fc92c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc933fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d941960 0x56390d941dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc932ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d9409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc933fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d941960 0x56390d941dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:33430/0 (socket says 172.21.15.103:33430) 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.017+0000 7fc933fff640 1 -- 172.21.15.103:0/164831651 learned_addr learned my addr 172.21.15.103:0/164831651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.018+0000 7fc933fff640 1 -- 172.21.15.103:0/164831651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 msgr2=0x56390d9409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.018+0000 7fc933fff640 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d9409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.018+0000 7fc933fff640 1 -- 172.21.15.103:0/164831651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 msgr2=0x56390d940460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.018+0000 7fc933fff640 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 0x56390d940460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.018+0000 7fc933fff640 1 -- 172.21.15.103:0/164831651 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56390d942350 con 0x56390d941960 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.018+0000 7fc933fff640 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d941960 0x56390d941dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92403cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=539b9a88cb12b38 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.018+0000 7fc912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:45:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.020+0000 7fc912ffd640 1 -- 172.21.15.103:0/164831651 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924053070 con 0x56390d941960 2024-08-19T01:45:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.020+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56390d89c450 con 0x56390d941960 2024-08-19T01:45:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.020+0000 7fc912ffd640 1 -- 172.21.15.103:0/164831651 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc9240318d0 con 0x56390d941960 2024-08-19T01:45:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.020+0000 7fc912ffd640 1 -- 172.21.15.103:0/164831651 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc92405f070 con 0x56390d941960 2024-08-19T01:45:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.020+0000 7fc93c88b740 10 client.26942 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.020+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56390d9c3500 con 0x56390d941960 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc912ffd640 1 -- 172.21.15.103:0/164831651 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56390d89c450 con 0x56390d941960 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc912ffd640 1 -- 172.21.15.103:0/164831651 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc924003ae0 con 0x56390d941960 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc93c88b740 10 client.26942 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc93c88b740 10 client.26942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc93c88b740 20 client.26942 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc93c88b740 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d9c3ee0 0x56390d9c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc93c88b740 4 client.26942 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:45:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56390d819900 con 0x56390d9c3ee0 2024-08-19T01:45:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.021+0000 7fc9337fe640 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d9c3ee0 0x56390d9c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.022+0000 7fc9337fe640 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d9c3ee0 0x56390d9c6300 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704b2e3eee6756a4 server_cookie=6ac08859e16f2dc3 in_seq=0 out_seq=0 2024-08-19T01:45:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.023+0000 7fc912ffd640 10 client.26942 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc912ffd640 1 -- 172.21.15.103:0/164831651 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56390d819900 con 0x56390d9c3ee0 2024-08-19T01:45:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc912ffd640 10 client.26942 handle_command_reply: tid=0 2024-08-19T01:45:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc93c88b740 1 client.26942 shutdown 2024-08-19T01:45:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d941960 msgr2=0x56390d941dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc93c88b740 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d941960 0x56390d941dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92403cc40 comp rx=0 tx=0).stop 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 shutdown_connections 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc93c88b740 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x56390d9c3ee0 0x56390d9c6300 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc93c88b740 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56390d941960 0x56390d941dc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.038+0000 7fc93c88b740 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56390d9371e0 0x56390d9409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.039+0000 7fc93c88b740 1 --2- 172.21.15.103:0/164831651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56390d932150 0x56390d940460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.039+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 >> 172.21.15.103:0/164831651 conn(0x56390d92ce80 msgr2=0x56390d92dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.039+0000 7fc912ffd640 0 client.26942 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.039+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 shutdown_connections 2024-08-19T01:45:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.039+0000 7fc93c88b740 1 -- 172.21.15.103:0/164831651 wait complete. 2024-08-19T01:45:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:45:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:45:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:45:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:45:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.039+0000 7fc93c88b740 20 client.26942 trim_cache size 0 max 16384 2024-08-19T01:45:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.046+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc9200a2d50 msgr2=0x7fc9200a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.046+0000 7fc938870640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc9200a2d50 0x7fc9200a5210 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009f20 tx=0x7fc92c00ef90 comp rx=0 tx=0).stop 2024-08-19T01:45:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.046+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934108eb0 msgr2=0x7fc93407fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.046+0000 7fc938870640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934108eb0 0x7fc93407fbf0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c970 tx=0x7fc91c00ce30 comp rx=0 tx=0).stop 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 shutdown_connections 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc9200a2d50 0x7fc9200a5210 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc934109870 0x7fc9341adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc934108eb0 0x7fc93407fbf0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 --2- 172.21.15.103:0/365843352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9341081a0 0x7fc93407f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 >> 172.21.15.103:0/365843352 conn(0x7fc9340771a0 msgr2=0x7fc93410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 shutdown_connections 2024-08-19T01:45:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:08.047+0000 7fc938870640 1 -- 172.21.15.103:0/365843352 wait complete. 2024-08-19T01:45:08.388 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.002+0000 7f59c4c2d640 1 -- 172.21.15.103:0/4047260918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 msgr2=0x7f59c0114e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.002+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/4047260918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c0114e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009ca0 tx=0x7f59b802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.002+0000 7f59c4c2d640 1 -- 172.21.15.103:0/4047260918 shutdown_connections 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.002+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/4047260918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c0114e60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.002+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/4047260918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59c0109970 0x7f59c010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.002+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/4047260918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59c0108fc0 0x7f59c01093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.002+0000 7f59c4c2d640 1 -- 172.21.15.103:0/4047260918 >> 172.21.15.103:0/4047260918 conn(0x7f59c0077940 msgr2=0x7f59c0079d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.003+0000 7f59c4c2d640 1 -- 172.21.15.103:0/4047260918 shutdown_connections 2024-08-19T01:45:09.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.003+0000 7f59c4c2d640 1 -- 172.21.15.103:0/4047260918 wait complete. 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.004+0000 7f59c4c2d640 1 Processor -- start 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.004+0000 7f59c4c2d640 1 -- start start 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.004+0000 7f59c4c2d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59c0108fc0 0x7f59c019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59c4c2d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59c0109970 0x7f59c019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59c4c2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c01a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59bdd9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59c0109970 0x7f59c019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59bed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c01a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59bed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c01a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43252/0 (socket says 172.21.15.103:43252) 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59bdd9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59c0109970 0x7f59c019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:33462/0 (socket says 172.21.15.103:33462) 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59bed9d640 1 -- 172.21.15.103:0/3620528565 learned_addr learned my addr 172.21.15.103:0/3620528565 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59c01181a0 con 0x7f59c0108fc0 2024-08-19T01:45:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0118020 con 0x7f59c0109970 2024-08-19T01:45:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.005+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0118320 con 0x7f59c010e4e0 2024-08-19T01:45:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.006+0000 7f59bed9d640 1 -- 172.21.15.103:0/3620528565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59c0109970 msgr2=0x7f59c019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.006+0000 7f59bed9d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59c0109970 0x7f59c019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.006+0000 7f59bed9d640 1 -- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59c0108fc0 msgr2=0x7f59c019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.006+0000 7f59bed9d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59c0108fc0 0x7f59c019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.006+0000 7f59bed9d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c01a3370 con 0x7f59c010e4e0 2024-08-19T01:45:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.007+0000 7f59bed9d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c01a2c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59b8003ba0 tx=0x7f59b8003bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.007+0000 7f59b77fe640 1 -- 172.21.15.103:0/3620528565 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8004040 con 0x7f59c010e4e0 2024-08-19T01:45:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.007+0000 7f59b77fe640 1 -- 172.21.15.103:0/3620528565 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b8004690 con 0x7f59c010e4e0 2024-08-19T01:45:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.008+0000 7f59b77fe640 1 -- 172.21.15.103:0/3620528565 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8030a30 con 0x7f59c010e4e0 2024-08-19T01:45:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.008+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c01a3600 con 0x7f59c010e4e0 2024-08-19T01:45:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.008+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c01a3a80 con 0x7f59c010e4e0 2024-08-19T01:45:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.011+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c019f720 con 0x7f59c010e4e0 2024-08-19T01:45:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.012+0000 7f59b77fe640 1 -- 172.21.15.103:0/3620528565 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f59b8030c30 con 0x7f59c010e4e0 2024-08-19T01:45:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.012+0000 7f59b77fe640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f59ac0a2d90 0x7f59ac0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.012+0000 7f59b77fe640 1 -- 172.21.15.103:0/3620528565 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f59b80f01b0 con 0x7f59c010e4e0 2024-08-19T01:45:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.013+0000 7f59be59c640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f59ac0a2d90 0x7f59ac0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.013+0000 7f59be59c640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f59ac0a2d90 0x7f59ac0a5250 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f59a80058c0 tx=0x7f59a8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:09.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.018+0000 7f59b77fe640 1 -- 172.21.15.103:0/3620528565 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f59b80bd0e0 con 0x7f59c010e4e0 2024-08-19T01:45:09.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:09 smithi103 ceph-mon[18436]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 6.55k op/s 2024-08-19T01:45:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:09 smithi190 ceph-mon[22032]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 6.55k op/s 2024-08-19T01:45:09.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.405+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59c01ab2e0 con 0x7f59c010e4e0 2024-08-19T01:45:09.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.406+0000 7f59b77fe640 1 -- 172.21.15.103:0/3620528565 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11996 (secure 0 0 0) 0x7f59b80bc9e0 con 0x7f59c010e4e0 2024-08-19T01:45:09.408 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:45:09.408 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":39,"btime":"2024-08-19T01:44:22:078808+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:44:21.578554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24452,24478,24454,24466,24485]},"id":1}]} 2024-08-19T01:45:09.409 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 39 2024-08-19T01:45:09.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.410+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f59ac0a2d90 msgr2=0x7f59ac0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:09.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.410+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f59ac0a2d90 0x7f59ac0a5250 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f59a80058c0 tx=0x7f59a8005850 comp rx=0 tx=0).stop 2024-08-19T01:45:09.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.410+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 msgr2=0x7f59c01a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:09.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.410+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c01a2c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59b8003ba0 tx=0x7f59b8003bd0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 shutdown_connections 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f59ac0a2d90 0x7f59ac0a5250 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c010e4e0 0x7f59c01a2c90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59c0109970 0x7f59c019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 --2- 172.21.15.103:0/3620528565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59c0108fc0 0x7f59c019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 >> 172.21.15.103:0/3620528565 conn(0x7f59c0077940 msgr2=0x7f59c007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 shutdown_connections 2024-08-19T01:45:09.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:09.411+0000 7f59c4c2d640 1 -- 172.21.15.103:0/3620528565 wait complete. 2024-08-19T01:45:09.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:09 smithi146 ceph-mon[22098]: pgmap v620: 97 pgs: 97 active+clean; 13 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 26 MiB/s wr, 6.55k op/s 2024-08-19T01:45:10.150 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:10 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3620528565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:45:10.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-19T01:45:10.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:10 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3620528565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:45:10.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:10 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3620528565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:45:10.651 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 -- 172.21.15.103:0/2366843823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f731810dd90 msgr2=0x7f731810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 --2- 172.21.15.103:0/2366843823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f731810dd90 0x7f731810e170 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7314009930 tx=0x7f731402a420 comp rx=0 tx=0).stop 2024-08-19T01:45:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 -- 172.21.15.103:0/2366843823 shutdown_connections 2024-08-19T01:45:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 --2- 172.21.15.103:0/2366843823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73181086b0 0x7f7318114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 --2- 172.21.15.103:0/2366843823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318107d50 0x7f7318108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 --2- 172.21.15.103:0/2366843823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f731810dd90 0x7f731810e170 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 -- 172.21.15.103:0/2366843823 >> 172.21.15.103:0/2366843823 conn(0x7f73180776b0 msgr2=0x7f7318079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:11.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 -- 172.21.15.103:0/2366843823 shutdown_connections 2024-08-19T01:45:11.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.261+0000 7f731f53a640 1 -- 172.21.15.103:0/2366843823 wait complete. 2024-08-19T01:45:11.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 Processor -- start 2024-08-19T01:45:11.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 -- start start 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318107d50 0x7f731819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73181086b0 0x7f731819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f731810dd90 0x7f73181a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7318117f90 con 0x7f73181086b0 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7318117e10 con 0x7f731810dd90 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.262+0000 7f731f53a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7318118110 con 0x7f7318107d50 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731e538640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318107d50 0x7f731819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731dd37640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73181086b0 0x7f731819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731e538640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318107d50 0x7f731819e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43278/0 (socket says 172.21.15.103:43278) 2024-08-19T01:45:11.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731e538640 1 -- 172.21.15.103:0/2569299325 learned_addr learned my addr 172.21.15.103:0/2569299325 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731ed39640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f731810dd90 0x7f73181a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731dd37640 1 -- 172.21.15.103:0/2569299325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318107d50 msgr2=0x7f731819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731dd37640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318107d50 0x7f731819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731dd37640 1 -- 172.21.15.103:0/2569299325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f731810dd90 msgr2=0x7f73181a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731dd37640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f731810dd90 0x7f73181a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.263+0000 7f731dd37640 1 -- 172.21.15.103:0/2569299325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73181a34a0 con 0x7f73181086b0 2024-08-19T01:45:11.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.264+0000 7f731dd37640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73181086b0 0x7f731819ea30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f730801fa00 tx=0x7f730801fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:11.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.264+0000 7f730f7fe640 1 -- 172.21.15.103:0/2569299325 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730800ccc0 con 0x7f73181086b0 2024-08-19T01:45:11.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.264+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73181a3790 con 0x7f73181086b0 2024-08-19T01:45:11.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.264+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731806b940 con 0x7f73181086b0 2024-08-19T01:45:11.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.264+0000 7f730f7fe640 1 -- 172.21.15.103:0/2569299325 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73080218f0 con 0x7f73181086b0 2024-08-19T01:45:11.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.264+0000 7f730f7fe640 1 -- 172.21.15.103:0/2569299325 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730800ccc0 con 0x7f73181086b0 2024-08-19T01:45:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.267+0000 7f7322b9a740 1 Processor -- start 2024-08-19T01:45:11.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f7322b9a740 1 -- start start 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e72ae0 0x558be6e7ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e7d3a0 0x558be6e7f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f7322b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558be6dd7a40 con 0x558be6e7d3a0 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f7322b9a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558be6d6f9d0 con 0x558be6e72ae0 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f7322b9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558be6dd3a00 con 0x558be6e721c0 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f731e538640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e72ae0 0x558be6e7ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f731e538640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e72ae0 0x558be6e7ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:33490/0 (socket says 172.21.15.103:33490) 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f731e538640 1 -- 172.21.15.103:0/2094085118 learned_addr learned my addr 172.21.15.103:0/2094085118 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f730f7fe640 1 -- 172.21.15.103:0/2569299325 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7308021440 con 0x7f73181086b0 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.268+0000 7f731ed39640 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.269+0000 7f731e538640 1 -- 172.21.15.103:0/2094085118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 msgr2=0x558be6e725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:11.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.269+0000 7f731dd37640 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e7d3a0 0x558be6e7f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.269+0000 7f731e538640 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.269+0000 7f731e538640 1 -- 172.21.15.103:0/2094085118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e7d3a0 msgr2=0x558be6e7f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.269+0000 7f731e538640 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e7d3a0 0x558be6e7f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.269+0000 7f731e538640 1 -- 172.21.15.103:0/2094085118 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be6d76650 con 0x558be6e72ae0 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.269+0000 7f731dd37640 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e7d3a0 0x558be6e7f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.270+0000 7f731e538640 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e72ae0 0x558be6e7ce60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f73140040f0 tx=0x7f7314004120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c746be36cde51c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.270+0000 7f730f7fe640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f73040a2f90 0x7f73040a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.271+0000 7f730f7fe640 1 -- 172.21.15.103:0/2569299325 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f730800cac0 con 0x7f73181086b0 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.272+0000 7f730cff9640 1 -- 172.21.15.103:0/2094085118 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314035030 con 0x558be6e72ae0 2024-08-19T01:45:11.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.272+0000 7f730cff9640 1 -- 172.21.15.103:0/2094085118 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73140054f0 con 0x558be6e72ae0 2024-08-19T01:45:11.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.272+0000 7f731ed39640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f73040a2f90 0x7f73040a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.272+0000 7f730cff9640 1 -- 172.21.15.103:0/2094085118 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140035f0 con 0x558be6e72ae0 2024-08-19T01:45:11.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f731ed39640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f73040a2f90 0x7f73040a5450 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f73100099a0 tx=0x7f7310009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:11.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f7322b9a740 1 -- 172.21.15.103:0/2094085118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e72ae0 msgr2=0x558be6e7ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e72ae0 0x558be6e7ce60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f73140040f0 tx=0x7f7314004120 comp rx=0 tx=0).stop 2024-08-19T01:45:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f7322b9a740 1 -- 172.21.15.103:0/2094085118 shutdown_connections 2024-08-19T01:45:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e7d3a0 0x558be6e7f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e72ae0 0x558be6e7ce60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2094085118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.273+0000 7f7322b9a740 1 -- 172.21.15.103:0/2094085118 >> 172.21.15.103:0/2094085118 conn(0x558be6e6e750 msgr2=0x558be6e6eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.274+0000 7f7322b9a740 1 -- 172.21.15.103:0/2094085118 shutdown_connections 2024-08-19T01:45:11.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.274+0000 7f7322b9a740 1 -- 172.21.15.103:0/2094085118 wait complete. 2024-08-19T01:45:11.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.274+0000 7f7322b9a740 1 -- start start 2024-08-19T01:45:11.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.274+0000 7f7322b9a740 1 Processor -- start 2024-08-19T01:45:11.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 0x558be6e7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e7a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f7322b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558be6e7ffc0 con 0x558be6e72ae0 2024-08-19T01:45:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f7322b9a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558be6d92b20 con 0x558be6e721c0 2024-08-19T01:45:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f7322b9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558be6e80140 con 0x558be6e7d3a0 2024-08-19T01:45:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f7322b9a740 10 client.0 _ll_register_callbacks cb 0x558be6d90bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f731ed39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e7a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f731ed39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e7a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43308/0 (socket says 172.21.15.103:43308) 2024-08-19T01:45:11.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f731ed39640 1 -- 172.21.15.103:0/2936607823 learned_addr learned my addr 172.21.15.103:0/2936607823 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.275+0000 7f731dd37640 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.276+0000 7f731e538640 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 0x558be6e7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.276+0000 7f731dd37640 1 -- 172.21.15.103:0/2936607823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 msgr2=0x558be6e7a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.276+0000 7f731dd37640 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e7a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.276+0000 7f731dd37640 1 -- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 msgr2=0x558be6e7ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.276+0000 7f731dd37640 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 0x558be6e7ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.276+0000 7f731dd37640 1 -- 172.21.15.103:0/2936607823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be6b493c0 con 0x558be6e72ae0 2024-08-19T01:45:11.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.276+0000 7f731dd37640 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7a160 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x558be6e6fad0 tx=0x7f7308021730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16f6950ad1b558d server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.277+0000 7f7301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:45:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.277+0000 7f7301ffb640 1 -- 172.21.15.103:0/2936607823 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080ac660 con 0x558be6e72ae0 2024-08-19T01:45:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.278+0000 7f7301ffb640 1 -- 172.21.15.103:0/2936607823 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f730800ccc0 con 0x558be6e72ae0 2024-08-19T01:45:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.278+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73140054f0 con 0x558be6e72ae0 2024-08-19T01:45:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.278+0000 7f7301ffb640 1 -- 172.21.15.103:0/2936607823 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f73080b0a10 con 0x558be6e72ae0 2024-08-19T01:45:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.278+0000 7f7322b9a740 10 client.17712 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:11.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.278+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558be6e8bb00 con 0x558be6e72ae0 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7301ffb640 1 -- 172.21.15.103:0/2936607823 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73080b52d0 con 0x558be6e72ae0 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7301ffb640 1 -- 172.21.15.103:0/2936607823 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f73080a85b0 con 0x558be6e72ae0 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7322b9a740 10 client.17712 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7322b9a740 10 client.17712 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7322b9a740 20 client.17712 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6e8d550 0x558be6e8f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7322b9a740 4 client.17712 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f73140054f0 con 0x558be6e8d550 2024-08-19T01:45:11.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.279+0000 7f731e538640 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6e8d550 0x558be6e8f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.280+0000 7f731e538640 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6e8d550 0x558be6e8f9d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a8aefc4c4dd8803 server_cookie=d219fa4d069b2942 in_seq=0 out_seq=0 2024-08-19T01:45:11.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.280+0000 7f7301ffb640 10 client.17712 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:45:11.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7301ffb640 1 -- 172.21.15.103:0/2936607823 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f73140054f0 con 0x558be6e8d550 2024-08-19T01:45:11.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7301ffb640 10 client.17712 handle_command_reply: tid=0 2024-08-19T01:45:11.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 client.17712 shutdown 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 msgr2=0x558be6e7a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7a160 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x558be6e6fad0 tx=0x7f7308021730 comp rx=0 tx=0).stop 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 shutdown_connections 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6e8d550 0x558be6e8f9d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7301ffb640 0 client.17712 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e7a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7a160 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 --2- 172.21.15.103:0/2936607823 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 0x558be6e7ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.281+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 >> 172.21.15.103:0/2936607823 conn(0x558be6e6e750 msgr2=0x558be6e74430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.282+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 shutdown_connections 2024-08-19T01:45:11.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.282+0000 7f7322b9a740 1 -- 172.21.15.103:0/2936607823 wait complete. 2024-08-19T01:45:11.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.282+0000 7f7322b9a740 20 client.17712 trim_cache size 0 max 16384 2024-08-19T01:45:11.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.306+0000 7f7322b9a740 1 Processor -- start 2024-08-19T01:45:11.308 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:11 smithi103 ceph-mon[18436]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 6.58k op/s 2024-08-19T01:45:11.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f7322b9a740 1 -- start start 2024-08-19T01:45:11.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e89300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e7d3a0 0x558be6e7ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f7322b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc00e440 con 0x558be6e72ae0 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f7322b9a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc010040 con 0x558be6e7d3a0 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f7322b9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc00eb10 con 0x558be6e721c0 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.307+0000 7f731e538640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e89300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e89300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43312/0 (socket says 172.21.15.103:43312) 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 -- 172.21.15.103:0/3751996385 learned_addr learned my addr 172.21.15.103:0/3751996385 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731ed39640 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e7d3a0 0x558be6e7ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 -- 172.21.15.103:0/3751996385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e7d3a0 msgr2=0x558be6e7ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e7d3a0 0x558be6e7ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 -- 172.21.15.103:0/3751996385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 msgr2=0x558be6e7c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 -- 172.21.15.103:0/3751996385 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be6e6e580 con 0x558be6e721c0 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f731e538640 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e89300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7314005c40 tx=0x7f7314004b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b45bcbfd1bf8f99 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.308+0000 7f7303fff640 1 -- 172.21.15.103:0/3751996385 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314035040 con 0x558be6e721c0 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.309+0000 7f7303fff640 1 -- 172.21.15.103:0/3751996385 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73140048c0 con 0x558be6e721c0 2024-08-19T01:45:11.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.309+0000 7f7303fff640 1 -- 172.21.15.103:0/3751996385 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140039e0 con 0x558be6e721c0 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 -- 172.21.15.103:0/3751996385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 msgr2=0x558be6e89300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e89300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7314005c40 tx=0x7f7314004b90 comp rx=0 tx=0).stop 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 -- 172.21.15.103:0/3751996385 shutdown_connections 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e7d3a0 0x558be6e7ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e7c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 --2- 172.21.15.103:0/3751996385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e721c0 0x558be6e89300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 -- 172.21.15.103:0/3751996385 >> 172.21.15.103:0/3751996385 conn(0x558be6e6e750 msgr2=0x558be6e79250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 -- 172.21.15.103:0/3751996385 shutdown_connections 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 -- 172.21.15.103:0/3751996385 wait complete. 2024-08-19T01:45:11.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 -- start start 2024-08-19T01:45:11.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.310+0000 7f7322b9a740 1 Processor -- start 2024-08-19T01:45:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 0x558be6e85870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f7322b9a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e77920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f7322b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc00f970 con 0x558be6e72ae0 2024-08-19T01:45:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f7322b9a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc00f1e0 con 0x558be6e721c0 2024-08-19T01:45:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f7322b9a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc00c420 con 0x558be6e7d3a0 2024-08-19T01:45:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f7322b9a740 10 client.0 _ll_register_callbacks cb 0x7f7308005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:11.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f731ed39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e77920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f731ed39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e77920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43324/0 (socket says 172.21.15.103:43324) 2024-08-19T01:45:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.311+0000 7f731ed39640 1 -- 172.21.15.103:0/4201457395 learned_addr learned my addr 172.21.15.103:0/4201457395 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.313+0000 7f731ed39640 1 -- 172.21.15.103:0/4201457395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 msgr2=0x558be6e85870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:45:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.313+0000 7f731ed39640 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 0x558be6e85870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.313+0000 7f731ed39640 1 -- 172.21.15.103:0/4201457395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 msgr2=0x558be6e773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:11.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.313+0000 7f731ed39640 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.313+0000 7f731ed39640 1 -- 172.21.15.103:0/4201457395 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be6e77e60 con 0x558be6e7d3a0 2024-08-19T01:45:11.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.313+0000 7f731ed39640 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e77920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7310011b90 tx=0x7f7310004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ceb4d47049173e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:11.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.314+0000 7f7301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:45:11.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.314+0000 7f7301ffb640 1 -- 172.21.15.103:0/4201457395 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731001f9a0 con 0x558be6e7d3a0 2024-08-19T01:45:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.314+0000 7f7301ffb640 1 -- 172.21.15.103:0/4201457395 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f731001fb40 con 0x558be6e7d3a0 2024-08-19T01:45:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.314+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73140048c0 con 0x558be6e7d3a0 2024-08-19T01:45:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.314+0000 7f7301ffb640 1 -- 172.21.15.103:0/4201457395 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f731001fd40 con 0x558be6e7d3a0 2024-08-19T01:45:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.314+0000 7f7322b9a740 10 client.26917 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.314+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558be6e7ec00 con 0x558be6e7d3a0 2024-08-19T01:45:11.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7301ffb640 1 -- 172.21.15.103:0/4201457395 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73140048c0 con 0x558be6e7d3a0 2024-08-19T01:45:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7301ffb640 1 -- 172.21.15.103:0/4201457395 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7310004630 con 0x558be6e7d3a0 2024-08-19T01:45:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7322b9a740 10 client.26917 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7322b9a740 10 client.26917 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-19T01:45:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7322b9a740 20 client.26917 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:11.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7322b9a740 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6ef8730 0x558be6efabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7322b9a740 4 client.26917 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:45:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.315+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 --> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558be6de4fb0 con 0x558be6ef8730 2024-08-19T01:45:11.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.316+0000 7f731e538640 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6ef8730 0x558be6efabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:11.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.316+0000 7f731e538640 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6ef8730 0x558be6efabb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=37dce2f32dfcf17d server_cookie=db934014d25eb6ca in_seq=0 out_seq=0 2024-08-19T01:45:11.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.316+0000 7f7301ffb640 10 client.26917 ms_handle_connect on v2:172.21.15.146:6836/3498679319 2024-08-19T01:45:11.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.316+0000 7f7301ffb640 1 -- 172.21.15.103:0/4201457395 <== mds.4 v2:172.21.15.146:6836/3498679319 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558be6de4fb0 con 0x558be6ef8730 2024-08-19T01:45:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.316+0000 7f7301ffb640 10 client.26917 handle_command_reply: tid=0 2024-08-19T01:45:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.316+0000 7f7322b9a740 1 client.26917 shutdown 2024-08-19T01:45:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.317+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 msgr2=0x558be6e77920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.317+0000 7f7322b9a740 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e77920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7310011b90 tx=0x7f7310004a40 comp rx=0 tx=0).stop 2024-08-19T01:45:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 shutdown_connections 2024-08-19T01:45:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.146:6836/3498679319,v1:172.21.15.146:6837/3498679319] conn(0x558be6ef8730 0x558be6efabb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7301ffb640 0 client.26917 ms_handle_reset on v2:172.21.15.146:6836/3498679319 2024-08-19T01:45:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558be6e7d3a0 0x558be6e77920 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558be6e72ae0 0x558be6e773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 --2- 172.21.15.103:0/4201457395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558be6e721c0 0x558be6e85870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 >> 172.21.15.103:0/4201457395 conn(0x558be6e6e750 msgr2=0x558be6e6d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:11.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 shutdown_connections 2024-08-19T01:45:11.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 1 -- 172.21.15.103:0/4201457395 wait complete. 2024-08-19T01:45:11.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.320+0000 7f7322b9a740 20 client.26917 trim_cache size 0 max 16384 2024-08-19T01:45:11.329 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:45:11.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.321+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f73040a2f90 msgr2=0x7f73040a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.321+0000 7f731f53a640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f73040a2f90 0x7f73040a5450 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f73100099a0 tx=0x7f7310009500 comp rx=0 tx=0).stop 2024-08-19T01:45:11.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.321+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73181086b0 msgr2=0x7f731819ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:11.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.321+0000 7f731f53a640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73181086b0 0x7f731819ea30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f730801fa00 tx=0x7f730801fec0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 shutdown_connections 2024-08-19T01:45:11.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f73040a2f90 0x7f73040a5450 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f731810dd90 0x7f73181a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73181086b0 0x7f731819ea30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 --2- 172.21.15.103:0/2569299325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318107d50 0x7f731819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:11.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 >> 172.21.15.103:0/2569299325 conn(0x7f73180776b0 msgr2=0x7f7318079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:11.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 shutdown_connections 2024-08-19T01:45:11.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:11.322+0000 7f731f53a640 1 -- 172.21.15.103:0/2569299325 wait complete. 2024-08-19T01:45:11.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:11 smithi190 ceph-mon[22032]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 6.58k op/s 2024-08-19T01:45:11.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:11 smithi146 ceph-mon[22098]: pgmap v621: 97 pgs: 97 active+clean; 13 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 MiB/s wr, 6.58k op/s 2024-08-19T01:45:12.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-19T01:45:12.661 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:13.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:13 smithi103 ceph-mon[18436]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 MiB/s wr, 7.43k op/s 2024-08-19T01:45:13.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.272+0000 7f977959a640 1 -- 172.21.15.103:0/2596136503 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774036070 con 0x7f977c109ac0 2024-08-19T01:45:13.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.272+0000 7f9780ce5640 1 -- 172.21.15.103:0/2596136503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 msgr2=0x7f977c109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.272+0000 7f9780ce5640 1 --2- 172.21.15.103:0/2596136503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 0x7f977c109ea0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9774009c10 tx=0x7f977402a6e0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.273+0000 7f9780ce5640 1 -- 172.21.15.103:0/2596136503 shutdown_connections 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.273+0000 7f9780ce5640 1 --2- 172.21.15.103:0/2596136503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c104460 0x7f977c110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.273+0000 7f9780ce5640 1 --2- 172.21.15.103:0/2596136503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c103ac0 0x7f977c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.273+0000 7f9780ce5640 1 --2- 172.21.15.103:0/2596136503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 0x7f977c109ea0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.273+0000 7f9780ce5640 1 -- 172.21.15.103:0/2596136503 >> 172.21.15.103:0/2596136503 conn(0x7f977c0ff7e0 msgr2=0x7f977c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.273+0000 7f9780ce5640 1 -- 172.21.15.103:0/2596136503 shutdown_connections 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.273+0000 7f9780ce5640 1 -- 172.21.15.103:0/2596136503 wait complete. 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 Processor -- start 2024-08-19T01:45:13.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 -- start start 2024-08-19T01:45:13.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c103ac0 0x7f977c076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c104460 0x7f977c076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 0x7f977c076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f977c113dc0 con 0x7f977c109ac0 2024-08-19T01:45:13.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f977c113c40 con 0x7f977c104460 2024-08-19T01:45:13.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9780ce5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f977c113f40 con 0x7f977c103ac0 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9779d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c104460 0x7f977c076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 0x7f977c076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9779d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c104460 0x7f977c076970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59124/0 (socket says 172.21.15.103:59124) 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 0x7f977c076eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40508/0 (socket says 172.21.15.103:40508) 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f9779d9b640 1 -- 172.21.15.103:0/3172954378 learned_addr learned my addr 172.21.15.103:0/3172954378 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.274+0000 7f977a59c640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c103ac0 0x7f977c076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f9779d9b640 1 -- 172.21.15.103:0/3172954378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c103ac0 msgr2=0x7f977c076430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f9779d9b640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c103ac0 0x7f977c076430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f9779d9b640 1 -- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 msgr2=0x7f977c076eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f9779d9b640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 0x7f977c076eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f9779d9b640 1 -- 172.21.15.103:0/3172954378 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1aa960 con 0x7f977c104460 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f977a59c640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c103ac0 0x7f977c076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f9779d9b640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c104460 0x7f977c076970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f977000ea50 tx=0x7f977000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f976b7fe640 1 -- 172.21.15.103:0/3172954378 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977000cd00 con 0x7f977c104460 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f976b7fe640 1 -- 172.21.15.103:0/3172954378 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97700108e0 con 0x7f977c104460 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1aac50 con 0x7f977c104460 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.275+0000 7f976b7fe640 1 -- 172.21.15.103:0/3172954378 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977000cd00 con 0x7f977c104460 2024-08-19T01:45:13.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.276+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1ab140 con 0x7f977c104460 2024-08-19T01:45:13.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.278+0000 7f976b7fe640 1 -- 172.21.15.103:0/3172954378 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9770010a90 con 0x7f977c104460 2024-08-19T01:45:13.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.279+0000 7f976b7fe640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97640a2d80 0x7f97640a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.279+0000 7f976b7fe640 1 -- 172.21.15.103:0/3172954378 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f97700cefa0 con 0x7f977c104460 2024-08-19T01:45:13.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f977a59c640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97640a2d80 0x7f97640a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 Processor -- start 2024-08-19T01:45:13.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 -- start start 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d57910 0x563788d61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d621d0 0x563788d645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563788cbca40 con 0x563788d621d0 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563788c549d0 con 0x563788d56ff0 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f97830e3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563788cb8a00 con 0x563788d57910 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f977a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f977a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59140/0 (socket says 172.21.15.103:59140) 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.280+0000 7f977a59c640 1 -- 172.21.15.103:0/1287998423 learned_addr learned my addr 172.21.15.103:0/1287998423 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:13.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9779d9b640 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d57910 0x563788d61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f977a59c640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97640a2d80 0x7f97640a5240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9774004af0 tx=0x7f9774004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f977ad9d640 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d621d0 0x563788d645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9779d9b640 1 -- 172.21.15.103:0/1287998423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 msgr2=0x563788d573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9779d9b640 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9779d9b640 1 -- 172.21.15.103:0/1287998423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d621d0 msgr2=0x563788d645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9779d9b640 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d621d0 0x563788d645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9779d9b640 1 -- 172.21.15.103:0/1287998423 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563788c5b650 con 0x563788d57910 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f977a59c640 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:45:13.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9779d9b640 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d57910 0x563788d61c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97700ccb20 tx=0x7f977000cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc29fc81e072e2e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.281+0000 7f9768ff9640 1 -- 172.21.15.103:0/1287998423 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977009fbe0 con 0x563788d57910 2024-08-19T01:45:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.282+0000 7f9768ff9640 1 -- 172.21.15.103:0/1287998423 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97700a44a0 con 0x563788d57910 2024-08-19T01:45:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.282+0000 7f9768ff9640 1 -- 172.21.15.103:0/1287998423 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770086280 con 0x563788d57910 2024-08-19T01:45:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.282+0000 7f97830e3740 1 -- 172.21.15.103:0/1287998423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d57910 msgr2=0x563788d61c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.282+0000 7f97830e3740 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d57910 0x563788d61c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97700ccb20 tx=0x7f977000cea0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 -- 172.21.15.103:0/1287998423 shutdown_connections 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d621d0 0x563788d645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d57910 0x563788d61c90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 --2- 172.21.15.103:0/1287998423 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 -- 172.21.15.103:0/1287998423 >> 172.21.15.103:0/1287998423 conn(0x563788d53580 msgr2=0x563788d53960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 -- 172.21.15.103:0/1287998423 shutdown_connections 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 -- 172.21.15.103:0/1287998423 wait complete. 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 -- start start 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.283+0000 7f97830e3740 1 Processor -- start 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f97830e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d5f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f97830e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d5f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f977a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d5f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f977a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d5f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:42852/0 (socket says 172.21.15.103:42852) 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f977a59c640 1 -- 172.21.15.103:0/1807228774 learned_addr learned my addr 172.21.15.103:0/1807228774 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f97830e3740 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d5fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563788d64da0 con 0x563788d57910 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563788c77b20 con 0x563788d621d0 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563788d64f20 con 0x563788d56ff0 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f97830e3740 10 client.0 _ll_register_callbacks cb 0x563788c75bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f9779d9b640 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d5f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.284+0000 7f977ad9d640 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d5fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f977a59c640 1 -- 172.21.15.103:0/1807228774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 msgr2=0x563788d5fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f977a59c640 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d5fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f977a59c640 1 -- 172.21.15.103:0/1807228774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 msgr2=0x563788d5f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f977a59c640 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d5f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f977a59c640 1 -- 172.21.15.103:0/1807228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563788a2e3c0 con 0x563788d56ff0 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f977a59c640 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d5f110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f977402acd0 tx=0x7f9774009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=707a1036f9969084 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f9761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f9761ffb640 1 -- 172.21.15.103:0/1807228774 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977404e020 con 0x563788d56ff0 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f9761ffb640 1 -- 172.21.15.103:0/1807228774 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9774004450 con 0x563788d56ff0 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.285+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97700a44a0 con 0x563788d56ff0 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f9761ffb640 1 -- 172.21.15.103:0/1807228774 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9774036040 con 0x563788d56ff0 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 10 client.26929 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:13.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563788d70a50 con 0x563788d56ff0 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f9761ffb640 1 -- 172.21.15.103:0/1807228774 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97700a44a0 con 0x563788d56ff0 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f9761ffb640 1 -- 172.21.15.103:0/1807228774 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f977404a440 con 0x563788d56ff0 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 10 client.26929 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 10 client.26929 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 20 client.26929 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788d71e60 0x563788d74270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 4 client.26929 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.286+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563788cc9fb0 con 0x563788d71e60 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.287+0000 7f9779d9b640 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788d71e60 0x563788d74270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.288+0000 7f9779d9b640 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788d71e60 0x563788d74270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24f5c41d7266723d server_cookie=d8d8a07608be3307 in_seq=0 out_seq=0 2024-08-19T01:45:13.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.288+0000 7f9761ffb640 10 client.26929 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:45:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f9761ffb640 1 -- 172.21.15.103:0/1807228774 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563788cc9fb0 con 0x563788d71e60 2024-08-19T01:45:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f9761ffb640 10 client.26929 handle_command_reply: tid=0 2024-08-19T01:45:13.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 client.26929 shutdown 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 msgr2=0x563788d5f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d5f110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f977402acd0 tx=0x7f9774009500 comp rx=0 tx=0).stop 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 shutdown_connections 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788d71e60 0x563788d74270 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d5fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d5f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 --2- 172.21.15.103:0/1807228774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d5f110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 >> 172.21.15.103:0/1807228774 conn(0x563788d53580 msgr2=0x563788d59260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:13.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.289+0000 7f9761ffb640 0 client.26929 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:45:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.291+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 shutdown_connections 2024-08-19T01:45:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.291+0000 7f97830e3740 1 -- 172.21.15.103:0/1807228774 wait complete. 2024-08-19T01:45:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.291+0000 7f97830e3740 20 client.26929 trim_cache size 0 max 16384 2024-08-19T01:45:13.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 Processor -- start 2024-08-19T01:45:13.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 -- start start 2024-08-19T01:45:13.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d79b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d621d0 0x563788d7a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b38c0 con 0x563788d57910 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b54c0 con 0x563788d56ff0 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.316+0000 7f97830e3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b3f90 con 0x563788d621d0 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d621d0 0x563788d7a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d621d0 0x563788d7a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:42864/0 (socket says 172.21.15.103:42864) 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 -- 172.21.15.103:0/3226275646 learned_addr learned my addr 172.21.15.103:0/3226275646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f9779d9b640 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d79b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 -- 172.21.15.103:0/3226275646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 msgr2=0x563788d524c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d524c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 -- 172.21.15.103:0/3226275646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 msgr2=0x563788d79b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d79b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 -- 172.21.15.103:0/3226275646 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563788d533b0 con 0x563788d621d0 2024-08-19T01:45:13.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.317+0000 7f977ad9d640 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d621d0 0x563788d7a080 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f976c009cc0 tx=0x7f976c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b17ec3792cc1294 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:13.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.318+0000 7f9763fff640 1 -- 172.21.15.103:0/3226275646 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c009db0 con 0x563788d621d0 2024-08-19T01:45:13.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.318+0000 7f9763fff640 1 -- 172.21.15.103:0/3226275646 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f976c0054d0 con 0x563788d621d0 2024-08-19T01:45:13.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.318+0000 7f9763fff640 1 -- 172.21.15.103:0/3226275646 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c02c440 con 0x563788d621d0 2024-08-19T01:45:13.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.318+0000 7f97830e3740 1 -- 172.21.15.103:0/3226275646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d621d0 msgr2=0x563788d7a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.318+0000 7f97830e3740 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d621d0 0x563788d7a080 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f976c009cc0 tx=0x7f976c010800 comp rx=0 tx=0).stop 2024-08-19T01:45:13.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.319+0000 7f97830e3740 1 -- 172.21.15.103:0/3226275646 shutdown_connections 2024-08-19T01:45:13.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.319+0000 7f97830e3740 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d621d0 0x563788d7a080 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.319+0000 7f97830e3740 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d79b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.319+0000 7f97830e3740 1 --2- 172.21.15.103:0/3226275646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d56ff0 0x563788d524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.319+0000 7f97830e3740 1 -- 172.21.15.103:0/3226275646 >> 172.21.15.103:0/3226275646 conn(0x563788d53580 msgr2=0x563788d51db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:13.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.320+0000 7f97830e3740 1 -- 172.21.15.103:0/3226275646 shutdown_connections 2024-08-19T01:45:13.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.320+0000 7f97830e3740 1 -- 172.21.15.103:0/3226275646 wait complete. 2024-08-19T01:45:13.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.321+0000 7f97830e3740 1 -- start start 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.321+0000 7f97830e3740 1 Processor -- start 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f97830e3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d6a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f97830e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d65ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f97830e3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f97830e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b4df0 con 0x563788d57910 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d6a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f97830e3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b4660 con 0x563788d621d0 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d6a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:42876/0 (socket says 172.21.15.103:42876) 2024-08-19T01:45:13.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f97830e3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b18a0 con 0x563788d56ff0 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 -- 172.21.15.103:0/2553663024 learned_addr learned my addr 172.21.15.103:0/2553663024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f97830e3740 10 client.0 _ll_register_callbacks cb 0x7f976c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977ad9d640 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 -- 172.21.15.103:0/2553663024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 msgr2=0x563788d662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 -- 172.21.15.103:0/2553663024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 msgr2=0x563788d65ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d65ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.322+0000 7f977a59c640 1 -- 172.21.15.103:0/2553663024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563788d54210 con 0x563788d56ff0 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.323+0000 7f977a59c640 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d6a9b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9774057040 tx=0x7f977404acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478e49ca7ee70fa server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:13.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.323+0000 7f9761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:45:13.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.323+0000 7f9761ffb640 1 -- 172.21.15.103:0/2553663024 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977404e070 con 0x563788d56ff0 2024-08-19T01:45:13.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.323+0000 7f9761ffb640 1 -- 172.21.15.103:0/2553663024 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f9774004660 con 0x563788d56ff0 2024-08-19T01:45:13.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.323+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976c0054d0 con 0x563788d56ff0 2024-08-19T01:45:13.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.324+0000 7f9761ffb640 1 -- 172.21.15.103:0/2553663024 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f977403d460 con 0x563788d56ff0 2024-08-19T01:45:13.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.324+0000 7f97830e3740 10 client.26935 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:13.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.324+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563788ddca00 con 0x563788d56ff0 2024-08-19T01:45:13.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f9761ffb640 1 -- 172.21.15.103:0/2553663024 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f977404ae50 con 0x563788d56ff0 2024-08-19T01:45:13.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f9761ffb640 1 -- 172.21.15.103:0/2553663024 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f977403dc70 con 0x563788d56ff0 2024-08-19T01:45:13.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f97830e3740 10 client.26935 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:13.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f97830e3740 10 client.26935 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-19T01:45:13.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f97830e3740 20 client.26935 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f97830e3740 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788ddd0d0 0x563788ddf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f97830e3740 4 client.26935 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.325+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 --> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f976c0054d0 con 0x563788ddd0d0 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.326+0000 7f9779d9b640 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788ddd0d0 0x563788ddf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.327+0000 7f9761ffb640 10 client.26935 ms_handle_connect on v2:172.21.15.190:6834/794893065 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.327+0000 7f9779d9b640 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788ddd0d0 0x563788ddf510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfd1ee651fb04fea server_cookie=d7394f220a12b315 in_seq=0 out_seq=0 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.327+0000 7f9761ffb640 1 -- 172.21.15.103:0/2553663024 <== mds.3 v2:172.21.15.190:6834/794893065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f976c0054d0 con 0x563788ddd0d0 2024-08-19T01:45:13.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.327+0000 7f9761ffb640 10 client.26935 handle_command_reply: tid=0 2024-08-19T01:45:13.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.327+0000 7f97830e3740 1 client.26935 shutdown 2024-08-19T01:45:13.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 msgr2=0x563788d6a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d6a9b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9774057040 tx=0x7f977404acb0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 shutdown_connections 2024-08-19T01:45:13.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:6834/794893065,v1:172.21.15.190:6835/794893065] conn(0x563788ddd0d0 0x563788ddf510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563788d621d0 0x563788d662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563788d57910 0x563788d65ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 --2- 172.21.15.103:0/2553663024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563788d56ff0 0x563788d6a9b0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9774057040 tx=0x7f977404acb0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 >> 172.21.15.103:0/2553663024 conn(0x563788d53580 msgr2=0x563788d525d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:13.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f9761ffb640 0 client.26935 ms_handle_reset on v2:172.21.15.190:6834/794893065 2024-08-19T01:45:13.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 shutdown_connections 2024-08-19T01:45:13.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.328+0000 7f97830e3740 1 -- 172.21.15.103:0/2553663024 wait complete. 2024-08-19T01:45:13.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.329+0000 7f97830e3740 20 client.26935 trim_cache size 0 max 16384 2024-08-19T01:45:13.331 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.329+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97640a2d80 msgr2=0x7f97640a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.329+0000 7f9780ce5640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97640a2d80 0x7f97640a5240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9774004af0 tx=0x7f9774004a40 comp rx=0 tx=0).stop 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.329+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c104460 msgr2=0x7f977c076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.329+0000 7f9780ce5640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c104460 0x7f977c076970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f977000ea50 tx=0x7f977000ee50 comp rx=0 tx=0).stop 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 shutdown_connections 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f97640a2d80 0x7f97640a5240 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f977c109ac0 0x7f977c076eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f977c104460 0x7f977c076970 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 --2- 172.21.15.103:0/3172954378 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f977c103ac0 0x7f977c076430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:13.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 >> 172.21.15.103:0/3172954378 conn(0x7f977c0ff7e0 msgr2=0x7f977c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:13.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 shutdown_connections 2024-08-19T01:45:13.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:13.330+0000 7f9780ce5640 1 -- 172.21.15.103:0/3172954378 wait complete. 2024-08-19T01:45:13.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:13 smithi190 ceph-mon[22032]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 MiB/s wr, 7.43k op/s 2024-08-19T01:45:13.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:13 smithi146 ceph-mon[22098]: pgmap v622: 97 pgs: 97 active+clean; 13 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 MiB/s wr, 7.43k op/s 2024-08-19T01:45:15.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:15 smithi103 ceph-mon[18436]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 6.49k op/s 2024-08-19T01:45:15.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:15 smithi190 ceph-mon[22032]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 6.49k op/s 2024-08-19T01:45:15.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:15 smithi146 ceph-mon[22098]: pgmap v623: 97 pgs: 97 active+clean; 13 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 6.49k op/s 2024-08-19T01:45:15.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-19T01:45:16.379 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:16.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.990+0000 7f5122300640 1 -- 172.21.15.103:0/2107824396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 msgr2=0x7f511c1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:16.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 --2- 172.21.15.103:0/2107824396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1056b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5104009930 tx=0x7f510402a440 comp rx=0 tx=0).stop 2024-08-19T01:45:16.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 -- 172.21.15.103:0/2107824396 shutdown_connections 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 --2- 172.21.15.103:0/2107824396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c105bf0 0x7f511c106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 --2- 172.21.15.103:0/2107824396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1056b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 --2- 172.21.15.103:0/2107824396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f511c1005f0 0x7f511c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 -- 172.21.15.103:0/2107824396 >> 172.21.15.103:0/2107824396 conn(0x7f511c0fc500 msgr2=0x7f511c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 -- 172.21.15.103:0/2107824396 shutdown_connections 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.991+0000 7f5122300640 1 -- 172.21.15.103:0/2107824396 wait complete. 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.992+0000 7f5122300640 1 Processor -- start 2024-08-19T01:45:16.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5122300640 1 -- start start 2024-08-19T01:45:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5122300640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c1005f0 0x7f511c1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5122300640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5122300640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f511c105bf0 0x7f511c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:16.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5122300640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f511c118240 con 0x7f511c1005f0 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5122300640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1180c0 con 0x7f511c100f10 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5122300640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1183c0 con 0x7f511c105bf0 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5113fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5113fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59166/0 (socket says 172.21.15.103:59166) 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f5113fff640 1 -- 172.21.15.103:0/626328200 learned_addr learned my addr 172.21.15.103:0/626328200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f511bfff640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c1005f0 0x7f511c1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.993+0000 7f512089d640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f511c105bf0 0x7f511c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f512089d640 1 -- 172.21.15.103:0/626328200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 msgr2=0x7f511c1a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f512089d640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f512089d640 1 -- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c1005f0 msgr2=0x7f511c1a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f512089d640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c1005f0 0x7f511c1a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f512089d640 1 -- 172.21.15.103:0/626328200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c19ef60 con 0x7f511c105bf0 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f5113fff640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f511bfff640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c1005f0 0x7f511c1a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:45:16.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f512089d640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f511c105bf0 0x7f511c19e730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f511400da10 tx=0x7f511400ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.994+0000 7f5119ffb640 1 -- 172.21.15.103:0/626328200 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400bd70 con 0x7f511c105bf0 2024-08-19T01:45:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.995+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511c19f250 con 0x7f511c105bf0 2024-08-19T01:45:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.995+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511c1ab5c0 con 0x7f511c105bf0 2024-08-19T01:45:16.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.995+0000 7f5119ffb640 1 -- 172.21.15.103:0/626328200 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5114003e20 con 0x7f511c105bf0 2024-08-19T01:45:17.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.996+0000 7f5119ffb640 1 -- 172.21.15.103:0/626328200 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140106c0 con 0x7f511c105bf0 2024-08-19T01:45:17.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.998+0000 7f51246fe740 1 Processor -- start 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.998+0000 7f51246fe740 1 -- start start 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f51246fe740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a58652eb0 0x558a58653290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f51246fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a5866eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f51246fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a5866f1e0 0x558a586715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f51246fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a585b8a40 con 0x558a5866f1e0 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f51246fe740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558a585509d0 con 0x558a58652eb0 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f51246fe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558a585b4a00 con 0x558a586537d0 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f511bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a5866eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:16.999+0000 7f5113fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a58652eb0 0x558a58653290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.000+0000 7f5113fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a58652eb0 0x558a58653290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59170/0 (socket says 172.21.15.103:59170) 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.000+0000 7f5113fff640 1 -- 172.21.15.103:0/1941931761 learned_addr learned my addr 172.21.15.103:0/1941931761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:17.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.000+0000 7f5113fff640 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a5866f1e0 0x558a586715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.002+0000 7f5113fff640 1 -- 172.21.15.103:0/1941931761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 msgr2=0x558a5866eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.002+0000 7f5113fff640 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a5866eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.002+0000 7f5113fff640 1 -- 172.21.15.103:0/1941931761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a5866f1e0 msgr2=0x558a586715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.002+0000 7f5113fff640 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a5866f1e0 0x558a586715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.002+0000 7f5113fff640 1 -- 172.21.15.103:0/1941931761 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a58557650 con 0x558a58652eb0 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.002+0000 7f5113fff640 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a58652eb0 0x558a58653290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5104005c40 tx=0x7f5104005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd9e6ad8207b1fe6 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.003+0000 7f51127fc640 1 -- 172.21.15.103:0/1941931761 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104035070 con 0x558a58652eb0 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.003+0000 7f5119ffb640 1 -- 172.21.15.103:0/626328200 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5114003970 con 0x7f511c105bf0 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.003+0000 7f51127fc640 1 -- 172.21.15.103:0/1941931761 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5104005500 con 0x558a58652eb0 2024-08-19T01:45:17.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.003+0000 7f51127fc640 1 -- 172.21.15.103:0/1941931761 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104004e40 con 0x558a58652eb0 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f51246fe740 1 -- 172.21.15.103:0/1941931761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a58652eb0 msgr2=0x558a58653290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f51246fe740 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a58652eb0 0x558a58653290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5104005c40 tx=0x7f5104005b50 comp rx=0 tx=0).stop 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f5119ffb640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51080a2c40 0x7f51080a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f51246fe740 1 -- 172.21.15.103:0/1941931761 shutdown_connections 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f51246fe740 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a5866f1e0 0x558a586715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f51246fe740 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a5866eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f51246fe740 1 --2- 172.21.15.103:0/1941931761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a58652eb0 0x558a58653290 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.004+0000 7f51246fe740 1 -- 172.21.15.103:0/1941931761 >> 172.21.15.103:0/1941931761 conn(0x558a5864f580 msgr2=0x558a5864f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.005+0000 7f5119ffb640 1 -- 172.21.15.103:0/626328200 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f51140cec20 con 0x7f511c105bf0 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.005+0000 7f5113fff640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51080a2c40 0x7f51080a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.005+0000 7f51246fe740 1 -- 172.21.15.103:0/1941931761 shutdown_connections 2024-08-19T01:45:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.005+0000 7f51246fe740 1 -- 172.21.15.103:0/1941931761 wait complete. 2024-08-19T01:45:17.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.005+0000 7f51246fe740 1 -- start start 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f51246fe740 1 Processor -- start 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f5113fff640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51080a2c40 0x7f51080a5100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5104005ab0 tx=0x7f510402a440 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f51246fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a5866c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f51246fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a5866c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f51246fe740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 0x558a5866cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f51246fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a58671da0 con 0x558a58652eb0 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f51246fe740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558a58573b20 con 0x558a5866f1e0 2024-08-19T01:45:17.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.006+0000 7f51246fe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558a58671f20 con 0x558a586537d0 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f51246fe740 10 client.0 _ll_register_callbacks cb 0x558a58571bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a5866c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a5866c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40592/0 (socket says 172.21.15.103:40592) 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 -- 172.21.15.103:0/479418182 learned_addr learned my addr 172.21.15.103:0/479418182 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f512089d640 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 0x558a5866cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 -- 172.21.15.103:0/479418182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 msgr2=0x558a5866c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a5866c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 -- 172.21.15.103:0/479418182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 msgr2=0x558a5866cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 0x558a5866cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 -- 172.21.15.103:0/479418182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a5832a3c0 con 0x558a58652eb0 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f511bfff640 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a5866c090 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f510c011660 tx=0x7f510c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397f55b4c7ac2a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:17.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.007+0000 7f50fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:45:17.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.009+0000 7f50fb7fe640 1 -- 172.21.15.103:0/479418182 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c018070 con 0x558a58652eb0 2024-08-19T01:45:17.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.009+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5104005500 con 0x558a58652eb0 2024-08-19T01:45:17.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.009+0000 7f50fb7fe640 1 -- 172.21.15.103:0/479418182 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f510c00fb40 con 0x558a58652eb0 2024-08-19T01:45:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.009+0000 7f50fb7fe640 1 -- 172.21.15.103:0/479418182 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f510c00fd40 con 0x558a58652eb0 2024-08-19T01:45:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.009+0000 7f51246fe740 10 client.17739 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.009+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558a5867daa0 con 0x558a58652eb0 2024-08-19T01:45:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f50fb7fe640 1 -- 172.21.15.103:0/479418182 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5104005500 con 0x558a58652eb0 2024-08-19T01:45:17.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f50fb7fe640 1 -- 172.21.15.103:0/479418182 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f510c003af0 con 0x558a58652eb0 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f51246fe740 10 client.17739 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f51246fe740 10 client.17739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f51246fe740 20 client.17739 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f51246fe740 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a5867f420 0x558a586818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f51246fe740 4 client.17739 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a585c5fb0 con 0x558a5867f420 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.010+0000 7f5113fff640 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a5867f420 0x558a586818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.011+0000 7f5113fff640 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a5867f420 0x558a586818a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66e0bba73cfe650b server_cookie=25d6a36145a68726 in_seq=0 out_seq=0 2024-08-19T01:45:17.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.011+0000 7f50fb7fe640 10 client.17739 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:45:17.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f50fb7fe640 1 -- 172.21.15.103:0/479418182 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a585c5fb0 con 0x558a5867f420 2024-08-19T01:45:17.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f50fb7fe640 10 client.17739 handle_command_reply: tid=0 2024-08-19T01:45:17.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 client.17739 shutdown 2024-08-19T01:45:17.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 msgr2=0x558a5866c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:17.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a5866c090 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f510c011660 tx=0x7f510c011b20 comp rx=0 tx=0).stop 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 shutdown_connections 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a5867f420 0x558a586818a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 0x558a5866cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a5866c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 --2- 172.21.15.103:0/479418182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a5866c090 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 >> 172.21.15.103:0/479418182 conn(0x558a5864f580 msgr2=0x558a586552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.012+0000 7f50fb7fe640 0 client.17739 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.013+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 shutdown_connections 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.013+0000 7f51246fe740 1 -- 172.21.15.103:0/479418182 wait complete. 2024-08-19T01:45:17.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.013+0000 7f51246fe740 20 client.17739 trim_cache size 0 max 16384 2024-08-19T01:45:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 Processor -- start 2024-08-19T01:45:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 -- start start 2024-08-19T01:45:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a586871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a586537d0 0x558a586876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a5866f1e0 0x558a58654950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc00e380 con 0x558a58652eb0 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc00ff80 con 0x558a586537d0 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f51246fe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc00ea50 con 0x558a5866f1e0 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f5113fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a586537d0 0x558a586876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f5113fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a586537d0 0x558a586876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59182/0 (socket says 172.21.15.103:59182) 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.037+0000 7f5113fff640 1 -- 172.21.15.103:0/687654025 learned_addr learned my addr 172.21.15.103:0/687654025 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.038+0000 7f512089d640 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a5866f1e0 0x558a58654950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.038+0000 7f5113fff640 1 -- 172.21.15.103:0/687654025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a5866f1e0 msgr2=0x558a58654950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.038+0000 7f5113fff640 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a5866f1e0 0x558a58654950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.038+0000 7f5113fff640 1 -- 172.21.15.103:0/687654025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 msgr2=0x558a586871b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.038+0000 7f5113fff640 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a586871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.038+0000 7f5113fff640 1 -- 172.21.15.103:0/687654025 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a58653600 con 0x558a586537d0 2024-08-19T01:45:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.038+0000 7f5113fff640 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a586537d0 0x558a586876f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5104009900 tx=0x7f51040044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e09ed924373d68 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.039+0000 7f5111ffb640 1 -- 172.21.15.103:0/687654025 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104063070 con 0x558a586537d0 2024-08-19T01:45:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.039+0000 7f5111ffb640 1 -- 172.21.15.103:0/687654025 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5104004760 con 0x558a586537d0 2024-08-19T01:45:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.039+0000 7f5111ffb640 1 -- 172.21.15.103:0/687654025 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104031700 con 0x558a586537d0 2024-08-19T01:45:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 -- 172.21.15.103:0/687654025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a586537d0 msgr2=0x558a586876f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a586537d0 0x558a586876f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5104009900 tx=0x7f51040044c0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 -- 172.21.15.103:0/687654025 shutdown_connections 2024-08-19T01:45:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a5866f1e0 0x558a58654950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a586537d0 0x558a586876f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 --2- 172.21.15.103:0/687654025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a586871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 -- 172.21.15.103:0/687654025 >> 172.21.15.103:0/687654025 conn(0x558a5864f580 msgr2=0x558a5866b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 -- 172.21.15.103:0/687654025 shutdown_connections 2024-08-19T01:45:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.040+0000 7f51246fe740 1 -- 172.21.15.103:0/687654025 wait complete. 2024-08-19T01:45:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.041+0000 7f51246fe740 1 -- start start 2024-08-19T01:45:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.041+0000 7f51246fe740 1 Processor -- start 2024-08-19T01:45:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.042+0000 7f51246fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a58672240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.042+0000 7f51246fe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a58672780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.042+0000 7f51246fe740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 0x558a58672cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.042+0000 7f51246fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc00f8b0 con 0x558a58652eb0 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.042+0000 7f51246fe740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc00f120 con 0x558a5866f1e0 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.042+0000 7f51246fe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc00c360 con 0x558a586537d0 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.042+0000 7f51246fe740 10 client.0 _ll_register_callbacks cb 0x7f510c005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a58672240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a58672240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40608/0 (socket says 172.21.15.103:40608) 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 -- 172.21.15.103:0/1829700027 learned_addr learned my addr 172.21.15.103:0/1829700027 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 -- 172.21.15.103:0/1829700027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 msgr2=0x558a58672780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a58672780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 -- 172.21.15.103:0/1829700027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 msgr2=0x558a58672cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 0x558a58672cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 -- 172.21.15.103:0/1829700027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a586e8410 con 0x558a58652eb0 2024-08-19T01:45:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.043+0000 7f511bfff640 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a58672240 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f510c00b6e0 tx=0x7f510c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22afaf466d7a231c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.044+0000 7f50fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:45:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.044+0000 7f50fb7fe640 1 -- 172.21.15.103:0/1829700027 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c035060 con 0x558a58652eb0 2024-08-19T01:45:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.044+0000 7f50fb7fe640 1 -- 172.21.15.103:0/1829700027 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f510c0041e0 con 0x558a58652eb0 2024-08-19T01:45:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.044+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5104004760 con 0x558a58652eb0 2024-08-19T01:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.048+0000 7f50fb7fe640 1 -- 172.21.15.103:0/1829700027 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f510c018040 con 0x558a58652eb0 2024-08-19T01:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.049+0000 7f51246fe740 10 client.17742 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.049+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558a586e96d0 con 0x558a58652eb0 2024-08-19T01:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.049+0000 7f50fb7fe640 1 -- 172.21.15.103:0/1829700027 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5104004760 con 0x558a58652eb0 2024-08-19T01:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.050+0000 7f50fb7fe640 1 -- 172.21.15.103:0/1829700027 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f510c005230 con 0x558a58652eb0 2024-08-19T01:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.050+0000 7f51246fe740 10 client.17742 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.050+0000 7f51246fe740 10 client.17742 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-19T01:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.050+0000 7f51246fe740 20 client.17742 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.050+0000 7f51246fe740 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a586e9e50 0x558a586ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.050+0000 7f51246fe740 4 client.17742 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.050+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 --> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a585c04c0 con 0x558a586e9e50 2024-08-19T01:45:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.052+0000 7f5113fff640 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a586e9e50 0x558a586ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.053+0000 7f5113fff640 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a586e9e50 0x558a586ec290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=480c2e910f5af96a server_cookie=47b4bef7b7f92a31 in_seq=0 out_seq=0 2024-08-19T01:45:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.053+0000 7f50fb7fe640 10 client.17742 ms_handle_connect on v2:172.21.15.190:6836/66432346 2024-08-19T01:45:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.054+0000 7f50fb7fe640 1 -- 172.21.15.103:0/1829700027 <== mds.2 v2:172.21.15.190:6836/66432346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a585c04c0 con 0x558a586e9e50 2024-08-19T01:45:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.054+0000 7f50fb7fe640 10 client.17742 handle_command_reply: tid=0 2024-08-19T01:45:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.054+0000 7f51246fe740 1 client.17742 shutdown 2024-08-19T01:45:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.056+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 msgr2=0x558a58672240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.056+0000 7f51246fe740 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a58672240 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f510c00b6e0 tx=0x7f510c00bf00 comp rx=0 tx=0).stop 2024-08-19T01:45:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.056+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 shutdown_connections 2024-08-19T01:45:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.056+0000 7f51246fe740 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.190:6836/66432346,v1:172.21.15.190:6837/66432346] conn(0x558a586e9e50 0x558a586ec290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.056+0000 7f51246fe740 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a5866f1e0 0x558a58672cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.056+0000 7f51246fe740 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558a586537d0 0x558a58672780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.057+0000 7f51246fe740 1 --2- 172.21.15.103:0/1829700027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a58652eb0 0x558a58672240 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.057+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 >> 172.21.15.103:0/1829700027 conn(0x558a5864f580 msgr2=0x558a5864dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.057+0000 7f50fb7fe640 0 client.17742 ms_handle_reset on v2:172.21.15.190:6836/66432346 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.057+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 shutdown_connections 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.057+0000 7f51246fe740 1 -- 172.21.15.103:0/1829700027 wait complete. 2024-08-19T01:45:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.058+0000 7f51246fe740 20 client.17742 trim_cache size 0 max 16384 2024-08-19T01:45:17.060 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.058+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51080a2c40 msgr2=0x7f51080a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.058+0000 7f5122300640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51080a2c40 0x7f51080a5100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5104005ab0 tx=0x7f510402a440 comp rx=0 tx=0).stop 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.058+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f511c105bf0 msgr2=0x7f511c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.058+0000 7f5122300640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f511c105bf0 0x7f511c19e730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f511400da10 tx=0x7f511400ded0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 shutdown_connections 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f51080a2c40 0x7f51080a5100 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f511c105bf0 0x7f511c19e730 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f511c100f10 0x7f511c1a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 --2- 172.21.15.103:0/626328200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c1005f0 0x7f511c1a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 >> 172.21.15.103:0/626328200 conn(0x7f511c0fc500 msgr2=0x7f511c10f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 shutdown_connections 2024-08-19T01:45:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:17.059+0000 7f5122300640 1 -- 172.21.15.103:0/626328200 wait complete. 2024-08-19T01:45:17.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:17 smithi103 ceph-mon[18436]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 6.53k op/s 2024-08-19T01:45:17.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:17 smithi190 ceph-mon[22032]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 6.53k op/s 2024-08-19T01:45:17.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:17 smithi146 ceph-mon[22098]: pgmap v624: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 MiB/s wr, 6.53k op/s 2024-08-19T01:45:17.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-19T01:45:18.265 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:18.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.900+0000 7f17c2f21640 1 -- 172.21.15.103:0/3412083010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc104450 msgr2=0x7f17bc110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.900+0000 7f17c2f21640 1 --2- 172.21.15.103:0/3412083010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc104450 0x7f17bc110980 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009cd0 tx=0x7f17b402a8f0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.901+0000 7f17c2f21640 1 -- 172.21.15.103:0/3412083010 shutdown_connections 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.901+0000 7f17c2f21640 1 --2- 172.21.15.103:0/3412083010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc104450 0x7f17bc110980 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.901+0000 7f17c2f21640 1 --2- 172.21.15.103:0/3412083010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17bc103ab0 0x7f17bc103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.901+0000 7f17c2f21640 1 --2- 172.21.15.103:0/3412083010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17bc109ab0 0x7f17bc109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.901+0000 7f17c2f21640 1 -- 172.21.15.103:0/3412083010 >> 172.21.15.103:0/3412083010 conn(0x7f17bc0ff750 msgr2=0x7f17bc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.902+0000 7f17c2f21640 1 -- 172.21.15.103:0/3412083010 shutdown_connections 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.902+0000 7f17c2f21640 1 -- 172.21.15.103:0/3412083010 wait complete. 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 Processor -- start 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 -- start start 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc103ab0 0x7f17bc19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17bc104450 0x7f17bc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17bc109ab0 0x7f17bc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc113d80 con 0x7f17bc103ab0 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc113c00 con 0x7f17bc104450 2024-08-19T01:45:18.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c2f21640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc113f00 con 0x7f17bc109ab0 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c0cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc103ab0 0x7f17bc19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c0cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc103ab0 0x7f17bc19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40626/0 (socket says 172.21.15.103:40626) 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c0cbd640 1 -- 172.21.15.103:0/2533875765 learned_addr learned my addr 172.21.15.103:0/2533875765 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17bbfff640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17bc104450 0x7f17bc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.903+0000 7f17c14be640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17bc109ab0 0x7f17bc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17c0cbd640 1 -- 172.21.15.103:0/2533875765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17bc109ab0 msgr2=0x7f17bc19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17bc109ab0 0x7f17bc19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17c0cbd640 1 -- 172.21.15.103:0/2533875765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17bc104450 msgr2=0x7f17bc19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17bc104450 0x7f17bc19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17c0cbd640 1 -- 172.21.15.103:0/2533875765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc19f2d0 con 0x7f17bc103ab0 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc103ab0 0x7f17bc19a320 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f17b000d950 tx=0x7f17b000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17b9ffb640 1 -- 172.21.15.103:0/2533875765 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0017070 con 0x7f17bc103ab0 2024-08-19T01:45:18.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17b9ffb640 1 -- 172.21.15.103:0/2533875765 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b000de80 con 0x7f17bc103ab0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17b9ffb640 1 -- 172.21.15.103:0/2533875765 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b000f620 con 0x7f17bc103ab0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.904+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc19f5c0 con 0x7f17bc103ab0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.905+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc1a6ea0 con 0x7f17bc103ab0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.908+0000 7f17c531f740 1 Processor -- start 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.908+0000 7f17c531f740 1 -- start start 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c531f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x562346872290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c531f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623468727d0 0x56234688dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c531f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688e1e0 0x5623468905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c531f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623467d7a40 con 0x56234688e1e0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c531f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56234676f9d0 con 0x562346871eb0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c531f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5623467d3a00 con 0x5623468727d0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x562346872290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x562346872290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59208/0 (socket says 172.21.15.103:59208) 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 -- 172.21.15.103:0/3374668687 learned_addr learned my addr 172.21.15.103:0/3374668687 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:18.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17bbfff640 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688e1e0 0x5623468905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 -- 172.21.15.103:0/3374668687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623468727d0 msgr2=0x56234688dca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623468727d0 0x56234688dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 -- 172.21.15.103:0/3374668687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688e1e0 msgr2=0x5623468905d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688e1e0 0x5623468905d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.909+0000 7f17c14be640 1 -- 172.21.15.103:0/3374668687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562346776650 con 0x562346871eb0 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.910+0000 7f17c14be640 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x562346872290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009cd0 tx=0x7f17b4005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce74e86280b8f83c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.910+0000 7f17a6ffd640 1 -- 172.21.15.103:0/3374668687 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4030690 con 0x562346871eb0 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.910+0000 7f17a6ffd640 1 -- 172.21.15.103:0/3374668687 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b4036030 con 0x562346871eb0 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.911+0000 7f17c531f740 1 -- 172.21.15.103:0/3374668687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 msgr2=0x562346872290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.911+0000 7f17c531f740 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x562346872290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009cd0 tx=0x7f17b4005fa0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.912+0000 7f17c531f740 1 -- 172.21.15.103:0/3374668687 shutdown_connections 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.912+0000 7f17c531f740 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688e1e0 0x5623468905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.912+0000 7f17c531f740 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623468727d0 0x56234688dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.912+0000 7f17c531f740 1 --2- 172.21.15.103:0/3374668687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x562346872290 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.912+0000 7f17c531f740 1 -- 172.21.15.103:0/3374668687 >> 172.21.15.103:0/3374668687 conn(0x56234686e580 msgr2=0x56234686e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.913+0000 7f17b9ffb640 1 -- 172.21.15.103:0/2533875765 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f17b0009c30 con 0x7f17bc103ab0 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.914+0000 7f17c531f740 1 -- 172.21.15.103:0/3374668687 shutdown_connections 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.914+0000 7f17b9ffb640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17ac0a2ac0 0x7f17ac0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.914+0000 7f17bbfff640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17ac0a2ac0 0x7f17ac0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.914+0000 7f17b9ffb640 1 -- 172.21.15.103:0/2533875765 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f17b00cec30 con 0x7f17bc103ab0 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.915+0000 7f17c531f740 1 -- 172.21.15.103:0/3374668687 wait complete. 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.915+0000 7f17c531f740 1 -- start start 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.915+0000 7f17bbfff640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17ac0a2ac0 0x7f17ac0a4f80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x56234686f670 tx=0x7f17a8009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:18.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.915+0000 7f17c531f740 1 Processor -- start 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.916+0000 7f17c531f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x56234688b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.916+0000 7f17c531f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688e1e0 0x56234688b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.916+0000 7f17c531f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688bb90 0x56234689a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17bbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688e1e0 0x56234688b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17bbfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688e1e0 0x56234688b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:42968/0 (socket says 172.21.15.103:42968) 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17c14be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688bb90 0x56234689a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17c14be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688bb90 0x56234689a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40642/0 (socket says 172.21.15.103:40642) 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17bbfff640 1 -- 172.21.15.103:0/591221088 learned_addr learned my addr 172.21.15.103:0/591221088 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562346890df0 con 0x56234688bb90 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562346792b20 con 0x562346871eb0 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x562346890f70 con 0x56234688e1e0 2024-08-19T01:45:18.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x56234688b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.917+0000 7f17c531f740 10 client.0 _ll_register_callbacks cb 0x562346790bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f17bbfff640 1 -- 172.21.15.103:0/591221088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 msgr2=0x56234688b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f17bbfff640 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x56234688b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f17bbfff640 1 -- 172.21.15.103:0/591221088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688bb90 msgr2=0x56234689a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f17bbfff640 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688bb90 0x56234689a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f17bbfff640 1 -- 172.21.15.103:0/591221088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623465493c0 con 0x56234688e1e0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f17bbfff640 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688e1e0 0x56234688b650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17a80147c0 tx=0x7f17a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bdbfe9880e0f5fe server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17b4036030 con 0x56234688e1e0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f179ffff640 10 client.26953 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f179ffff640 1 -- 172.21.15.103:0/591221088 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8025070 con 0x56234688e1e0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.918+0000 7f179ffff640 1 -- 172.21.15.103:0/591221088 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f17a8021600 con 0x56234688e1e0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f179ffff640 1 -- 172.21.15.103:0/591221088 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17a8031050 con 0x56234688e1e0 2024-08-19T01:45:18.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 10 client.26953 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=39}) -- 0x56234689cf50 con 0x56234688e1e0 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f179ffff640 1 -- 172.21.15.103:0/591221088 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f17a8020070 con 0x56234688e1e0 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 10 client.26953 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 10 client.26953 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 20 client.26953 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56234689e350 0x5623468a0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 4 client.26953 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623467e4fb0 con 0x56234689e350 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.919+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56234689e350 0x5623468a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.920+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56234689e350 0x5623468a0740 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=960c8f4b4cf773c8 server_cookie=203e3fb2ac976578 in_seq=0 out_seq=0 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.921+0000 7f179ffff640 10 client.26953 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.921+0000 7f179ffff640 1 -- 172.21.15.103:0/591221088 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623467e4fb0 con 0x56234689e350 2024-08-19T01:45:18.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.921+0000 7f179ffff640 10 client.26953 handle_command_reply: tid=0 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.921+0000 7f17c531f740 1 client.26953 shutdown 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.921+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688e1e0 msgr2=0x56234688b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f17c531f740 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688e1e0 0x56234688b650 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17a80147c0 tx=0x7f17a80046c0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 shutdown_connections 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f17c531f740 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x56234689e350 0x5623468a0740 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f17c531f740 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234688bb90 0x56234689a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f17c531f740 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688e1e0 0x56234688b650 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17a80147c0 tx=0x7f17a80046c0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f17c531f740 1 --2- 172.21.15.103:0/591221088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562346871eb0 0x56234688b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 >> 172.21.15.103:0/591221088 conn(0x56234686e580 msgr2=0x5623468742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.922+0000 7f179ffff640 0 client.26953 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.923+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 shutdown_connections 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.923+0000 7f17c531f740 1 -- 172.21.15.103:0/591221088 wait complete. 2024-08-19T01:45:18.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.923+0000 7f17c531f740 20 client.26953 trim_cache size 0 max 16384 2024-08-19T01:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.949+0000 7f17c531f740 1 Processor -- start 2024-08-19T01:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.949+0000 7f17c531f740 1 -- start start 2024-08-19T01:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c531f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x56234688a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c531f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x56234689a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c531f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x5623468a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c531f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc073c80 con 0x562346871eb0 2024-08-19T01:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c531f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1a87b0 con 0x56234688bb90 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c531f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1a7280 con 0x56234688a0f0 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c0cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x56234688a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c0cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x56234688a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40652/0 (socket says 172.21.15.103:40652) 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c0cbd640 1 -- 172.21.15.103:0/3625043702 learned_addr learned my addr 172.21.15.103:0/3625043702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17bbfff640 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x56234689a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.950+0000 7f17c14be640 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x5623468a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.951+0000 7f17c0cbd640 1 -- 172.21.15.103:0/3625043702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 msgr2=0x5623468a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.951+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x5623468a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.951+0000 7f17c0cbd640 1 -- 172.21.15.103:0/3625043702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 msgr2=0x56234689a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.951+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x56234689a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.951+0000 7f17c0cbd640 1 -- 172.21.15.103:0/3625043702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562346872600 con 0x562346871eb0 2024-08-19T01:45:18.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.951+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x56234688a9d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f17b00978c0 tx=0x7f17b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8e0419cd299e2e server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:18.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.953+0000 7f17a67fc640 1 -- 172.21.15.103:0/3625043702 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0020040 con 0x562346871eb0 2024-08-19T01:45:18.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.953+0000 7f17a67fc640 1 -- 172.21.15.103:0/3625043702 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b0082050 con 0x562346871eb0 2024-08-19T01:45:18.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17a67fc640 1 -- 172.21.15.103:0/3625043702 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0097ad0 con 0x562346871eb0 2024-08-19T01:45:18.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17c531f740 1 -- 172.21.15.103:0/3625043702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 msgr2=0x56234688a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17c531f740 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x56234688a9d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f17b00978c0 tx=0x7f17b0000bc0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17c531f740 1 -- 172.21.15.103:0/3625043702 shutdown_connections 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17c531f740 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x5623468a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17c531f740 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x56234689a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17c531f740 1 --2- 172.21.15.103:0/3625043702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x56234688a9d0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.954+0000 7f17c531f740 1 -- 172.21.15.103:0/3625043702 >> 172.21.15.103:0/3625043702 conn(0x56234686e580 msgr2=0x5623468899e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.955+0000 7f17c531f740 1 -- 172.21.15.103:0/3625043702 shutdown_connections 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.955+0000 7f17c531f740 1 -- 172.21.15.103:0/3625043702 wait complete. 2024-08-19T01:45:18.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.955+0000 7f17c531f740 1 -- start start 2024-08-19T01:45:18.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.955+0000 7f17c531f740 1 Processor -- start 2024-08-19T01:45:18.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c531f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x5623468964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c531f740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x562346896a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c531f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x562346901950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c531f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1a80e0 con 0x562346871eb0 2024-08-19T01:45:18.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c531f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1a7950 con 0x56234688bb90 2024-08-19T01:45:18.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c531f740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc071c60 con 0x56234688a0f0 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c531f740 10 client.0 _ll_register_callbacks cb 0x7f17a8009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c14be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x562346901950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c14be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x562346901950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59240/0 (socket says 172.21.15.103:59240) 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17c14be640 1 -- 172.21.15.103:0/2478131817 learned_addr learned my addr 172.21.15.103:0/2478131817 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.956+0000 7f17bbfff640 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x562346896a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.957+0000 7f17c14be640 1 -- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 msgr2=0x562346896a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.957+0000 7f17c14be640 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x562346896a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.957+0000 7f17c14be640 1 -- 172.21.15.103:0/2478131817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 msgr2=0x5623468964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:18.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.957+0000 7f17c14be640 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x5623468964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.957+0000 7f17c14be640 1 -- 172.21.15.103:0/2478131817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623469021d0 con 0x56234688bb90 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.957+0000 7f17c14be640 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x562346901950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f17b4006010 tx=0x7f17b4004960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa816c1aa3d42cc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.957+0000 7f179ffff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.958+0000 7f179ffff640 1 -- 172.21.15.103:0/2478131817 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4030690 con 0x56234688bb90 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.958+0000 7f179ffff640 1 -- 172.21.15.103:0/2478131817 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f17b4003b80 con 0x56234688bb90 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.958+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17b0082050 con 0x56234688bb90 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.958+0000 7f179ffff640 1 -- 172.21.15.103:0/2478131817 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17b4003d80 con 0x56234688bb90 2024-08-19T01:45:18.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.958+0000 7f17c531f740 10 client.27005 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.958+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562346904c60 con 0x56234688bb90 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.958+0000 7f179ffff640 1 -- 172.21.15.103:0/2478131817 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b0082050 con 0x56234688bb90 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.959+0000 7f179ffff640 1 -- 172.21.15.103:0/2478131817 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f17b4043070 con 0x56234688bb90 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.959+0000 7f17c531f740 10 client.27005 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.959+0000 7f17c531f740 10 client.27005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.959+0000 7f17c531f740 20 client.27005 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.959+0000 7f17c531f740 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x562346904fa0 0x562346907400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.959+0000 7f17c531f740 4 client.27005 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:45:18.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.959+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 --> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623467df4c0 con 0x562346904fa0 2024-08-19T01:45:18.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.960+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x562346904fa0 0x562346907400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:18.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.961+0000 7f17c0cbd640 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x562346904fa0 0x562346907400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb83862fa1b27979 server_cookie=d93e0d710cd424fd in_seq=0 out_seq=0 2024-08-19T01:45:18.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.961+0000 7f179ffff640 10 client.27005 ms_handle_connect on v2:172.21.15.190:6838/639745782 2024-08-19T01:45:18.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f179ffff640 1 -- 172.21.15.103:0/2478131817 <== mds.1 v2:172.21.15.190:6838/639745782 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623467df4c0 con 0x562346904fa0 2024-08-19T01:45:18.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f179ffff640 10 client.27005 handle_command_reply: tid=0 2024-08-19T01:45:18.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 client.27005 shutdown 2024-08-19T01:45:18.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 msgr2=0x562346901950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x562346901950 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f17b4006010 tx=0x7f17b4004960 comp rx=0 tx=0).stop 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 shutdown_connections 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:6838/639745782,v1:172.21.15.190:6839/639745782] conn(0x562346904fa0 0x562346907400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56234688bb90 0x562346901950 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56234688a0f0 0x562346896a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 --2- 172.21.15.103:0/2478131817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562346871eb0 0x5623468964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 >> 172.21.15.103:0/2478131817 conn(0x56234686e580 msgr2=0x56234688c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f179ffff640 0 client.27005 ms_handle_reset on v2:172.21.15.190:6838/639745782 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 shutdown_connections 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.962+0000 7f17c531f740 1 -- 172.21.15.103:0/2478131817 wait complete. 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c531f740 20 client.27005 trim_cache size 0 max 16384 2024-08-19T01:45:18.965 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:45:18.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17ac0a2ac0 msgr2=0x7f17ac0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17ac0a2ac0 0x7f17ac0a4f80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x56234686f670 tx=0x7f17a8009110 comp rx=0 tx=0).stop 2024-08-19T01:45:18.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc103ab0 msgr2=0x7f17bc19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc103ab0 0x7f17bc19a320 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f17b000d950 tx=0x7f17b000dd50 comp rx=0 tx=0).stop 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 shutdown_connections 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f17ac0a2ac0 0x7f17ac0a4f80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17bc109ab0 0x7f17bc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17bc104450 0x7f17bc19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.963+0000 7f17c2f21640 1 --2- 172.21.15.103:0/2533875765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17bc103ab0 0x7f17bc19a320 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.964+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 >> 172.21.15.103:0/2533875765 conn(0x7f17bc0ff750 msgr2=0x7f17bc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.964+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 shutdown_connections 2024-08-19T01:45:18.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:18.964+0000 7f17c2f21640 1 -- 172.21.15.103:0/2533875765 wait complete. 2024-08-19T01:45:19.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:19 smithi103 ceph-mon[18436]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 6.43k op/s 2024-08-19T01:45:19.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:19 smithi190 ceph-mon[22032]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 6.43k op/s 2024-08-19T01:45:19.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:19 smithi146 ceph-mon[22098]: pgmap v625: 97 pgs: 97 active+clean; 13 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 6.43k op/s 2024-08-19T01:45:19.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-19T01:45:20.295 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 -- 172.21.15.103:0/286555295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 msgr2=0x7fe0c0073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 --2- 172.21.15.103:0/286555295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0073620 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009900 tx=0x7fe0b002a410 comp rx=0 tx=0).stop 2024-08-19T01:45:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 -- 172.21.15.103:0/286555295 shutdown_connections 2024-08-19T01:45:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 --2- 172.21.15.103:0/286555295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c010ab40 0x7fe0c01113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 --2- 172.21.15.103:0/286555295 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0c010a180 0x7fe0c010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 --2- 172.21.15.103:0/286555295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0073620 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 -- 172.21.15.103:0/286555295 >> 172.21.15.103:0/286555295 conn(0x7fe0c006ec50 msgr2=0x7fe0c0071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 -- 172.21.15.103:0/286555295 shutdown_connections 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 -- 172.21.15.103:0/286555295 wait complete. 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 Processor -- start 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.156+0000 7fe0c688f640 1 -- start start 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c688f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c688f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0c010a180 0x7fe0c01a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c688f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c010ab40 0x7fe0c01a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c688f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0114ad0 con 0x7fe0c0073220 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c688f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0114950 con 0x7fe0c010ab40 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c688f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0114c50 con 0x7fe0c010a180 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0109a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40682/0 (socket says 172.21.15.103:40682) 2024-08-19T01:45:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 -- 172.21.15.103:0/3160099199 learned_addr learned my addr 172.21.15.103:0/3160099199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bf7fe640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0c010a180 0x7fe0c01a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c010ab40 0x7fe0c01a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 -- 172.21.15.103:0/3160099199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0c010a180 msgr2=0x7fe0c01a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0c010a180 0x7fe0c01a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 -- 172.21.15.103:0/3160099199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c010ab40 msgr2=0x7fe0c01a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c010ab40 0x7fe0c01a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 -- 172.21.15.103:0/3160099199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c01a7ee0 con 0x7fe0c0073220 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c010ab40 0x7fe0c01a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bffff640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0109a90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0005940 tx=0x7fe0b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bd7fa640 1 -- 172.21.15.103:0/3160099199 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b003f030 con 0x7fe0c0073220 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bd7fa640 1 -- 172.21.15.103:0/3160099199 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b0003cb0 con 0x7fe0c0073220 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c01a8110 con 0x7fe0c0073220 2024-08-19T01:45:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.157+0000 7fe0bd7fa640 1 -- 172.21.15.103:0/3160099199 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0035070 con 0x7fe0c0073220 2024-08-19T01:45:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.158+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c01a8650 con 0x7fe0c0073220 2024-08-19T01:45:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.160+0000 7fe0bd7fa640 1 -- 172.21.15.103:0/3160099199 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe0b0042030 con 0x7fe0c0073220 2024-08-19T01:45:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.160+0000 7fe0c964b740 1 Processor -- start 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.160+0000 7fe0c964b740 1 -- start start 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ecda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ed3050 0x55ed37ed5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c964b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed37e309d0 con 0x55ed37ecdfc0 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c964b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ed37dd8820 con 0x55ed37ed3050 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c964b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ed37e2c990 con 0x55ed37ecd680 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40684/0 (socket says 172.21.15.103:40684) 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 -- 172.21.15.103:0/2187708368 learned_addr learned my addr 172.21.15.103:0/2187708368 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bf7fe640 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ed3050 0x55ed37ed5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ecda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 -- 172.21.15.103:0/2187708368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 msgr2=0x55ed37ecda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ecda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 -- 172.21.15.103:0/2187708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ed3050 msgr2=0x55ed37ed5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ed3050 0x55ed37ed5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 -- 172.21.15.103:0/2187708368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed37b44210 con 0x55ed37ecdfc0 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bf7fe640 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ed3050 0x55ed37ed5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bd7fa640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0a80a2cb0 0x7fe0a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0a80a2cb0 0x7fe0a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed2a80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00f5710 tx=0x7fe0b00f5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd068c269d9f996c server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0bd7fa640 1 -- 172.21.15.103:0/3160099199 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe0b00ef590 con 0x7fe0c0073220 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.161+0000 7fe0a67fc640 1 -- 172.21.15.103:0/2187708368 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00b80c0 con 0x55ed37ecdfc0 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0a67fc640 1 -- 172.21.15.103:0/2187708368 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b00bc520 con 0x55ed37ecdfc0 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0a67fc640 1 -- 172.21.15.103:0/2187708368 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0003e60 con 0x55ed37ecdfc0 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 -- 172.21.15.103:0/2187708368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 msgr2=0x55ed37ed2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed2a80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00f5710 tx=0x7fe0b00f5800 comp rx=0 tx=0).stop 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0a80a2cb0 0x7fe0a80a5170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8005fd0 tx=0x7fe0b8009580 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 -- 172.21.15.103:0/2187708368 shutdown_connections 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ed3050 0x55ed37ed5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed2a80 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2187708368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ecda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 -- 172.21.15.103:0/2187708368 >> 172.21.15.103:0/2187708368 conn(0x55ed37ec8cf0 msgr2=0x55ed37ec90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 -- 172.21.15.103:0/2187708368 shutdown_connections 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.162+0000 7fe0c964b740 1 -- 172.21.15.103:0/2187708368 wait complete. 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 -- start start 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 Processor -- start 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecd680 0x55ed37ed7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37ede9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed37ed5de0 con 0x55ed37ecd680 2024-08-19T01:45:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55ed37dec520 con 0x55ed37ecdfc0 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55ed37ed5f60 con 0x55ed37ed3050 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0c964b740 10 client.0 _ll_register_callbacks cb 0x55ed37b43a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecd680 0x55ed37ed7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecd680 0x55ed37ed7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40690/0 (socket says 172.21.15.103:40690) 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 -- 172.21.15.103:0/1913259357 learned_addr learned my addr 172.21.15.103:0/1913259357 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bf7fe640 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 -- 172.21.15.103:0/1913259357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 msgr2=0x55ed37ede9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37ede9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 -- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 msgr2=0x55ed37ed79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.163+0000 7fe0bffff640 1 -- 172.21.15.103:0/1913259357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed37c10620 con 0x55ed37ecd680 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0bffff640 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecd680 0x55ed37ed7470 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00f5710 tx=0x7fe0b0031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2855e87d9f0b637 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe09f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe09f7fe640 1 -- 172.21.15.103:0/1913259357 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0049030 con 0x55ed37ecd680 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe09f7fe640 1 -- 172.21.15.103:0/1913259357 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe0b00ba120 con 0x55ed37ecd680 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b00bc520 con 0x55ed37ecd680 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe09f7fe640 1 -- 172.21.15.103:0/1913259357 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe0b00bb780 con 0x55ed37ecd680 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 10 client.17769 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed37ed5de0 con 0x55ed37ecd680 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe09f7fe640 1 -- 172.21.15.103:0/1913259357 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b00bc520 con 0x55ed37ecd680 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe09f7fe640 1 -- 172.21.15.103:0/1913259357 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fe0b00bc080 con 0x55ed37ecd680 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 10 client.17769 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 20 client.17769 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37ee2c30 0x55ed37ee5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 4 client.17769 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed37e3e100 con 0x55ed37ee2c30 2024-08-19T01:45:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.164+0000 7fe0bf7fe640 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37ee2c30 0x55ed37ee5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.165+0000 7fe09f7fe640 10 client.17769 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.165+0000 7fe0bf7fe640 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37ee2c30 0x55ed37ee5050 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0d7db2c33bbcd4 server_cookie=d39ad21264439a64 in_seq=0 out_seq=0 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.165+0000 7fe09f7fe640 1 -- 172.21.15.103:0/1913259357 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed37e3e100 con 0x55ed37ee2c30 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.165+0000 7fe09f7fe640 10 client.17769 handle_command_reply: tid=0 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.165+0000 7fe0c964b740 1 client.17769 shutdown 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecd680 msgr2=0x55ed37ed7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecd680 0x55ed37ed7470 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00f5710 tx=0x7fe0b0031980 comp rx=0 tx=0).stop 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 shutdown_connections 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37ee2c30 0x55ed37ee5050 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37ede9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ed79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 --2- 172.21.15.103:0/1913259357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecd680 0x55ed37ed7470 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe09f7fe640 0 client.17769 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 >> 172.21.15.103:0/1913259357 conn(0x55ed37ec8cf0 msgr2=0x55ed37eca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 shutdown_connections 2024-08-19T01:45:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 1 -- 172.21.15.103:0/1913259357 wait complete. 2024-08-19T01:45:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.166+0000 7fe0c964b740 20 client.17769 trim_cache size 0 max 16384 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 Processor -- start 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 -- start start 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ed7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ede910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ed3050 0x55ed37ec3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c01b1390 con 0x55ed37ed3050 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c01b2f90 con 0x55ed37ecdfc0 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c964b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c01b1a60 con 0x55ed37ecd680 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ed7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0bf7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ede910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ed7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.103:43014/0 (socket says 172.21.15.103:43014) 2024-08-19T01:45:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0bf7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ede910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59292/0 (socket says 172.21.15.103:59292) 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0bffff640 1 -- 172.21.15.103:0/2182570853 learned_addr learned my addr 172.21.15.103:0/2182570853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.196+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ed3050 0x55ed37ec3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0bffff640 1 -- 172.21.15.103:0/2182570853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 msgr2=0x55ed37ede910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ede910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0bffff640 1 -- 172.21.15.103:0/2182570853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ed3050 msgr2=0x55ed37ec3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ed3050 0x55ed37ec3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0bffff640 1 -- 172.21.15.103:0/2182570853 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed37ee56b0 con 0x55ed37ecd680 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ed7930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00f5930 tx=0x7fe0b00319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f12186fa5bc6fd63 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:21.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0a5ffb640 1 -- 172.21.15.103:0/2182570853 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0049070 con 0x55ed37ecd680 2024-08-19T01:45:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0a5ffb640 1 -- 172.21.15.103:0/2182570853 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b00bc520 con 0x55ed37ecd680 2024-08-19T01:45:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.197+0000 7fe0a5ffb640 1 -- 172.21.15.103:0/2182570853 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00bb960 con 0x55ed37ecd680 2024-08-19T01:45:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 -- 172.21.15.103:0/2182570853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 msgr2=0x55ed37ed7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ed7930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00f5930 tx=0x7fe0b00319f0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 -- 172.21.15.103:0/2182570853 shutdown_connections 2024-08-19T01:45:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ed3050 0x55ed37ec3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecdfc0 0x55ed37ede910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2182570853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ecd680 0x55ed37ed7930 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 -- 172.21.15.103:0/2182570853 >> 172.21.15.103:0/2182570853 conn(0x55ed37ec8cf0 msgr2=0x55ed37ed7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 -- 172.21.15.103:0/2182570853 shutdown_connections 2024-08-19T01:45:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.199+0000 7fe0c964b740 1 -- 172.21.15.103:0/2182570853 wait complete. 2024-08-19T01:45:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.200+0000 7fe0c964b740 1 -- start start 2024-08-19T01:45:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.200+0000 7fe0c964b740 1 Processor -- start 2024-08-19T01:45:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.200+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecd680 0x55ed37edbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.200+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37edc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.200+0000 7fe0c964b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37edd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecd680 0x55ed37edbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecd680 0x55ed37edbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.103:59306/0 (socket says 172.21.15.103:59306) 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0bffff640 1 -- 172.21.15.103:0/2851294659 learned_addr learned my addr 172.21.15.103:0/2851294659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c01b28c0 con 0x55ed37ecdfc0 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37edd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c01b2130 con 0x55ed37ecd680 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c01af370 con 0x55ed37ed3050 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c964b740 10 client.0 _ll_register_callbacks cb 0x7fe0b00c00c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0bf7fe640 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37edc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c4e2c640 1 -- 172.21.15.103:0/2851294659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecd680 msgr2=0x55ed37edbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecd680 0x55ed37edbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c4e2c640 1 -- 172.21.15.103:0/2851294659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 msgr2=0x55ed37edc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37edc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c4e2c640 1 -- 172.21.15.103:0/2851294659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed37edda80 con 0x55ed37ed3050 2024-08-19T01:45:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe0c4e2c640 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37edd470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009fd0 tx=0x7fe0b8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf9bde7fa5eca2bc server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.201+0000 7fe09f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe09f7fe640 1 -- 172.21.15.103:0/2851294659 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b802b020 con 0x55ed37ed3050 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe09f7fe640 1 -- 172.21.15.103:0/2851294659 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe0b801d6b0 con 0x55ed37ed3050 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b00bc520 con 0x55ed37ed3050 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe09f7fe640 1 -- 172.21.15.103:0/2851294659 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe0b801d8f0 con 0x55ed37ed3050 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe0c964b740 10 client.26974 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ed37f56120 con 0x55ed37ed3050 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe09f7fe640 1 -- 172.21.15.103:0/2851294659 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b00bc520 con 0x55ed37ed3050 2024-08-19T01:45:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.202+0000 7fe09f7fe640 1 -- 172.21.15.103:0/2851294659 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fe0b8026070 con 0x55ed37ed3050 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.203+0000 7fe0c964b740 10 client.26974 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.203+0000 7fe0c964b740 10 client.26974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.203+0000 7fe0c964b740 20 client.26974 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.203+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37f56b00 0x55ed37f58f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.203+0000 7fe0c964b740 4 client.26974 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.203+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed37e38450 con 0x55ed37f56b00 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.203+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37f56b00 0x55ed37f58f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.204+0000 7fe0bffff640 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37f56b00 0x55ed37f58f20 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3b7403f390876 server_cookie=5c943edb1f05e383 in_seq=0 out_seq=0 2024-08-19T01:45:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.204+0000 7fe09f7fe640 10 client.26974 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.218+0000 7fe09f7fe640 1 -- 172.21.15.103:0/2851294659 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed37e38450 con 0x55ed37f56b00 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.218+0000 7fe09f7fe640 10 client.26974 handle_command_reply: tid=0 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.218+0000 7fe0c964b740 1 client.26974 shutdown 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.220+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 msgr2=0x55ed37edd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.220+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37edd470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009fd0 tx=0x7fe0b8004ab0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 shutdown_connections 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55ed37f56b00 0x55ed37f58f20 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ed37ed3050 0x55ed37edd470 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed37ecdfc0 0x55ed37edc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 --2- 172.21.15.103:0/2851294659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55ed37ecd680 0x55ed37edbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 >> 172.21.15.103:0/2851294659 conn(0x55ed37ec8cf0 msgr2=0x55ed37ec7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe09f7fe640 0 client.26974 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 shutdown_connections 2024-08-19T01:45:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 1 -- 172.21.15.103:0/2851294659 wait complete. 2024-08-19T01:45:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.221+0000 7fe0c964b740 20 client.26974 trim_cache size 0 max 16384 2024-08-19T01:45:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-08-19T01:45:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-08-19T01:45:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-08-19T01:45:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-08-19T01:45:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0a80a2cb0 msgr2=0x7fe0a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0a80a2cb0 0x7fe0a80a5170 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8005fd0 tx=0x7fe0b8009580 comp rx=0 tx=0).stop 2024-08-19T01:45:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 msgr2=0x7fe0c0109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0109a90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0005940 tx=0x7fe0b00042f0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 shutdown_connections 2024-08-19T01:45:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe0a80a2cb0 0x7fe0a80a5170 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0c010ab40 0x7fe0c01a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0c010a180 0x7fe0c01a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 --2- 172.21.15.103:0/3160099199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0073220 0x7fe0c0109a90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 >> 172.21.15.103:0/3160099199 conn(0x7fe0c006ec50 msgr2=0x7fe0c010d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 shutdown_connections 2024-08-19T01:45:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-08-19T01:45:21.224+0000 7fe0c688f640 1 -- 172.21.15.103:0/3160099199 wait complete. 2024-08-19T01:45:21.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:21 smithi103 ceph-mon[18436]: pgmap v626: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 6.46k op/s 2024-08-19T01:45:21.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:21 smithi190 ceph-mon[22032]: pgmap v626: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 6.46k op/s 2024-08-19T01:45:21.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:21 smithi146 ceph-mon[22098]: pgmap v626: 97 pgs: 97 active+clean; 13 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 MiB/s wr, 6.46k op/s 2024-08-19T01:45:23.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:23 smithi103 ceph-mon[18436]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 7.25k op/s 2024-08-19T01:45:23.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:23 smithi190 ceph-mon[22032]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 7.25k op/s 2024-08-19T01:45:23.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:23 smithi146 ceph-mon[22098]: pgmap v627: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 28 MiB/s wr, 7.25k op/s 2024-08-19T01:45:24.952 INFO:tasks.workunit.client.0.smithi103.stdout:Wrote -1 instead of 4096 bytes. 2024-08-19T01:45:24.952 INFO:tasks.workunit.client.0.smithi103.stdout:Probably out of disk space 2024-08-19T01:45:24.953 INFO:tasks.workunit.client.0.smithi103.stderr:write: Bad file descriptor 2024-08-19T01:45:24.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:45:24.955 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-08-19T01:45:24.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-19T01:45:25.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:25 smithi103 ceph-mon[18436]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 6.04k op/s 2024-08-19T01:45:25.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:25 smithi190 ceph-mon[22032]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 6.04k op/s 2024-08-19T01:45:25.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:25 smithi146 ceph-mon[22098]: pgmap v628: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 6.04k op/s 2024-08-19T01:45:27.267 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi103 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=78215f5a1cf5eaf321d506a66539942e12851f34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-08-19T01:45:27.441 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7ac1a3b0f2a044d184cbc21944e8e37b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi103 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=78215f5a1cf5eaf321d506a66539942e12851f34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-08-19T01:45:27.443 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-08-19T01:45:27.453 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:45:27.467 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-19T01:45:27.477 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-19T01:45:27.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 -- 172.21.15.103:0/1079426439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301078b0 msgr2=0x7fc330107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 --2- 172.21.15.103:0/1079426439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301078b0 0x7fc330107c90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009930 tx=0x7fc32c02a420 comp rx=0 tx=0).stop 2024-08-19T01:45:27.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 -- 172.21.15.103:0/1079426439 shutdown_connections 2024-08-19T01:45:27.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 --2- 172.21.15.103:0/1079426439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc330102250 0x7fc33010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 --2- 172.21.15.103:0/1079426439 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3301018b0 0x7fc330101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 --2- 172.21.15.103:0/1079426439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301078b0 0x7fc330107c90 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 -- 172.21.15.103:0/1079426439 >> 172.21.15.103:0/1079426439 conn(0x7fc3300fd5f0 msgr2=0x7fc3300ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 -- 172.21.15.103:0/1079426439 shutdown_connections 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.165+0000 7fc337270640 1 -- 172.21.15.103:0/1079426439 wait complete. 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 Processor -- start 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 -- start start 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301018b0 0x7fc330074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc330102250 0x7fc330074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3301078b0 0x7fc330074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc330111b60 con 0x7fc3301018b0 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301119e0 con 0x7fc330102250 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc337270640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc330111ce0 con 0x7fc3301078b0 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc33500c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301018b0 0x7fc330074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc33500c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301018b0 0x7fc330074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51194/0 (socket says 172.21.15.103:51194) 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc33500c640 1 -- 172.21.15.103:0/1011093413 learned_addr learned my addr 172.21.15.103:0/1011093413 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc33580d640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3301078b0 0x7fc330074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.166+0000 7fc33480b640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc330102250 0x7fc330074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc33500c640 1 -- 172.21.15.103:0/1011093413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3301078b0 msgr2=0x7fc330074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc33500c640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3301078b0 0x7fc330074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc33500c640 1 -- 172.21.15.103:0/1011093413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc330102250 msgr2=0x7fc330074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc33500c640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc330102250 0x7fc330074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc33500c640 1 -- 172.21.15.103:0/1011093413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3301a65a0 con 0x7fc3301018b0 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc33500c640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301018b0 0x7fc330074220 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc32c004770 tx=0x7fc32c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc3267fc640 1 -- 172.21.15.103:0/1011093413 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c03f070 con 0x7fc3301018b0 2024-08-19T01:45:27.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc3267fc640 1 -- 172.21.15.103:0/1011093413 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc32c03b460 con 0x7fc3301018b0 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3301a6830 con 0x7fc3301018b0 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc3267fc640 1 -- 172.21.15.103:0/1011093413 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c035070 con 0x7fc3301018b0 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.167+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3301a6d60 con 0x7fc3301018b0 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.169+0000 7fc3267fc640 1 -- 172.21.15.103:0/1011093413 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc32c03b610 con 0x7fc3301018b0 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 Processor -- start 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 -- start start 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc3267fc640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc31c0a2c00 0x7fc31c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a07bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a07e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558179fd6a40 con 0x55817a07c1d0 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558179f6e9d0 con 0x55817a071910 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.170+0000 7fc33966e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x558179fd2a00 con 0x55817a070ff0 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc31c0a2c00 0x7fc31c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc3267fc640 1 -- 172.21.15.103:0/1011093413 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc32c0ef270 con 0x7fc3301018b0 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a07e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a07e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51198/0 (socket says 172.21.15.103:51198) 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 -- 172.21.15.103:0/3335806272 learned_addr learned my addr 172.21.15.103:0/3335806272 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33500c640 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33480b640 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a07bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 -- 172.21.15.103:0/3335806272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 msgr2=0x55817a0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 -- 172.21.15.103:0/3335806272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 msgr2=0x55817a07bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a07bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 -- 172.21.15.103:0/3335806272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558179f75650 con 0x55817a07c1d0 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33500c640 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc31c0a2c00 0x7fc31c0a50c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc328012c00 tx=0x7fc328009e00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc33580d640 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a07e5c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc328010800 tx=0x7fc32801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87facfd026d9d926 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc31b7fe640 1 -- 172.21.15.103:0/3335806272 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328024070 con 0x55817a07c1d0 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc31b7fe640 1 -- 172.21.15.103:0/3335806272 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc32800e820 con 0x55817a07c1d0 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.171+0000 7fc31b7fe640 1 -- 172.21.15.103:0/3335806272 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328005260 con 0x55817a07c1d0 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 -- 172.21.15.103:0/3335806272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 msgr2=0x55817a07e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a07e5c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc328010800 tx=0x7fc32801c000 comp rx=0 tx=0).stop 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 -- 172.21.15.103:0/3335806272 shutdown_connections 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a07e5c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a07bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 --2- 172.21.15.103:0/3335806272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 -- 172.21.15.103:0/3335806272 >> 172.21.15.103:0/3335806272 conn(0x55817a06d580 msgr2=0x55817a06d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 -- 172.21.15.103:0/3335806272 shutdown_connections 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.172+0000 7fc33966e740 1 -- 172.21.15.103:0/3335806272 wait complete. 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 -- start start 2024-08-19T01:45:27.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 Processor -- start 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a079100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a079640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a079b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55817a07eda0 con 0x55817a070ff0 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558179f91b20 con 0x55817a071910 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55817a07ef20 con 0x55817a07c1d0 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33966e740 10 client.0 _ll_register_callbacks cb 0x558179f8fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:27.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33500c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33500c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a079100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51200/0 (socket says 172.21.15.103:51200) 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.173+0000 7fc33500c640 1 -- 172.21.15.103:0/1465486412 learned_addr learned my addr 172.21.15.103:0/1465486412 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33580d640 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a079b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33480b640 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a079640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33500c640 1 -- 172.21.15.103:0/1465486412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 msgr2=0x55817a079b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33500c640 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a079b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33500c640 1 -- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 msgr2=0x55817a079640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33500c640 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a079640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33500c640 1 -- 172.21.15.103:0/1465486412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558179d483c0 con 0x55817a070ff0 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33500c640 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a079100 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005c20 tx=0x7fc32c0bb5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f6638eee2d59b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc318ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc318ff9640 1 -- 172.21.15.103:0/1465486412 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c048040 con 0x55817a070ff0 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc318ff9640 1 -- 172.21.15.103:0/1465486412 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc32c0b8220 con 0x55817a070ff0 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc32800e820 con 0x55817a070ff0 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc318ff9640 1 -- 172.21.15.103:0/1465486412 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc32c0c4770 con 0x55817a070ff0 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33966e740 10 client.17787 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:27.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55817a08aa30 con 0x55817a070ff0 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc318ff9640 1 -- 172.21.15.103:0/1465486412 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc32800e820 con 0x55817a070ff0 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.174+0000 7fc318ff9640 1 -- 172.21.15.103:0/1465486412 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc32c04b060 con 0x55817a070ff0 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33966e740 10 client.17787 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33966e740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33966e740 20 client.17787 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33966e740 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a08bee0 0x55817a08e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33966e740 4 client.17787 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-19T01:45:27.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558179fe3fb0 con 0x55817a08bee0 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33480b640 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a08bee0 0x55817a08e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc33480b640 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a08bee0 0x55817a08e330 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21563ee8c52323b server_cookie=da30a3d775ea1317 in_seq=0 out_seq=0 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.175+0000 7fc318ff9640 10 client.17787 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc318ff9640 1 -- 172.21.15.103:0/1465486412 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558179fe3fb0 con 0x55817a08bee0 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc318ff9640 10 client.17787 handle_command_reply: tid=0 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 client.17787 shutdown 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 msgr2=0x55817a079100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a079100 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005c20 tx=0x7fc32c0bb5d0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 shutdown_connections 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a08bee0 0x55817a08e330 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a079b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a079640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc318ff9640 0 client.17787 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 --2- 172.21.15.103:0/1465486412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a079100 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 >> 172.21.15.103:0/1465486412 conn(0x55817a06d580 msgr2=0x55817a0732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:27.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 shutdown_connections 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 1 -- 172.21.15.103:0/1465486412 wait complete. 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.176+0000 7fc33966e740 20 client.17787 trim_cache size 0 max 16384 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 Processor -- start 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 -- start start 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0789c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a093c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a094180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301af330 con 0x55817a07c1d0 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301b0f30 con 0x55817a071910 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33966e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301afa00 con 0x55817a070ff0 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a094180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a094180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51214/0 (socket says 172.21.15.103:51214) 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 -- 172.21.15.103:0/477742010 learned_addr learned my addr 172.21.15.103:0/477742010 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33480b640 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a093c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 -- 172.21.15.103:0/477742010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 msgr2=0x55817a0789c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33500c640 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0789c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 -- 172.21.15.103:0/477742010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 msgr2=0x55817a093c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a093c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 -- 172.21.15.103:0/477742010 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817a07e900 con 0x55817a07c1d0 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33480b640 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a093c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.190+0000 7fc33580d640 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a094180 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc328009440 tx=0x7fc328005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa79ddc8c3d3203 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc31affd640 1 -- 172.21.15.103:0/477742010 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328024050 con 0x55817a07c1d0 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc31affd640 1 -- 172.21.15.103:0/477742010 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc328009e70 con 0x55817a07c1d0 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc31affd640 1 -- 172.21.15.103:0/477742010 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3280050b0 con 0x55817a07c1d0 2024-08-19T01:45:27.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 -- 172.21.15.103:0/477742010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 msgr2=0x55817a094180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a094180 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc328009440 tx=0x7fc328005fd0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 -- 172.21.15.103:0/477742010 shutdown_connections 2024-08-19T01:45:27.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a07c1d0 0x55817a094180 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a093c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 --2- 172.21.15.103:0/477742010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a070ff0 0x55817a0789c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 -- 172.21.15.103:0/477742010 >> 172.21.15.103:0/477742010 conn(0x55817a06d580 msgr2=0x55817a0779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:27.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 -- 172.21.15.103:0/477742010 shutdown_connections 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.191+0000 7fc33966e740 1 -- 172.21.15.103:0/477742010 wait complete. 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 -- start start 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 Processor -- start 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a0841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a084700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301b0860 con 0x55817a070ff0 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301b00d0 con 0x55817a071910 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc3301ad310 con 0x55817a07c1d0 2024-08-19T01:45:27.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33966e740 10 client.0 _ll_register_callbacks cb 0x7fc32c0b7730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33500c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a0841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33500c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a0841c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51226/0 (socket says 172.21.15.103:51226) 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33500c640 1 -- 172.21.15.103:0/1747360796 learned_addr learned my addr 172.21.15.103:0/1747360796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33480b640 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a084700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.192+0000 7fc33580d640 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33500c640 1 -- 172.21.15.103:0/1747360796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 msgr2=0x55817a0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33500c640 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33500c640 1 -- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 msgr2=0x55817a084700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33500c640 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a084700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33500c640 1 -- 172.21.15.103:0/1747360796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817a075a30 con 0x55817a070ff0 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33500c640 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a0841c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0a57a0 tx=0x7fc32c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d153402e1e84b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc318ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc318ff9640 1 -- 172.21.15.103:0/1747360796 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c048070 con 0x55817a070ff0 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc318ff9640 1 -- 172.21.15.103:0/1747360796 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fc32c0a5360 con 0x55817a070ff0 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc328009e70 con 0x55817a070ff0 2024-08-19T01:45:27.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc318ff9640 1 -- 172.21.15.103:0/1747360796 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc32c0bb6f0 con 0x55817a070ff0 2024-08-19T01:45:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 10 client.17793 fetch_fsmap learned FSMap version 39 2024-08-19T01:45:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55817a075a30 con 0x55817a070ff0 2024-08-19T01:45:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc318ff9640 1 -- 172.21.15.103:0/1747360796 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc328009e70 con 0x55817a070ff0 2024-08-19T01:45:27.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc318ff9640 1 -- 172.21.15.103:0/1747360796 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fc32c0bab00 con 0x55817a070ff0 2024-08-19T01:45:27.746 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 10 client.17793 fetch_fsmap finished waiting for FSMap version 39 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 20 client.17793 populate_metadata read hostname 'smithi103' 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a0f6520 0x55817a0f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 4 client.17793 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.193+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 --> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558179fde4c0 con 0x55817a0f6520 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33480b640 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a0f6520 0x55817a0f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:27.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33480b640 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a0f6520 0x55817a0f8910 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f1a7aeb8ac1b5e server_cookie=18d01efbac0443f3 in_seq=0 out_seq=0 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc318ff9640 10 client.17793 ms_handle_connect on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc318ff9640 1 -- 172.21.15.103:0/1747360796 <== mds.0 v2:172.21.15.146:6838/1128555603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558179fde4c0 con 0x55817a0f6520 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc318ff9640 10 client.17793 handle_command_reply: tid=0 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 client.17793 shutdown 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 msgr2=0x55817a0841c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a0841c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0a57a0 tx=0x7fc32c009560 comp rx=0 tx=0).stop 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 shutdown_connections 2024-08-19T01:45:27.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:6838/1128555603,v1:172.21.15.146:6839/1128555603] conn(0x55817a0f6520 0x55817a0f8910 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55817a07c1d0 0x55817a0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55817a071910 0x55817a084700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 --2- 172.21.15.103:0/1747360796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55817a070ff0 0x55817a0841c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc318ff9640 0 client.17793 ms_handle_reset on v2:172.21.15.146:6838/1128555603 2024-08-19T01:45:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.194+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 >> 172.21.15.103:0/1747360796 conn(0x55817a06d580 msgr2=0x55817a07a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 shutdown_connections 2024-08-19T01:45:27.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc33966e740 1 -- 172.21.15.103:0/1747360796 wait complete. 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc33966e740 20 client.17793 trim_cache size 0 max 16384 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc31c0a2c00 msgr2=0x7fc31c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc31c0a2c00 0x7fc31c0a50c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc328012c00 tx=0x7fc328009e00 comp rx=0 tx=0).stop 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301018b0 msgr2=0x7fc330074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301018b0 0x7fc330074220 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc32c004770 tx=0x7fc32c0042f0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 shutdown_connections 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fc31c0a2c00 0x7fc31c0a50c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3301078b0 0x7fc330074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc330102250 0x7fc330074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 --2- 172.21.15.103:0/1011093413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc3301018b0 0x7fc330074220 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:27.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 >> 172.21.15.103:0/1011093413 conn(0x7fc3300fd5f0 msgr2=0x7fc3300fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:27.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 shutdown_connections 2024-08-19T01:45:27.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:26.195+0000 7fc337270640 1 -- 172.21.15.103:0/1011093413 wait complete. 2024-08-19T01:45:27.950 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:27 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:45:27.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:27 smithi146 ceph-mon[22098]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 5.68k op/s 2024-08-19T01:45:28.053 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:27 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:45:28.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:27 smithi103 ceph-mon[18436]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 5.68k op/s 2024-08-19T01:45:28.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:27 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:45:28.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:27 smithi190 ceph-mon[22032]: pgmap v629: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 5.68k op/s 2024-08-19T01:45:29.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:29 smithi103 ceph-mon[18436]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.08k op/s 2024-08-19T01:45:29.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:29 smithi190 ceph-mon[22032]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.08k op/s 2024-08-19T01:45:29.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:29 smithi146 ceph-mon[22098]: pgmap v630: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 5.08k op/s 2024-08-19T01:45:31.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:31 smithi103 ceph-mon[18436]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2024-08-19T01:45:31.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:31 smithi190 ceph-mon[22032]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2024-08-19T01:45:31.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:31 smithi146 ceph-mon[22098]: pgmap v631: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 4.35k op/s 2024-08-19T01:45:33.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:33 smithi103 ceph-mon[18436]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-08-19T01:45:33.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:33 smithi190 ceph-mon[22032]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-08-19T01:45:33.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:33 smithi146 ceph-mon[22098]: pgmap v632: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-08-19T01:45:33.526 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-19T01:45:33.530 INFO:tasks.fwd_scrub:done joining 2024-08-19T01:45:33.531 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-19T01:45:33.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-19T01:45:33.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 -- 172.21.15.103:0/36801817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc100760 msgr2=0x7ff1cc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/36801817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc100760 0x7ff1cc100b60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009900 tx=0x7ff1bc02a410 comp rx=0 tx=0).stop 2024-08-19T01:45:33.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 -- 172.21.15.103:0/36801817 shutdown_connections 2024-08-19T01:45:33.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/36801817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc102310 0x7ff1cc108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/36801817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1cc101950 0x7ff1cc101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/36801817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc100760 0x7ff1cc100b60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 -- 172.21.15.103:0/36801817 >> 172.21.15.103:0/36801817 conn(0x7ff1cc0fbec0 msgr2=0x7ff1cc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 -- 172.21.15.103:0/36801817 shutdown_connections 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.654+0000 7ff1d17a8640 1 -- 172.21.15.103:0/36801817 wait complete. 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 Processor -- start 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 -- start start 2024-08-19T01:45:33.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc100760 0x7ff1cc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1cc101950 0x7ff1cc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc102310 0x7ff1cc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d3e0 con 0x7ff1cc102310 2024-08-19T01:45:33.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d260 con 0x7ff1cc101950 2024-08-19T01:45:33.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1d17a8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d560 con 0x7ff1cc100760 2024-08-19T01:45:33.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc102310 0x7ff1cc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc102310 0x7ff1cc19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59348/0 (socket says 172.21.15.103:59348) 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1cb7fe640 1 -- 172.21.15.103:0/2605269487 learned_addr learned my addr 172.21.15.103:0/2605269487 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.655+0000 7ff1c27fc640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1cc101950 0x7ff1cc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1caffd640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc100760 0x7ff1cc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1cb7fe640 1 -- 172.21.15.103:0/2605269487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc100760 msgr2=0x7ff1cc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1cb7fe640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc100760 0x7ff1cc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1cb7fe640 1 -- 172.21.15.103:0/2605269487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1cc101950 msgr2=0x7ff1cc1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1cb7fe640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1cc101950 0x7ff1cc1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1cb7fe640 1 -- 172.21.15.103:0/2605269487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc19f7a0 con 0x7ff1cc102310 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1caffd640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc100760 0x7ff1cc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1cb7fe640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc102310 0x7ff1cc19f130 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cad0 tx=0x7ff1c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1c8ff9640 1 -- 172.21.15.103:0/2605269487 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400ae20 con 0x7ff1cc102310 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1c8ff9640 1 -- 172.21.15.103:0/2605269487 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1c4005020 con 0x7ff1cc102310 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc19fa90 con 0x7ff1cc102310 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1c8ff9640 1 -- 172.21.15.103:0/2605269487 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400f6b0 con 0x7ff1cc102310 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.656+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1a0030 con 0x7ff1cc102310 2024-08-19T01:45:33.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.657+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc107c90 con 0x7ff1cc102310 2024-08-19T01:45:33.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.658+0000 7ff1c8ff9640 1 -- 172.21.15.103:0/2605269487 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff1c40051d0 con 0x7ff1cc102310 2024-08-19T01:45:33.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.659+0000 7ff1c8ff9640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff1b80a2c10 0x7ff1b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.659+0000 7ff1caffd640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff1b80a2c10 0x7ff1b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.659+0000 7ff1c8ff9640 1 -- 172.21.15.103:0/2605269487 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7ff1c40cf380 con 0x7ff1cc102310 2024-08-19T01:45:33.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.659+0000 7ff1caffd640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff1b80a2c10 0x7ff1b80a50d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc0314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:33.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.663+0000 7ff1c8ff9640 1 -- 172.21.15.103:0/2605269487 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff1c4014030 con 0x7ff1cc102310 2024-08-19T01:45:33.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.830+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1cc02d830 con 0x7ff1cc102310 2024-08-19T01:45:33.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.831+0000 7ff1c8ff9640 1 -- 172.21.15.103:0/2605269487 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22704 (secure 0 0 0) 0x7ff1c409c390 con 0x7ff1cc102310 2024-08-19T01:45:33.833 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:45:33.834 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:34:01.874407+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:45:33.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff1b80a2c10 msgr2=0x7ff1b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff1b80a2c10 0x7ff1b80a50d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc0314a0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc102310 msgr2=0x7ff1cc19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc102310 0x7ff1cc19f130 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cad0 tx=0x7ff1c400cf90 comp rx=0 tx=0).stop 2024-08-19T01:45:33.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 shutdown_connections 2024-08-19T01:45:33.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7ff1b80a2c10 0x7ff1b80a50d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1cc102310 0x7ff1cc19f130 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.833+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1cc101950 0x7ff1cc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.834+0000 7ff1d17a8640 1 --2- 172.21.15.103:0/2605269487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1cc100760 0x7ff1cc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.834+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 >> 172.21.15.103:0/2605269487 conn(0x7ff1cc0fbec0 msgr2=0x7ff1cc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:33.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.834+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 shutdown_connections 2024-08-19T01:45:33.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.834+0000 7ff1d17a8640 1 -- 172.21.15.103:0/2605269487 wait complete. 2024-08-19T01:45:33.847 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-19T01:29:05.010792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-19T01:33:19.641931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-19T01:33:22.107240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-19T01:45:33.847 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-19T01:45:33.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 -- 172.21.15.103:0/1587004616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 msgr2=0x7feb74109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1587004616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb74109e00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009ca0 tx=0x7feb6c02a8c0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 -- 172.21.15.103:0/1587004616 shutdown_connections 2024-08-19T01:45:33.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1587004616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb74109e00 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1587004616 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb74102bb0 0x7feb74103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1587004616 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb740ffb40 0x7feb740fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 -- 172.21.15.103:0/1587004616 >> 172.21.15.103:0/1587004616 conn(0x7feb740fb680 msgr2=0x7feb740fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:33.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 -- 172.21.15.103:0/1587004616 shutdown_connections 2024-08-19T01:45:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.948+0000 7feb7bad8640 1 -- 172.21.15.103:0/1587004616 wait complete. 2024-08-19T01:45:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 Processor -- start 2024-08-19T01:45:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 -- start start 2024-08-19T01:45:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb740ffb40 0x7feb74197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb74102bb0 0x7feb74197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb7419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb7410d370 con 0x7feb74103570 2024-08-19T01:45:33.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feb7410d1f0 con 0x7feb740ffb40 2024-08-19T01:45:33.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7bad8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feb7410d4f0 con 0x7feb74102bb0 2024-08-19T01:45:33.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb7419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb7419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59358/0 (socket says 172.21.15.103:59358) 2024-08-19T01:45:33.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 -- 172.21.15.103:0/1474192429 learned_addr learned my addr 172.21.15.103:0/1474192429 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:33.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb79874640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb740ffb40 0x7feb74197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb79073640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb74102bb0 0x7feb74197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 -- 172.21.15.103:0/1474192429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb74102bb0 msgr2=0x7feb74197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb74102bb0 0x7feb74197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 -- 172.21.15.103:0/1474192429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb740ffb40 msgr2=0x7feb74197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb740ffb40 0x7feb74197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.949+0000 7feb7a075640 1 -- 172.21.15.103:0/1474192429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7419f640 con 0x7feb74103570 2024-08-19T01:45:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.950+0000 7feb7a075640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb7419efd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009dc0 tx=0x7feb6c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:33.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.950+0000 7feb66ffd640 1 -- 172.21.15.103:0/1474192429 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0317c0 con 0x7feb74103570 2024-08-19T01:45:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.950+0000 7feb66ffd640 1 -- 172.21.15.103:0/1474192429 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb6c0044f0 con 0x7feb74103570 2024-08-19T01:45:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.950+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7419f8d0 con 0x7feb74103570 2024-08-19T01:45:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.950+0000 7feb66ffd640 1 -- 172.21.15.103:0/1474192429 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c030cd0 con 0x7feb74103570 2024-08-19T01:45:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.950+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7419fe20 con 0x7feb74103570 2024-08-19T01:45:33.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.951+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb74108e00 con 0x7feb74103570 2024-08-19T01:45:33.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.952+0000 7feb66ffd640 1 -- 172.21.15.103:0/1474192429 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feb6c0046a0 con 0x7feb74103570 2024-08-19T01:45:33.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.953+0000 7feb66ffd640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb600a2b90 0x7feb600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:33.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.953+0000 7feb79874640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb600a2b90 0x7feb600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:33.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.953+0000 7feb66ffd640 1 -- 172.21.15.103:0/1474192429 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7feb6c0f0820 con 0x7feb74103570 2024-08-19T01:45:33.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.953+0000 7feb79874640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb600a2b90 0x7feb600a5050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb700058c0 tx=0x7feb70005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:33.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:33.956+0000 7feb66ffd640 1 -- 172.21.15.103:0/1474192429 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feb6c0bd7b0 con 0x7feb74103570 2024-08-19T01:45:34.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.125+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb7402d7e0 con 0x7feb74103570 2024-08-19T01:45:34.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.125+0000 7feb66ffd640 1 -- 172.21.15.103:0/1474192429 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7feb6c0bd0b0 con 0x7feb74103570 2024-08-19T01:45:34.127 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-08-19T01:45:34.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb600a2b90 msgr2=0x7feb600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb600a2b90 0x7feb600a5050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb700058c0 tx=0x7feb70005850 comp rx=0 tx=0).stop 2024-08-19T01:45:34.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 msgr2=0x7feb7419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb7419efd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009dc0 tx=0x7feb6c003ba0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 shutdown_connections 2024-08-19T01:45:34.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7feb600a2b90 0x7feb600a5050 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb74103570 0x7feb7419efd0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb74102bb0 0x7feb74197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 --2- 172.21.15.103:0/1474192429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb740ffb40 0x7feb74197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 >> 172.21.15.103:0/1474192429 conn(0x7feb740fb680 msgr2=0x7feb74105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:34.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 shutdown_connections 2024-08-19T01:45:34.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.128+0000 7feb7bad8640 1 -- 172.21.15.103:0/1474192429 wait complete. 2024-08-19T01:45:34.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:34 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2605269487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:45:34.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-19T01:45:34.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 -- 172.21.15.103:0/1223847329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d0101910 msgr2=0x7f80d0101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 --2- 172.21.15.103:0/1223847329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d0101910 0x7f80d0101d90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009900 tx=0x7f80bc02a410 comp rx=0 tx=0).stop 2024-08-19T01:45:34.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 -- 172.21.15.103:0/1223847329 shutdown_connections 2024-08-19T01:45:34.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 --2- 172.21.15.103:0/1223847329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d01022d0 0x7f80d0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 --2- 172.21.15.103:0/1223847329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d0101910 0x7f80d0101d90 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 --2- 172.21.15.103:0/1223847329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d0100720 0x7f80d0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 -- 172.21.15.103:0/1223847329 >> 172.21.15.103:0/1223847329 conn(0x7f80d00fbe80 msgr2=0x7f80d00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 -- 172.21.15.103:0/1223847329 shutdown_connections 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 -- 172.21.15.103:0/1223847329 wait complete. 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 Processor -- start 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 -- start start 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.241+0000 7f80d7049640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d0100720 0x7f80d0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d7049640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d0101910 0x7f80d0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d7049640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d01022d0 0x7f80d019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d7049640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010c2b0 con 0x7f80d01022d0 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d7049640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010c130 con 0x7f80d0100720 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d7049640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010c430 con 0x7f80d0101910 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d01022d0 0x7f80d019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d01022d0 0x7f80d019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59364/0 (socket says 172.21.15.103:59364) 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 -- 172.21.15.103:0/1793933086 learned_addr learned my addr 172.21.15.103:0/1793933086 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d4de5640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d0100720 0x7f80d0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80cffff640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d0101910 0x7f80d0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 -- 172.21.15.103:0/1793933086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d0101910 msgr2=0x7f80d0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d0101910 0x7f80d0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 -- 172.21.15.103:0/1793933086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d0100720 msgr2=0x7f80d0197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d0100720 0x7f80d0197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 -- 172.21.15.103:0/1793933086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d019f6a0 con 0x7f80d01022d0 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80cffff640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d0101910 0x7f80d0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d4de5640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d0100720 0x7f80d0197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d55e6640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d01022d0 0x7f80d019f030 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b970 tx=0x7f80c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:34.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80cdffb640 1 -- 172.21.15.103:0/1793933086 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8019070 con 0x7f80d01022d0 2024-08-19T01:45:34.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d019f990 con 0x7f80d01022d0 2024-08-19T01:45:34.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80cdffb640 1 -- 172.21.15.103:0/1793933086 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80c8014440 con 0x7f80d01022d0 2024-08-19T01:45:34.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.242+0000 7f80cdffb640 1 -- 172.21.15.103:0/1793933086 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80075d0 con 0x7f80d01022d0 2024-08-19T01:45:34.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.243+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d019ff10 con 0x7f80d01022d0 2024-08-19T01:45:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.244+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d01a01a0 con 0x7f80d01022d0 2024-08-19T01:45:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.245+0000 7f80cdffb640 1 -- 172.21.15.103:0/1793933086 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f80c801c070 con 0x7f80d01022d0 2024-08-19T01:45:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.245+0000 7f80cdffb640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f80c00a2b90 0x7f80c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.245+0000 7f80d4de5640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f80c00a2b90 0x7f80c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.245+0000 7f80cdffb640 1 -- 172.21.15.103:0/1793933086 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f80c80ce690 con 0x7f80d01022d0 2024-08-19T01:45:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.246+0000 7f80d4de5640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f80c00a2b90 0x7f80c00a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f80d0101770 tx=0x7f80c400a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.249+0000 7f80cdffb640 1 -- 172.21.15.103:0/1793933086 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f80c809b620 con 0x7f80d01022d0 2024-08-19T01:45:34.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:34 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2605269487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:45:34.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.417+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80d01a0370 con 0x7f80d01022d0 2024-08-19T01:45:34.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.418+0000 7f80cdffb640 1 -- 172.21.15.103:0/1793933086 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f80c809af20 con 0x7f80d01022d0 2024-08-19T01:45:34.420 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-08-19T01:45:34.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f80c00a2b90 msgr2=0x7f80c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f80c00a2b90 0x7f80c00a5050 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f80d0101770 tx=0x7f80c400a560 comp rx=0 tx=0).stop 2024-08-19T01:45:34.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d01022d0 msgr2=0x7f80d019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d01022d0 0x7f80d019f030 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b970 tx=0x7f80c800be30 comp rx=0 tx=0).stop 2024-08-19T01:45:34.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 shutdown_connections 2024-08-19T01:45:34.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f80c00a2b90 0x7f80c00a5050 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d01022d0 0x7f80d019f030 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d0101910 0x7f80d0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 --2- 172.21.15.103:0/1793933086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d0100720 0x7f80d0197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 >> 172.21.15.103:0/1793933086 conn(0x7f80d00fbe80 msgr2=0x7f80d00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:34.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 shutdown_connections 2024-08-19T01:45:34.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.421+0000 7f80d7049640 1 -- 172.21.15.103:0/1793933086 wait complete. 2024-08-19T01:45:34.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-19T01:45:34.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:34 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2605269487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:45:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.534+0000 7f152f74d640 1 -- 172.21.15.103:0/3123652650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528100730 msgr2=0x7f1528100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.534+0000 7f152f74d640 1 --2- 172.21.15.103:0/3123652650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528100730 0x7f1528100b30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1518009ca0 tx=0x7f151802a8c0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- 172.21.15.103:0/3123652650 shutdown_connections 2024-08-19T01:45:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 --2- 172.21.15.103:0/3123652650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15281022e0 0x7f1528108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 --2- 172.21.15.103:0/3123652650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1528101920 0x7f1528101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 --2- 172.21.15.103:0/3123652650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528100730 0x7f1528100b30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- 172.21.15.103:0/3123652650 >> 172.21.15.103:0/3123652650 conn(0x7f15280fbe70 msgr2=0x7f15280fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- 172.21.15.103:0/3123652650 shutdown_connections 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- 172.21.15.103:0/3123652650 wait complete. 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 Processor -- start 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- start start 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528100730 0x7f1528069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528101920 0x7f152819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15281022e0 0x7f152819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f152810c240 con 0x7f1528101920 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f152810c0c0 con 0x7f15281022e0 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.535+0000 7f152f74d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f152810c3c0 con 0x7f1528100730 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528101920 0x7f152819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528101920 0x7f152819c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59388/0 (socket says 172.21.15.103:59388) 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 -- 172.21.15.103:0/528991508 learned_addr learned my addr 172.21.15.103:0/528991508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152dcea640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15281022e0 0x7f152819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152d4e9640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528100730 0x7f1528069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 -- 172.21.15.103:0/528991508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528100730 msgr2=0x7f1528069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528100730 0x7f1528069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 -- 172.21.15.103:0/528991508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15281022e0 msgr2=0x7f152819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15281022e0 0x7f152819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 -- 172.21.15.103:0/528991508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152819f730 con 0x7f1528101920 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152dcea640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15281022e0 0x7f152819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152cce8640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528101920 0x7f152819c820 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f151c00d950 tx=0x7f151c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f15267fc640 1 -- 172.21.15.103:0/528991508 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c017070 con 0x7f1528101920 2024-08-19T01:45:34.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152819fa20 con 0x7f1528101920 2024-08-19T01:45:34.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f15267fc640 1 -- 172.21.15.103:0/528991508 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f151c009c30 con 0x7f1528101920 2024-08-19T01:45:34.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f15267fc640 1 -- 172.21.15.103:0/528991508 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c00f620 con 0x7f1528101920 2024-08-19T01:45:34.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.536+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152819ffc0 con 0x7f1528101920 2024-08-19T01:45:34.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.537+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1528107c50 con 0x7f1528101920 2024-08-19T01:45:34.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.538+0000 7f15267fc640 1 -- 172.21.15.103:0/528991508 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f151c0109a0 con 0x7f1528101920 2024-08-19T01:45:34.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.539+0000 7f15267fc640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15100a2b90 0x7f15100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.539+0000 7f152d4e9640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15100a2b90 0x7f15100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.539+0000 7f15267fc640 1 -- 172.21.15.103:0/528991508 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f151c0ced10 con 0x7f1528101920 2024-08-19T01:45:34.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.539+0000 7f152d4e9640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15100a2b90 0x7f15100a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1518006010 tx=0x7f1518000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:34.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.542+0000 7f15267fc640 1 -- 172.21.15.103:0/528991508 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f151c009de0 con 0x7f1528101920 2024-08-19T01:45:34.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.708+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f152802d5e0 con 0x7f1528101920 2024-08-19T01:45:34.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.708+0000 7f15267fc640 1 -- 172.21.15.103:0/528991508 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f151c09bca0 con 0x7f1528101920 2024-08-19T01:45:34.709 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-08-19T01:45:34.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15100a2b90 msgr2=0x7f15100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15100a2b90 0x7f15100a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1518006010 tx=0x7f1518000f40 comp rx=0 tx=0).stop 2024-08-19T01:45:34.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528101920 msgr2=0x7f152819c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528101920 0x7f152819c820 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f151c00d950 tx=0x7f151c00de10 comp rx=0 tx=0).stop 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 shutdown_connections 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15100a2b90 0x7f15100a5050 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f15281022e0 0x7f152819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1528101920 0x7f152819c820 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 --2- 172.21.15.103:0/528991508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1528100730 0x7f1528069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 >> 172.21.15.103:0/528991508 conn(0x7f15280fbe70 msgr2=0x7f1528104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 shutdown_connections 2024-08-19T01:45:34.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.710+0000 7f152f74d640 1 -- 172.21.15.103:0/528991508 wait complete. 2024-08-19T01:45:34.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-19T01:45:34.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1404150751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c102310 msgr2=0x7fb39c108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1404150751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c102310 0x7fb39c108ba0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb394009ca0 tx=0x7fb39402a8c0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1404150751 shutdown_connections 2024-08-19T01:45:34.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1404150751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c102310 0x7fb39c108ba0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1404150751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb39c101950 0x7fb39c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1404150751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb39c100760 0x7fb39c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1404150751 >> 172.21.15.103:0/1404150751 conn(0x7fb39c0fbee0 msgr2=0x7fb39c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1404150751 shutdown_connections 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.820+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1404150751 wait complete. 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 Processor -- start 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 -- start start 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c100760 0x7fb39c06aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb39c101950 0x7fb39c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb39c102310 0x7fb39c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10c270 con 0x7fb39c100760 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10c0f0 con 0x7fb39c101950 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a30fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c10c3f0 con 0x7fb39c102310 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c100760 0x7fb39c06aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c100760 0x7fb39c06aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59414/0 (socket says 172.21.15.103:59414) 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 -- 172.21.15.103:0/1035963782 learned_addr learned my addr 172.21.15.103:0/1035963782 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb39bfff640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb39c101950 0x7fb39c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a1699640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb39c102310 0x7fb39c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 -- 172.21.15.103:0/1035963782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb39c102310 msgr2=0x7fb39c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb39c102310 0x7fb39c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 -- 172.21.15.103:0/1035963782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb39c101950 msgr2=0x7fb39c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:34.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb39c101950 0x7fb39c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:34.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 -- 172.21.15.103:0/1035963782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c19f6d0 con 0x7fb39c100760 2024-08-19T01:45:34.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.821+0000 7fb3a0e98640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c100760 0x7fb39c06aca0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c970 tx=0x7fb39000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:34.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.822+0000 7fb399ffb640 1 -- 172.21.15.103:0/1035963782 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390004220 con 0x7fb39c100760 2024-08-19T01:45:34.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.822+0000 7fb399ffb640 1 -- 172.21.15.103:0/1035963782 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3900043c0 con 0x7fb39c100760 2024-08-19T01:45:34.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.822+0000 7fb399ffb640 1 -- 172.21.15.103:0/1035963782 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39000f450 con 0x7fb39c100760 2024-08-19T01:45:34.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.822+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c19f960 con 0x7fb39c100760 2024-08-19T01:45:34.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.822+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c19feb0 con 0x7fb39c100760 2024-08-19T01:45:34.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.823+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c107ba0 con 0x7fb39c100760 2024-08-19T01:45:34.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.824+0000 7fb399ffb640 1 -- 172.21.15.103:0/1035963782 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb3900079b0 con 0x7fb39c100760 2024-08-19T01:45:34.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.825+0000 7fb399ffb640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb38c0a2b90 0x7fb38c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:45:34.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.825+0000 7fb39bfff640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb38c0a2b90 0x7fb38c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:45:34.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.825+0000 7fb399ffb640 1 -- 172.21.15.103:0/1035963782 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fb3900cee80 con 0x7fb39c100760 2024-08-19T01:45:34.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.826+0000 7fb39bfff640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb38c0a2b90 0x7fb38c0a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb388005fd0 tx=0x7fb388005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:45:34.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:34.829+0000 7fb399ffb640 1 -- 172.21.15.103:0/1035963782 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb390014030 con 0x7fb39c100760 2024-08-19T01:45:35.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.041+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb39c062bc0 con 0x7fb39c100760 2024-08-19T01:45:35.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.048+0000 7fb399ffb640 1 -- 172.21.15.103:0/1035963782 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740378 (secure 0 0 0) 0x7fb39009be10 con 0x7fb39c100760 2024-08-19T01:45:35.058 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:45:35.058 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 18, 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: "active_gid": 14148, 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: "active_name": "x", 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addrs": { 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6800", 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2656970648 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6801", 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2656970648 2024-08-19T01:45:35.060 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addr": "172.21.15.103:6801/2656970648", 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: "active_change": "2024-08-19T01:27:00.697997+0000", 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: "active_mgr_features": 4540701547738038271, 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: "available": true, 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: "standbys": [ 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.061 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24112, 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "z", 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.062 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.064 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.065 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.066 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.068 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.070 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.071 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.072 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.073 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.074 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.078 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-08-19T01:45:35.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.083 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-08-19T01:45:35.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-08-19T01:45:35.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-08-19T01:45:35.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-08-19T01:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-08-19T01:45:35.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-08-19T01:45:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.097 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:45:35.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.099 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.100 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.101 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.103 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.104 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.104 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.104 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.105 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.106 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-08-19T01:45:35.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:35.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.116 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.117 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.118 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-08-19T01:45:35.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.122 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-19T01:45:35.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-08-19T01:45:35.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-19T01:45:35.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-19T01:45:35.130 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.130 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.130 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.131 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-19T01:45:35.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.132 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-08-19T01:45:35.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-08-19T01:45:35.135 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.136 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-19T01:45:35.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.138 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-08-19T01:45:35.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.141 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.141 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.141 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-19T01:45:35.141 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.142 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-08-19T01:45:35.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-08-19T01:45:35.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-19T01:45:35.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.144 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-08-19T01:45:35.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-08-19T01:45:35.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.144 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-19T01:45:35.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-08-19T01:45:35.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.146 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-08-19T01:45:35.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-08-19T01:45:35.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-19T01:45:35.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.147 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-08-19T01:45:35.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.148 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-08-19T01:45:35.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-08-19T01:45:35.148 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.150 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-08-19T01:45:35.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-08-19T01:45:35.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-19T01:45:35.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.151 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-08-19T01:45:35.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.152 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-08-19T01:45:35.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-08-19T01:45:35.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-19T01:45:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-08-19T01:45:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.154 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-08-19T01:45:35.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-08-19T01:45:35.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-19T01:45:35.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-08-19T01:45:35.155 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.156 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-08-19T01:45:35.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-08-19T01:45:35.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.157 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-19T01:45:35.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-08-19T01:45:35.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.158 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-08-19T01:45:35.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-08-19T01:45:35.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.160 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-08-19T01:45:35.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-08-19T01:45:35.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-19T01:45:35.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-08-19T01:45:35.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.162 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-08-19T01:45:35.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-08-19T01:45:35.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.163 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-19T01:45:35.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.166 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-08-19T01:45:35.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-08-19T01:45:35.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.166 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-19T01:45:35.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.167 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.167 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-08-19T01:45:35.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-08-19T01:45:35.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-19T01:45:35.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-08-19T01:45:35.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.171 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-08-19T01:45:35.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-08-19T01:45:35.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.173 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-08-19T01:45:35.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-08-19T01:45:35.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-08-19T01:45:35.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:35.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.176 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.178 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.178 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-08-19T01:45:35.178 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.178 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-08-19T01:45:35.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.180 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.180 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-19T01:45:35.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.181 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-08-19T01:45:35.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-08-19T01:45:35.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-08-19T01:45:35.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-19T01:45:35.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:35.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-08-19T01:45:35.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-08-19T01:45:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-08-19T01:45:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-19T01:45:35.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-19T01:45:35.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.189 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.189 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-08-19T01:45:35.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-08-19T01:45:35.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.190 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:35.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-08-19T01:45:35.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.191 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-19T01:45:35.192 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.192 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-19T01:45:35.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.196 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.196 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.196 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.196 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.196 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.196 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.197 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-08-19T01:45:35.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-08-19T01:45:35.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.198 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.198 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.199 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.200 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.200 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.200 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.200 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-19T01:45:35.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.201 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.203 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.203 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.203 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.203 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.203 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.203 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.204 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.204 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-19T01:45:35.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-19T01:45:35.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-08-19T01:45:35.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-19T01:45:35.210 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-08-19T01:45:35.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-08-19T01:45:35.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.211 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.211 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-08-19T01:45:35.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.212 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-08-19T01:45:35.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-08-19T01:45:35.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-19T01:45:35.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.214 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-08-19T01:45:35.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-08-19T01:45:35.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.216 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-08-19T01:45:35.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-08-19T01:45:35.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-08-19T01:45:35.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.217 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.217 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-08-19T01:45:35.217 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-08-19T01:45:35.217 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-08-19T01:45:35.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.218 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-08-19T01:45:35.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-08-19T01:45:35.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-08-19T01:45:35.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.220 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-19T01:45:35.220 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.220 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.220 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-08-19T01:45:35.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-08-19T01:45:35.221 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.221 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.221 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.221 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-19T01:45:35.221 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.221 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-19T01:45:35.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.222 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-08-19T01:45:35.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-08-19T01:45:35.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-19T01:45:35.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-19T01:45:35.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.226 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-08-19T01:45:35.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-19T01:45:35.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.230 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-08-19T01:45:35.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-08-19T01:45:35.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.231 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-19T01:45:35.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.233 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-08-19T01:45:35.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-08-19T01:45:35.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.234 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-08-19T01:45:35.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-08-19T01:45:35.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.234 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-19T01:45:35.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.237 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.238 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-08-19T01:45:35.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-08-19T01:45:35.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-19T01:45:35.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-08-19T01:45:35.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.242 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-08-19T01:45:35.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-08-19T01:45:35.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-19T01:45:35.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.244 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-08-19T01:45:35.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-08-19T01:45:35.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.246 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.246 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-08-19T01:45:35.246 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.246 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.247 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.247 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.248 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.248 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.248 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.250 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.254 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.254 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.254 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.255 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.256 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.256 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.258 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-08-19T01:45:35.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-08-19T01:45:35.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-08-19T01:45:35.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-19T01:45:35.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.259 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-08-19T01:45:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-08-19T01:45:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-08-19T01:45:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.261 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.262 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-08-19T01:45:35.262 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.262 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.262 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.262 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-19T01:45:35.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-19T01:45:35.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-08-19T01:45:35.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-19T01:45:35.265 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.265 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.265 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.266 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-19T01:45:35.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-19T01:45:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.268 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-08-19T01:45:35.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.270 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.270 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.270 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.270 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-19T01:45:35.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-19T01:45:35.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.271 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.271 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.272 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-19T01:45:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-19T01:45:35.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-19T01:45:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-19T01:45:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.278 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-19T01:45:35.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-19T01:45:35.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.278 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.282 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-19T01:45:35.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-19T01:45:35.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.286 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-19T01:45:35.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-19T01:45:35.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.286 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.286 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-19T01:45:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-19T01:45:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-08-19T01:45:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-19T01:45:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:35.290 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.293 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-08-19T01:45:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-08-19T01:45:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-08-19T01:45:35.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-19T01:45:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-19T01:45:35.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.299 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-19T01:45:35.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-19T01:45:35.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.301 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.301 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.301 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.301 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-19T01:45:35.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-19T01:45:35.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.303 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-19T01:45:35.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.305 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.305 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.305 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.305 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-08-19T01:45:35.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-19T01:45:35.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.307 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-19T01:45:35.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-19T01:45:35.307 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.309 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-19T01:45:35.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-19T01:45:35.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.309 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.310 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.310 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.311 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-19T01:45:35.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-19T01:45:35.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.317 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-19T01:45:35.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-19T01:45:35.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.321 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.323 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-19T01:45:35.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.326 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.327 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-19T01:45:35.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-19T01:45:35.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.328 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-19T01:45:35.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-19T01:45:35.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.329 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.329 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.329 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-08-19T01:45:35.329 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.329 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.330 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.330 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.330 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.330 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.330 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.332 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.332 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.332 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.332 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-08-19T01:45:35.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.334 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-19T01:45:35.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-19T01:45:35.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.335 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.335 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:35.335 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.335 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.335 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.337 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-08-19T01:45:35.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-19T01:45:35.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.339 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.339 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.339 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.339 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.340 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.341 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.341 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.341 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.341 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-19T01:45:35.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-19T01:45:35.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.342 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.342 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.342 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.342 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.343 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-19T01:45:35.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-19T01:45:35.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.344 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.344 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-19T01:45:35.345 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:35.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.347 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-08-19T01:45:35.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-08-19T01:45:35.347 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.349 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-08-19T01:45:35.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-08-19T01:45:35.349 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.349 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.351 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-08-19T01:45:35.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-08-19T01:45:35.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-08-19T01:45:35.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.353 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-08-19T01:45:35.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-08-19T01:45:35.353 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.353 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-08-19T01:45:35.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-08-19T01:45:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-08-19T01:45:35.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.357 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.358 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.358 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.360 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.362 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.362 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.362 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.362 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.363 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.363 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.363 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.363 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.364 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.364 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.364 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.364 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.367 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-08-19T01:45:35.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-08-19T01:45:35.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-08-19T01:45:35.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.368 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.369 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-08-19T01:45:35.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-08-19T01:45:35.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.370 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.370 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.371 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:35.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:35.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.371 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-08-19T01:45:35.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.372 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.373 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:35.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:35.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-08-19T01:45:35.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.375 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-08-19T01:45:35.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-08-19T01:45:35.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.378 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.378 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.378 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.378 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.380 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.380 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-08-19T01:45:35.380 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.380 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.382 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-08-19T01:45:35.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-08-19T01:45:35.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-08-19T01:45:35.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.383 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-08-19T01:45:35.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-08-19T01:45:35.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.384 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.385 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.385 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.385 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-08-19T01:45:35.386 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.386 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.386 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-08-19T01:45:35.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-08-19T01:45:35.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.389 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.389 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.390 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.390 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.390 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.390 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.390 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.394 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.395 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.395 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.395 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.395 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.395 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-08-19T01:45:35.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-08-19T01:45:35.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.398 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-08-19T01:45:35.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-19T01:45:35.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.400 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-08-19T01:45:35.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-08-19T01:45:35.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-08-19T01:45:35.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-19T01:45:35.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.402 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-08-19T01:45:35.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-08-19T01:45:35.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-08-19T01:45:35.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-08-19T01:45:35.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.404 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-08-19T01:45:35.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-08-19T01:45:35.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.404 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-08-19T01:45:35.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-19T01:45:35.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.406 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-08-19T01:45:35.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-08-19T01:45:35.406 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.407 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:35.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:35.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:35.408 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-08-19T01:45:35.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-08-19T01:45:35.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.410 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-08-19T01:45:35.410 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.410 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-19T01:45:35.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.411 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.411 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.412 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-08-19T01:45:35.412 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.412 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.412 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.412 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.413 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.414 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.414 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.414 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.414 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.414 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.414 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.415 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.415 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.415 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.415 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.416 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.417 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.417 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.418 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.418 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.420 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.422 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.422 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.423 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-08-19T01:45:35.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-08-19T01:45:35.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.423 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-08-19T01:45:35.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.424 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.424 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.424 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.424 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.424 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.425 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-08-19T01:45:35.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-08-19T01:45:35.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.425 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-08-19T01:45:35.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.427 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:35.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:35.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.427 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:35.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-08-19T01:45:35.429 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-08-19T01:45:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-08-19T01:45:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-08-19T01:45:35.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-19T01:45:35.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.432 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-08-19T01:45:35.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-08-19T01:45:35.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-08-19T01:45:35.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.435 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-08-19T01:45:35.435 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.435 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.435 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.435 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:35.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-08-19T01:45:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-08-19T01:45:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.436 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.443 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.444 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.444 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.446 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-08-19T01:45:35.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-08-19T01:45:35.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.448 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-08-19T01:45:35.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.450 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-08-19T01:45:35.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-08-19T01:45:35.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-08-19T01:45:35.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:45:35.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-08-19T01:45:35.453 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-19T01:45:35.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.454 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.454 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.454 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-08-19T01:45:35.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-08-19T01:45:35.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.456 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-08-19T01:45:35.456 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.456 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.457 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:45:35.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.458 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.458 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.460 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.461 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.461 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.461 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.461 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.461 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.462 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.462 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.462 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.462 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.462 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.464 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.465 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.466 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.466 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.467 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.468 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.472 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.472 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.472 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.472 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.472 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.473 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.475 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.476 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.476 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.476 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.476 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.476 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.476 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.477 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.477 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.478 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.478 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-08-19T01:45:35.479 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.480 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.480 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.480 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-08-19T01:45:35.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-08-19T01:45:35.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.480 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-08-19T01:45:35.481 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-19T01:45:35.481 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.482 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-08-19T01:45:35.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-08-19T01:45:35.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-08-19T01:45:35.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-19T01:45:35.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.484 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.485 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.485 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.485 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.486 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.486 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.486 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.486 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.487 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.487 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.488 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.488 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.488 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.489 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.489 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.490 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.490 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.490 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.490 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.491 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.491 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.491 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.491 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.491 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.492 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.493 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.493 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.494 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.494 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.494 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.494 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-08-19T01:45:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-08-19T01:45:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-08-19T01:45:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-19T01:45:35.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.497 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.498 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.499 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.499 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.499 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.501 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.502 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.502 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.502 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.503 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.504 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.504 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.504 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.504 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.505 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-08-19T01:45:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-08-19T01:45:35.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.507 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.507 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-08-19T01:45:35.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-08-19T01:45:35.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.509 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.509 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.509 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-08-19T01:45:35.509 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.509 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.509 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.510 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-08-19T01:45:35.510 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.510 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-08-19T01:45:35.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-08-19T01:45:35.511 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.511 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.511 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.511 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-08-19T01:45:35.511 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.511 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.511 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-08-19T01:45:35.512 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.513 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.513 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.513 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.513 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.513 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.513 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-08-19T01:45:35.514 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.514 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.514 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.514 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-08-19T01:45:35.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-08-19T01:45:35.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-19T01:45:35.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-19T01:45:35.516 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.516 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.516 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.516 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.516 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.517 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.518 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.518 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.518 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.518 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.518 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.518 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.519 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.520 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.520 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.520 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.520 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.520 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.521 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.521 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.521 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.521 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.521 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.522 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.522 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.522 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.522 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.522 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.523 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.524 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.525 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.525 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.525 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.525 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.526 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.526 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.526 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.526 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.527 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.528 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.528 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.528 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.528 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.528 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.528 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.529 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.530 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.530 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.530 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.530 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.530 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.530 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.531 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.532 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.534 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.534 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.534 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.534 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.534 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.534 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.534 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.536 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.536 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.536 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.536 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.536 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.538 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.538 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.538 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.538 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.539 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.539 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.539 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.539 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.539 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-08-19T01:45:35.540 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.540 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.540 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.540 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.540 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.540 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.541 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.541 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.541 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.541 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.541 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.541 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.541 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.542 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.542 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.542 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.542 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.542 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.542 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.542 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.543 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.544 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.544 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.544 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.544 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.544 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.544 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.544 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.546 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.546 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.546 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.546 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.546 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.546 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.547 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.547 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.547 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.547 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.547 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.547 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.548 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.548 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.548 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.548 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.549 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.549 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.549 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.549 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.549 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.549 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.549 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.550 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.550 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.550 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.550 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-08-19T01:45:35.550 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.551 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.551 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.551 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-08-19T01:45:35.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-08-19T01:45:35.551 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.551 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.551 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.552 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.552 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.552 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.552 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-19T01:45:35.552 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.552 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.553 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.554 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.554 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.554 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.554 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.554 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.554 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.554 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.555 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.556 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.557 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.557 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.557 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.557 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.557 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.558 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.558 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.558 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.558 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.558 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.558 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.559 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.560 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.561 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.562 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.562 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-08-19T01:45:35.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-08-19T01:45:35.562 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.562 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.562 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-08-19T01:45:35.563 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.564 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.565 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.566 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.567 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.567 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.567 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.567 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.567 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.567 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.567 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.568 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.568 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.568 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.568 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.569 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.570 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.570 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.570 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.570 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.570 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.571 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.572 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.572 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.572 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.572 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.572 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.572 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.573 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.573 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.573 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.573 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.573 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.573 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.574 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.574 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.574 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.574 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.575 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.576 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.576 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.576 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.576 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.576 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.577 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.577 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.577 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.577 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.577 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.577 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.577 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.578 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.578 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.578 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.578 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.578 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.578 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.579 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.580 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.580 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.580 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.580 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.580 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.582 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.582 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.582 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.582 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.582 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.582 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.582 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.583 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.583 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.583 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.583 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.583 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.583 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.584 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.585 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.586 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.587 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.589 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.590 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.591 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.592 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.592 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.592 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.592 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.592 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.592 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.592 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.593 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.595 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:35.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:35.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.596 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.597 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.598 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.599 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.600 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-19T01:45:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-08-19T01:45:35.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-08-19T01:45:35.602 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.602 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.602 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.602 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.602 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.602 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.602 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.603 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.604 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.605 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.606 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.608 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.609 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.610 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.611 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.611 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.611 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.612 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.612 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.612 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.614 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-08-19T01:45:35.614 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.614 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:35.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:35.615 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.615 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.615 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.615 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:35.615 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.615 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.616 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-08-19T01:45:35.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.616 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.616 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.616 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-08-19T01:45:35.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.619 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-08-19T01:45:35.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-08-19T01:45:35.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-19T01:45:35.620 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-19T01:45:35.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.621 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.621 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.623 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.627 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.628 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-08-19T01:45:35.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.631 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.632 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.632 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.632 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-19T01:45:35.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-19T01:45:35.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.634 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-08-19T01:45:35.635 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.635 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-08-19T01:45:35.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.637 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:35.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:35.638 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.638 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.638 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.638 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-08-19T01:45:35.638 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-08-19T01:45:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.641 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-08-19T01:45:35.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-08-19T01:45:35.642 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-08-19T01:45:35.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-08-19T01:45:35.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-08-19T01:45:35.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-08-19T01:45:35.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-08-19T01:45:35.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.646 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-08-19T01:45:35.646 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.646 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.646 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.646 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.647 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.647 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.647 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.647 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.647 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.648 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.648 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.648 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.648 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.648 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.648 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.648 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.649 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.650 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.650 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.650 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.651 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.651 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.652 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.652 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.653 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.653 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.653 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.653 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.653 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.653 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.654 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.654 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.654 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.655 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.656 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.657 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.657 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.657 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.657 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.657 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.658 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-08-19T01:45:35.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-08-19T01:45:35.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.659 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.659 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.659 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.660 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.660 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-08-19T01:45:35.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-08-19T01:45:35.660 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.660 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.661 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.661 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.661 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.661 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.662 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.662 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.662 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.662 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-08-19T01:45:35.663 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.663 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.663 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.663 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-08-19T01:45:35.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-08-19T01:45:35.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.663 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.664 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.664 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.664 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.665 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.666 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.668 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.670 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.671 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.673 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.675 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.678 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-08-19T01:45:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.682 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.683 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.684 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.685 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.686 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.687 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.688 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-08-19T01:45:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.692 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.693 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.694 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.695 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.698 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.700 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.701 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-08-19T01:45:35.702 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-08-19T01:45:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-08-19T01:45:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.703 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-19T01:45:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.705 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-08-19T01:45:35.705 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.705 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.705 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.705 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.706 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.706 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.706 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.706 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.706 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.706 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.706 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.707 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.710 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.712 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.713 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.714 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.716 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-08-19T01:45:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-08-19T01:45:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-08-19T01:45:35.718 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-08-19T01:45:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-08-19T01:45:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.720 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.721 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-08-19T01:45:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-08-19T01:45:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-08-19T01:45:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-08-19T01:45:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.724 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.725 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-08-19T01:45:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-08-19T01:45:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.726 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.727 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-08-19T01:45:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-08-19T01:45:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.728 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.729 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-08-19T01:45:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-08-19T01:45:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.730 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.731 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-08-19T01:45:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-08-19T01:45:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.732 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-08-19T01:45:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.733 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-08-19T01:45:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-08-19T01:45:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.734 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.735 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.736 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-08-19T01:45:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-08-19T01:45:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.739 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.741 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.741 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-08-19T01:45:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-08-19T01:45:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.742 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.743 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-08-19T01:45:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.744 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-08-19T01:45:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-08-19T01:45:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-08-19T01:45:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.746 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-08-19T01:45:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-08-19T01:45:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.747 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-08-19T01:45:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.749 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.750 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.750 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.750 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.750 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.751 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.755 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.756 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.758 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.759 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-08-19T01:45:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.760 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.761 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.762 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.764 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.766 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.767 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.768 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.769 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.770 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-08-19T01:45:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.771 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.772 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.773 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.776 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.776 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.776 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.776 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.777 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.777 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.777 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.777 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.777 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.777 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.777 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.778 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.778 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.778 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.778 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.778 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.778 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.778 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.779 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.779 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.779 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.779 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.780 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.780 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.780 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.780 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.780 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.781 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.781 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.781 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.781 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.781 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.781 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-08-19T01:45:35.782 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.782 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.782 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.782 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-08-19T01:45:35.782 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-08-19T01:45:35.782 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.782 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.783 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.783 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-19T01:45:35.783 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.783 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.783 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.783 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.784 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.784 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.784 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.784 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:35.784 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:35.784 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.785 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.785 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.785 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-08-19T01:45:35.785 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.785 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.785 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.785 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.786 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.786 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.786 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.786 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.788 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.788 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.788 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.788 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.788 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.788 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.788 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.789 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.789 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.789 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.789 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.790 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.790 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.790 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.790 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.790 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.790 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.790 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.791 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.791 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.791 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.792 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.792 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.792 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.792 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.792 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.792 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.792 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.794 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.794 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.794 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.795 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.795 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.795 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.795 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.796 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.796 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.796 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.796 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.796 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.796 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.797 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.797 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-08-19T01:45:35.797 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.797 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.797 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.797 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-08-19T01:45:35.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-08-19T01:45:35.798 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.798 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.798 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.798 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.798 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.798 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.799 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.799 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-19T01:45:35.799 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.799 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.799 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.799 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-08-19T01:45:35.800 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-08-19T01:45:35.800 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.800 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.800 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.800 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.800 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.800 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.801 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.801 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-19T01:45:35.801 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.801 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.801 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.801 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-08-19T01:45:35.802 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-08-19T01:45:35.802 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.802 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.802 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.802 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.802 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.802 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.803 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.803 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-19T01:45:35.803 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.803 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.803 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.803 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.803 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-08-19T01:45:35.804 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-08-19T01:45:35.804 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.804 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.804 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.804 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.804 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.804 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.805 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.805 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-19T01:45:35.805 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.805 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.805 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.805 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-08-19T01:45:35.806 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-08-19T01:45:35.806 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.806 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.806 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.806 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.806 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.807 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-08-19T01:45:35.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-08-19T01:45:35.808 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.808 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.808 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.808 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.808 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.808 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.809 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.809 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.809 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.809 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.809 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.809 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-08-19T01:45:35.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-08-19T01:45:35.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.810 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.810 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.811 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.811 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.811 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.811 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.812 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-08-19T01:45:35.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-08-19T01:45:35.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.812 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.812 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.812 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-19T01:45:35.812 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.813 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.813 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.813 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.813 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.813 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.813 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.814 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-08-19T01:45:35.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-08-19T01:45:35.814 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.814 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.814 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.814 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.814 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.815 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.815 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.815 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.815 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.815 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-08-19T01:45:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-08-19T01:45:35.817 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.817 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.817 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.817 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.818 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.819 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.819 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.819 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.819 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.819 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-08-19T01:45:35.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-08-19T01:45:35.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.820 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-08-19T01:45:35.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-08-19T01:45:35.822 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.822 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.822 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.822 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.822 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.822 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.822 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.823 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.823 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.823 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.823 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.823 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.823 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.824 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.824 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.824 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.824 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.824 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.824 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.824 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.825 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.825 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.825 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.825 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.825 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.825 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.825 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.826 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.826 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.827 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.827 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.827 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.827 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.827 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.827 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.828 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.829 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.829 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.829 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.829 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.830 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.830 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.830 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.830 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.830 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.830 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.830 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.831 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.831 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.831 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.831 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.831 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.831 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.832 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.832 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.832 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.832 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.832 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.832 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.832 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.833 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.833 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-08-19T01:45:35.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-08-19T01:45:35.833 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.834 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.834 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.834 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.834 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.834 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.834 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.834 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.835 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.836 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.836 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.836 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.836 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.836 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.837 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.837 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.837 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-08-19T01:45:35.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-08-19T01:45:35.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.837 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-19T01:45:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.838 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-08-19T01:45:35.839 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.840 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.840 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.840 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.840 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.840 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.840 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.841 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.841 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.841 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.841 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.841 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.841 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.841 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.842 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.842 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.842 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.842 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.842 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.842 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.843 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.843 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.843 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.843 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.844 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.844 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.844 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.844 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.844 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.844 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.845 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.846 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.846 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.846 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.846 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.846 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.846 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.847 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.849 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.850 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.850 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.850 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-08-19T01:45:35.850 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.850 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.851 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.852 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.852 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.852 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.852 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.852 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.852 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.852 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.854 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.854 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.854 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.854 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.854 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.855 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.855 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.855 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.855 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.855 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.856 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.856 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.856 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.856 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.856 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.857 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.857 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.857 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.857 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.857 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.857 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.857 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.858 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.858 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.858 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.858 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.859 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.860 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.860 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.860 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.860 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.860 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.860 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.861 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.862 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-19T01:45:35.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.862 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.862 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-08-19T01:45:35.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-08-19T01:45:35.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.863 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.863 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.863 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.863 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.864 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-08-19T01:45:35.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.864 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.864 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-08-19T01:45:35.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-08-19T01:45:35.865 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.865 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.865 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.865 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.865 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.865 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.866 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.866 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-19T01:45:35.866 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.866 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.866 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.866 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-08-19T01:45:35.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-08-19T01:45:35.867 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.867 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.867 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.869 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-08-19T01:45:35.869 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.869 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.869 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.869 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-08-19T01:45:35.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-08-19T01:45:35.870 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.870 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.870 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.870 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-08-19T01:45:35.870 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.870 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.870 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.871 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.871 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.871 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.871 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.871 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-08-19T01:45:35.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-08-19T01:45:35.872 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.872 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.872 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.872 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.872 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.872 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.872 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:35.873 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.874 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.874 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.874 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:35.874 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.874 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.874 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.874 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.875 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.875 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.875 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.875 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.876 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.876 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.876 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.876 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.876 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.876 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.876 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.877 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.877 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.877 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.877 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.877 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.877 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.877 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.878 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.879 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.879 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.879 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.879 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.879 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.879 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.879 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.880 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.881 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.882 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.882 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.882 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.882 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.882 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.882 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.883 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.883 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.883 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.883 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.883 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.884 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.885 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.885 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-08-19T01:45:35.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-08-19T01:45:35.885 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.885 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.885 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.886 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.886 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.886 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.886 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.886 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.886 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.886 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.887 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.887 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-08-19T01:45:35.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-08-19T01:45:35.887 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.887 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.887 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.888 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-08-19T01:45:35.888 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.888 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.888 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.888 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.888 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.888 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.889 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.889 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-08-19T01:45:35.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-08-19T01:45:35.889 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.890 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-19T01:45:35.890 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.890 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.890 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.890 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.891 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.891 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.891 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:35.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.891 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.892 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24113, 2024-08-19T01:45:35.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "y", 2024-08-19T01:45:35.892 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-08-19T01:45:35.892 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-08-19T01:45:35.892 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-08-19T01:45:35.892 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:35.894 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.894 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.894 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.894 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-08-19T01:45:35.894 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.894 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.894 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.895 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.895 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.895 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.895 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.895 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.896 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.897 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.897 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.897 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.897 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.897 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.897 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.898 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.900 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.900 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.900 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.900 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.900 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.900 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.900 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.901 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.901 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.901 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.901 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.901 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.901 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.901 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.902 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.903 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.903 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.903 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.903 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.903 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.903 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.904 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.905 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.905 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.905 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.905 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.905 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.905 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-08-19T01:45:35.905 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.906 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.906 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.906 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-08-19T01:45:35.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-08-19T01:45:35.906 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.906 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.907 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.908 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.908 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-08-19T01:45:35.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-08-19T01:45:35.908 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.908 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.908 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.909 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.909 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.909 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.909 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.909 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-08-19T01:45:35.909 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.910 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.910 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.910 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-08-19T01:45:35.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-08-19T01:45:35.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.910 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.911 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.911 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-08-19T01:45:35.912 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.912 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.912 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.912 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-08-19T01:45:35.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-08-19T01:45:35.912 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-08-19T01:45:35.913 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.914 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.915 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.915 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-08-19T01:45:35.915 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.917 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-19T01:45:35.917 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.918 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-08-19T01:45:35.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-08-19T01:45:35.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.918 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.918 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.919 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.919 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-08-19T01:45:35.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.920 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.920 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.920 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.920 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-08-19T01:45:35.920 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.921 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.921 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.921 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-08-19T01:45:35.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-08-19T01:45:35.921 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.921 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.922 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.922 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.922 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.922 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.922 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.922 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-08-19T01:45:35.922 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.923 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.924 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-08-19T01:45:35.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.924 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-08-19T01:45:35.924 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:45:35.924 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.925 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-08-19T01:45:35.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-08-19T01:45:35.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.925 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.926 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.926 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-08-19T01:45:35.926 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-08-19T01:45:35.926 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.926 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-19T01:45:35.926 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.927 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.928 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-08-19T01:45:35.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:35.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-08-19T01:45:35.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-19T01:45:35.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.929 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.929 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-08-19T01:45:35.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-08-19T01:45:35.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.929 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-08-19T01:45:35.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.930 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.930 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.930 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-19T01:45:35.930 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-19T01:45:35.931 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.931 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.931 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-08-19T01:45:35.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-08-19T01:45:35.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-19T01:45:35.932 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.933 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.933 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-08-19T01:45:35.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-08-19T01:45:35.933 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.933 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.933 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.934 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.935 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-08-19T01:45:35.936 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-08-19T01:45:35.936 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.936 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-19T01:45:35.936 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:45:35.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.936 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.937 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.938 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.938 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.938 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.938 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.938 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.938 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.938 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:35.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.940 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.941 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.942 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:35.942 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.942 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.942 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.942 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:35.942 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:35.942 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:35.943 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:35.943 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:35.943 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:35.943 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.943 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.943 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.943 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.944 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.945 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.946 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-08-19T01:45:35.947 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-19T01:45:35.948 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.949 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-08-19T01:45:35.950 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.951 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.952 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.953 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.955 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.957 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.958 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-19T01:45:35.958 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.958 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.958 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.958 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.958 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-08-19T01:45:35.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-19T01:45:35.960 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.961 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-08-19T01:45:35.962 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.962 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.962 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.962 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-19T01:45:35.962 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.962 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.962 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-08-19T01:45:35.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.964 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.965 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-08-19T01:45:35.966 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:35.967 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.967 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.967 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:35.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.967 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.967 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.968 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.969 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:35.970 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-19T01:45:35.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.972 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.973 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:35.974 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-08-19T01:45:35.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-08-19T01:45:35.976 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:35.976 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.976 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.976 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:35.976 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.976 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.976 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.977 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.978 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-19T01:45:35.979 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.981 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-08-19T01:45:35.982 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.983 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.984 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.986 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-08-19T01:45:35.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.988 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-19T01:45:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-08-19T01:45:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.990 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-19T01:45:35.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.992 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.993 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-08-19T01:45:35.994 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.995 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:35.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-08-19T01:45:35.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.000 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.001 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.002 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-19T01:45:36.003 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-08-19T01:45:36.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.005 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.006 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.007 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.008 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.009 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.010 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-08-19T01:45:36.011 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.012 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.013 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.015 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-08-19T01:45:36.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-19T01:45:36.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.018 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.020 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.021 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.022 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.024 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.028 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.029 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.030 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.032 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-19T01:45:36.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-19T01:45:36.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.036 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-08-19T01:45:36.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.039 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.040 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.041 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-08-19T01:45:36.042 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.043 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-08-19T01:45:36.044 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.045 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.046 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.047 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-08-19T01:45:36.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-19T01:45:36.053 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.055 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-08-19T01:45:36.056 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-08-19T01:45:36.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.061 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.062 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.063 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-08-19T01:45:36.064 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-19T01:45:36.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.066 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.067 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.068 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.070 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.071 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.072 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.073 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.074 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-08-19T01:45:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-19T01:45:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.078 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-08-19T01:45:36.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-19T01:45:36.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-08-19T01:45:36.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.087 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-19T01:45:36.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-19T01:45:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.092 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.097 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-19T01:45:36.098 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.101 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.102 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-19T01:45:36.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.106 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.107 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.108 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.109 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.110 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.113 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-19T01:45:36.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-19T01:45:36.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.124 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-19T01:45:36.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-19T01:45:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-19T01:45:36.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.129 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-19T01:45:36.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.134 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.137 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-19T01:45:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-08-19T01:45:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-19T01:45:36.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-08-19T01:45:36.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.151 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-08-19T01:45:36.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.159 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-08-19T01:45:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.161 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-08-19T01:45:36.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.166 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.167 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.169 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.170 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-08-19T01:45:36.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-08-19T01:45:36.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-08-19T01:45:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-08-19T01:45:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-08-19T01:45:36.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-08-19T01:45:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-08-19T01:45:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-08-19T01:45:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-08-19T01:45:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-08-19T01:45:36.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.190 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.194 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.197 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.197 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.199 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.200 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-08-19T01:45:36.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-08-19T01:45:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-08-19T01:45:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-19T01:45:36.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.204 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-08-19T01:45:36.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-19T01:45:36.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.206 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-08-19T01:45:36.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-08-19T01:45:36.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-08-19T01:45:36.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.210 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-08-19T01:45:36.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-08-19T01:45:36.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-19T01:45:36.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:36.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-08-19T01:45:36.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-08-19T01:45:36.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-19T01:45:36.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.215 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-08-19T01:45:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.217 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.217 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.218 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.218 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.218 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.218 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.218 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.218 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.221 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.221 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.223 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.224 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.225 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.226 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-08-19T01:45:36.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.228 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-08-19T01:45:36.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-08-19T01:45:36.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.230 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:36.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:36.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.231 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:36.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.232 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.232 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.232 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-08-19T01:45:36.233 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-08-19T01:45:36.233 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-08-19T01:45:36.233 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.233 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-08-19T01:45:36.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-08-19T01:45:36.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-08-19T01:45:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-08-19T01:45:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-08-19T01:45:36.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-19T01:45:36.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.237 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.237 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-08-19T01:45:36.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-08-19T01:45:36.237 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-08-19T01:45:36.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.239 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:36.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:36.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.241 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.242 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.242 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.243 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.243 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.243 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.243 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.243 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.244 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.246 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.246 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.246 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.246 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.247 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.247 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.247 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.247 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.247 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.247 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.249 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.249 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.249 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.250 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.250 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.251 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-08-19T01:45:36.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-08-19T01:45:36.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.251 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.251 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-08-19T01:45:36.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-08-19T01:45:36.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.255 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-08-19T01:45:36.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-08-19T01:45:36.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-08-19T01:45:36.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.256 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-08-19T01:45:36.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-19T01:45:36.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.258 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-08-19T01:45:36.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-08-19T01:45:36.259 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-08-19T01:45:36.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-08-19T01:45:36.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:45:36.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.262 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.262 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.262 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-08-19T01:45:36.263 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.263 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.263 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.263 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.264 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.266 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.269 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.269 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.269 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.270 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.270 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.270 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.270 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.270 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.270 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.270 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.271 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.273 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.274 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.274 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.275 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.276 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.276 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.276 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.276 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.276 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.276 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.280 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.281 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.281 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.282 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.283 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.284 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-08-19T01:45:36.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-08-19T01:45:36.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-08-19T01:45:36.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.286 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.286 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.286 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-19T01:45:36.286 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.286 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-08-19T01:45:36.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-19T01:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.288 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.290 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.290 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.290 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.290 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.290 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.290 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.290 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.291 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.291 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.293 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.294 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.296 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.296 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.298 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.298 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.298 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-08-19T01:45:36.299 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.299 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.299 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.299 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-08-19T01:45:36.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-08-19T01:45:36.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-08-19T01:45:36.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-19T01:45:36.301 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.301 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.301 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.302 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.302 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.303 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.303 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.303 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.303 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.303 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.304 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.304 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.304 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.305 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.307 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.307 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.307 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.308 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.308 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.308 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.308 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.308 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.309 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.310 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.310 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-08-19T01:45:36.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-08-19T01:45:36.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-08-19T01:45:36.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-08-19T01:45:36.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-08-19T01:45:36.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.315 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-08-19T01:45:36.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-08-19T01:45:36.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-08-19T01:45:36.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.318 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-19T01:45:36.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.321 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.322 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.322 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.323 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.323 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.323 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.323 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.324 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.324 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.324 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.326 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.327 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.328 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.329 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.329 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.329 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.331 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.331 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.332 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-08-19T01:45:36.332 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.332 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.332 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.334 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.334 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.334 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.334 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.334 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.334 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.336 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.336 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.336 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.336 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.336 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.336 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.337 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.339 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.339 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.339 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.339 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.339 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.339 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.340 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.340 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.340 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.341 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.341 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.342 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.342 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.342 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.342 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.343 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.343 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.343 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-08-19T01:45:36.344 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.344 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.344 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.344 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.345 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.345 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.345 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.346 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.346 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.346 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.346 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.346 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.346 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.347 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.347 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.349 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.350 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.350 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.350 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.351 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.351 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.351 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.351 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.352 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.352 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.353 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.353 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.353 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.353 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.354 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.354 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.354 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.354 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-08-19T01:45:36.354 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.355 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.355 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.355 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-08-19T01:45:36.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-08-19T01:45:36.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.356 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-19T01:45:36.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.357 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.357 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.357 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.358 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.358 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.359 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.359 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.359 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.359 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.359 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.362 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.363 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.364 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.368 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-08-19T01:45:36.368 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-08-19T01:45:36.368 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-08-19T01:45:36.368 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-08-19T01:45:36.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.369 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.369 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.369 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-08-19T01:45:36.370 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.370 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.370 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.370 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.371 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.372 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.372 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.372 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.372 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.372 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.373 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.373 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.373 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.374 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.374 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.374 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.374 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.375 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.376 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.378 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.378 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.378 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.379 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.380 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.381 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.381 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.382 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.382 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.382 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.382 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.383 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.383 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.383 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.387 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.387 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.387 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.387 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.387 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.387 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.387 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.388 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.389 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.390 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.390 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.390 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.391 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-08-19T01:45:36.391 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.391 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.391 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.391 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.392 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.392 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.392 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.393 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.393 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.393 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.393 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.393 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.393 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.394 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.394 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.395 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.395 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.395 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.395 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.395 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.395 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.396 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.396 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.396 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.396 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.396 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.396 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.397 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.397 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.397 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.398 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.398 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.398 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.398 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.398 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.399 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.399 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.402 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:36.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:36.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:36.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.403 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-08-19T01:45:36.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-08-19T01:45:36.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:36.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.404 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-08-19T01:45:36.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-08-19T01:45:36.404 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-08-19T01:45:36.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-19T01:45:36.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.405 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.406 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-08-19T01:45:36.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-08-19T01:45:36.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.407 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-19T01:45:36.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.408 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-08-19T01:45:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-08-19T01:45:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.410 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.410 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.410 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.412 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.412 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.412 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.412 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.412 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.412 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.413 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.413 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.413 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.414 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.414 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.414 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.414 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.415 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.415 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.415 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.415 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.415 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.417 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.418 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.418 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.418 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.420 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.422 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-08-19T01:45:36.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.422 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-08-19T01:45:36.424 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.424 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.424 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-08-19T01:45:36.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-08-19T01:45:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-08-19T01:45:36.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-19T01:45:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-19T01:45:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.429 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.430 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.431 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.431 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.431 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.431 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.431 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.431 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.434 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.434 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.434 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.434 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.435 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.435 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.436 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.436 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.436 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.436 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.436 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.438 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.438 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.438 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.439 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-08-19T01:45:36.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-08-19T01:45:36.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.440 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.440 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.441 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-08-19T01:45:36.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.443 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.443 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.443 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-08-19T01:45:36.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-08-19T01:45:36.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:36.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.445 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:36.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:36.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.446 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-08-19T01:45:36.446 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.446 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-19T01:45:36.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-08-19T01:45:36.448 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.448 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.448 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-19T01:45:36.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-08-19T01:45:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-08-19T01:45:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-08-19T01:45:36.450 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.451 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-08-19T01:45:36.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-08-19T01:45:36.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-08-19T01:45:36.452 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.452 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-08-19T01:45:36.452 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-08-19T01:45:36.452 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-08-19T01:45:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-08-19T01:45:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.454 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-08-19T01:45:36.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-08-19T01:45:36.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-08-19T01:45:36.454 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.455 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.455 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.455 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.455 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.455 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.457 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.457 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.457 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.457 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.457 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.457 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.459 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.459 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.459 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.461 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.461 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.461 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.462 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.462 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.463 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.463 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.463 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.463 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.463 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.463 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.464 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.464 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.465 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.465 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.468 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.469 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.469 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.469 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.469 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.469 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.471 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.471 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.472 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-08-19T01:45:36.472 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.472 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.472 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.474 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.476 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.476 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.476 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.476 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.477 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.477 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.478 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.478 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.478 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.478 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.478 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.478 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.479 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.479 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.479 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.480 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.480 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.480 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.481 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.481 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.483 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.483 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.483 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.483 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.483 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.483 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.484 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.484 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.486 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.486 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:36.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:36.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.488 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:36.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:36.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.490 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.490 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-08-19T01:45:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.493 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.494 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.494 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.494 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.494 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.494 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.495 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.496 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.496 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.498 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.498 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.498 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.498 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.498 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.499 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.499 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.500 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.500 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.500 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.500 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.501 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.502 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-08-19T01:45:36.502 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.502 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.502 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.502 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.504 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.504 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.504 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.504 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.504 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.504 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.504 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.505 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.505 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.505 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.505 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.506 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.506 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.506 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.506 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.506 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.506 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.507 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.507 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.507 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.507 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.509 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.509 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.509 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.509 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.509 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.509 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.509 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.510 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.510 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.510 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.511 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.511 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.511 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.511 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.511 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.512 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.512 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.512 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.512 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.512 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.512 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.513 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-08-19T01:45:36.514 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.514 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.514 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.514 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-19T01:45:36.514 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.514 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.514 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.515 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.516 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.516 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.516 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.516 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.516 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.516 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.517 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.518 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.518 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.518 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.518 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.518 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.518 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.519 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.520 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.520 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.520 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.520 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.520 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.521 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.521 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.521 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.521 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.521 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.521 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.522 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.523 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.523 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.523 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.523 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.523 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.524 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-08-19T01:45:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-08-19T01:45:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.526 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.526 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.526 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.526 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.527 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.527 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-08-19T01:45:36.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-08-19T01:45:36.527 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.527 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.528 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.528 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-08-19T01:45:36.528 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.528 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.528 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.528 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.528 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.529 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.530 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.530 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.530 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.530 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.530 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.530 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.530 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.531 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.532 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.533 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-08-19T01:45:36.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-08-19T01:45:36.533 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:36.533 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.533 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.534 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.535 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-08-19T01:45:36.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-08-19T01:45:36.535 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.535 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.535 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.535 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.536 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.536 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.536 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.536 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.536 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.536 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.536 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.537 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-08-19T01:45:36.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-08-19T01:45:36.537 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.537 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.537 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.537 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-08-19T01:45:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-08-19T01:45:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.539 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.540 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.540 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.540 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.540 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.540 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.540 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:36.541 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-08-19T01:45:36.542 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.542 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.542 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.542 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.542 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.542 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-08-19T01:45:36.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-08-19T01:45:36.543 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.543 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.543 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.543 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.543 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.543 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.544 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.544 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.544 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.544 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.544 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.544 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-08-19T01:45:36.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-08-19T01:45:36.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.545 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.545 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-08-19T01:45:36.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.548 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.548 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.548 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.548 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-08-19T01:45:36.549 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.549 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.549 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.549 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-08-19T01:45:36.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-08-19T01:45:36.549 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.550 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.550 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.550 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.550 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.550 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.550 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.552 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.552 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.552 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.552 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.552 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-08-19T01:45:36.552 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.553 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.554 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.554 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.554 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.554 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.554 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.554 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.554 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.555 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.555 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.557 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.557 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.557 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.557 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.558 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.558 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.558 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.558 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.558 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.559 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.560 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.561 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.562 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.562 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.562 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.562 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.562 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.562 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.564 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-08-19T01:45:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.565 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.566 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.566 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.566 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.566 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.566 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.567 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.569 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.570 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.570 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.570 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.570 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.570 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.571 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.572 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.572 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.572 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.572 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.572 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.572 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.572 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.573 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.574 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.575 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-08-19T01:45:36.575 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.575 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.576 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.577 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.577 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.577 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.577 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.577 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.577 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.577 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.578 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.578 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.578 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.578 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.578 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.578 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.579 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.579 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.579 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.579 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.579 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.579 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.581 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.581 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.581 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.582 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.582 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.582 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.582 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.582 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.582 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.583 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.583 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.583 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.583 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.583 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.583 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.584 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.584 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.584 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.584 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.585 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-08-19T01:45:36.586 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.586 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.586 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.586 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-08-19T01:45:36.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-08-19T01:45:36.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-19T01:45:36.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.588 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.589 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:36.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:36.589 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.589 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.589 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.589 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-08-19T01:45:36.589 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.590 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.590 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.590 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.590 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.591 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.592 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.592 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.592 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.592 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.592 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.592 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.592 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.593 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.595 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.597 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.597 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.597 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.597 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.597 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.597 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.597 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.598 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.598 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.598 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.599 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.600 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-08-19T01:45:36.601 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.601 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.601 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.601 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-08-19T01:45:36.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-08-19T01:45:36.601 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.602 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.602 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.602 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.602 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.602 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.602 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.602 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.603 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.604 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-19T01:45:36.604 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.604 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.605 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.606 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.606 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-19T01:45:36.606 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.607 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-08-19T01:45:36.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-08-19T01:45:36.607 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.607 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.607 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.607 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.608 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.608 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-19T01:45:36.608 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.610 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-19T01:45:36.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.610 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-08-19T01:45:36.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-08-19T01:45:36.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.611 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.611 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-08-19T01:45:36.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.614 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.614 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-08-19T01:45:36.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-08-19T01:45:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-08-19T01:45:36.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.617 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.618 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.618 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.618 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:36.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:36.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.620 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-08-19T01:45:36.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-08-19T01:45:36.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.622 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-08-19T01:45:36.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-08-19T01:45:36.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.624 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-08-19T01:45:36.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-08-19T01:45:36.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.624 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.624 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.624 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.625 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.626 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.626 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.626 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.626 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.627 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.627 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.627 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.627 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.627 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.627 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.629 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.629 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.629 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.629 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.629 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.630 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.631 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.632 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.633 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.635 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-08-19T01:45:36.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-08-19T01:45:36.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.635 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.635 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-08-19T01:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-08-19T01:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.638 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.638 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.638 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.638 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.638 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.639 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-08-19T01:45:36.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-08-19T01:45:36.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.639 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.639 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-19T01:45:36.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.641 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.641 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.641 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-08-19T01:45:36.641 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.642 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.642 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.642 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.643 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.644 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.644 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.644 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.644 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.644 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.645 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.645 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.645 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.645 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.645 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.646 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.646 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.646 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.646 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.646 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.646 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.646 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.647 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.647 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.647 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.648 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.648 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.648 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.648 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.648 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.648 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.650 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.651 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.651 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.651 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.651 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.651 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.652 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.653 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.654 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.654 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.654 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.654 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.654 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.654 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.655 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.655 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.655 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.655 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.656 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.657 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.658 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.660 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.661 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-08-19T01:45:36.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-08-19T01:45:36.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.662 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-08-19T01:45:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-08-19T01:45:36.666 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.666 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.666 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:36.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-08-19T01:45:36.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.668 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.670 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-08-19T01:45:36.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.671 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.671 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.671 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.671 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.672 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-08-19T01:45:36.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-08-19T01:45:36.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.673 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.673 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.673 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.673 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.673 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.673 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.674 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.675 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.676 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.676 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.676 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.676 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.677 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.678 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.679 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.680 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.681 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.681 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.681 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.681 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.681 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.681 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.681 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.682 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.682 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.682 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.682 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.682 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.682 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.682 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.683 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.684 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.684 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.684 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.684 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.684 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.684 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.684 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.686 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.686 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.686 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.686 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.686 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.686 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.686 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.687 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.687 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-08-19T01:45:36.687 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-08-19T01:45:36.687 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.687 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.688 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.689 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-08-19T01:45:36.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-08-19T01:45:36.689 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.689 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.689 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.689 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-19T01:45:36.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.690 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.690 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.690 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.690 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.690 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.691 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:36.691 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.691 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.691 INFO:teuthology.orchestra.run.smithi103.stdout: "modules": [ 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard", 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: "iostat", 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: "nfs", 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: "restful" 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-08-19T01:45:36.692 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-08-19T01:45:36.693 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.693 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.693 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.693 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:36.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:36.693 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.694 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.694 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.694 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:36.694 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.694 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.694 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.694 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-08-19T01:45:36.695 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.695 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.695 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.695 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.695 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.695 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.696 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.697 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.697 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.697 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.697 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.697 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.697 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.697 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.698 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.699 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.699 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.699 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.699 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.699 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.699 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.700 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.701 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.701 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.701 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.701 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.701 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.701 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.702 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.702 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.702 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.702 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.703 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.703 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.703 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.704 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.704 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.704 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.704 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.704 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.704 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.705 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.706 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.706 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.706 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-08-19T01:45:36.706 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.706 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.706 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.707 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.708 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.708 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-08-19T01:45:36.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.708 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-08-19T01:45:36.709 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-08-19T01:45:36.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.709 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.709 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.709 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.710 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.710 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-08-19T01:45:36.710 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.710 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.710 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.710 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.711 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.712 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-08-19T01:45:36.712 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.712 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.712 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.712 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-08-19T01:45:36.712 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-08-19T01:45:36.713 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.713 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.713 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.713 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-08-19T01:45:36.713 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.713 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.713 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.714 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-08-19T01:45:36.714 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.714 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.714 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.714 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.715 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-08-19T01:45:36.716 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.717 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.717 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.717 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.717 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.717 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-19T01:45:36.718 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.718 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.718 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.718 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-08-19T01:45:36.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-08-19T01:45:36.718 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-08-19T01:45:36.719 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-08-19T01:45:36.720 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.721 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.721 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.721 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-08-19T01:45:36.721 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-08-19T01:45:36.721 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.721 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.721 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.722 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.722 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.722 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.722 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-08-19T01:45:36.722 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.722 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-08-19T01:45:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-08-19T01:45:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:45:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.724 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-08-19T01:45:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-08-19T01:45:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-08-19T01:45:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-08-19T01:45:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-08-19T01:45:36.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.728 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-08-19T01:45:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-08-19T01:45:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-08-19T01:45:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-19T01:45:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-19T01:45:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.730 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-08-19T01:45:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-08-19T01:45:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-08-19T01:45:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-08-19T01:45:36.731 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-08-19T01:45:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-08-19T01:45:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-19T01:45:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.732 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-08-19T01:45:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-08-19T01:45:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-08-19T01:45:36.733 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-08-19T01:45:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-19T01:45:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-08-19T01:45:36.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-08-19T01:45:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-19T01:45:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-19T01:45:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.736 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.737 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.738 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.740 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.741 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.742 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.744 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.744 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.744 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.744 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.744 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.745 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-08-19T01:45:36.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-08-19T01:45:36.746 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:36.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.746 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.746 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:36.747 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.747 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.747 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.747 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-19T01:45:36.747 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.747 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-08-19T01:45:36.748 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-08-19T01:45:36.749 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.750 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-08-19T01:45:36.750 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.750 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.750 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.750 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-08-19T01:45:36.750 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.751 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.752 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:36.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.753 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.753 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-19T01:45:36.753 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.754 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.754 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.754 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-08-19T01:45:36.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-08-19T01:45:36.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.754 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-19T01:45:36.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.756 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.756 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-08-19T01:45:36.756 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-08-19T01:45:36.756 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:36.756 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.756 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.757 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.758 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.758 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.758 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.758 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-08-19T01:45:36.758 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.758 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-08-19T01:45:36.759 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.760 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.760 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.760 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-19T01:45:36.760 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.760 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.760 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.761 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-08-19T01:45:36.761 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-08-19T01:45:36.761 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.761 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.761 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-08-19T01:45:36.761 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.762 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.762 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.762 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-19T01:45:36.762 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.762 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.762 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.763 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-08-19T01:45:36.764 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.765 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-19T01:45:36.766 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-19T01:45:36.766 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.766 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.766 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-08-19T01:45:36.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-08-19T01:45:36.766 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-19T01:45:36.767 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.768 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.768 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.769 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-08-19T01:45:36.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-08-19T01:45:36.770 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:36.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.770 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.770 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-08-19T01:45:36.770 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.770 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-08-19T01:45:36.771 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-08-19T01:45:36.772 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.772 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.772 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.772 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.772 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.772 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.772 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.773 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-08-19T01:45:36.773 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-08-19T01:45:36.773 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-08-19T01:45:36.773 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.773 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-19T01:45:36.773 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.773 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.774 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.775 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.775 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.775 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-19T01:45:36.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.775 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.775 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.776 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-08-19T01:45:36.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-08-19T01:45:36.776 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.776 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.776 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.776 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.778 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.779 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.779 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-19T01:45:36.779 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.780 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.781 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.781 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:36.781 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.781 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.781 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.781 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-08-19T01:45:36.782 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-08-19T01:45:36.782 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.782 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.782 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.782 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-19T01:45:36.782 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.782 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-08-19T01:45:36.783 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-08-19T01:45:36.784 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.784 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.784 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.784 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-19T01:45:36.784 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.784 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.784 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-08-19T01:45:36.785 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.786 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.786 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.786 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-19T01:45:36.786 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.786 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.786 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.786 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.788 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.789 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.789 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-08-19T01:45:36.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-08-19T01:45:36.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.789 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.789 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.790 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-19T01:45:36.790 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.790 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.790 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.790 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-08-19T01:45:36.790 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.790 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.791 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-08-19T01:45:36.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-08-19T01:45:36.791 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.791 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.791 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.792 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.793 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-08-19T01:45:36.793 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-08-19T01:45:36.793 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.793 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.793 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.793 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.795 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-08-19T01:45:36.795 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-08-19T01:45:36.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.795 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.795 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.795 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-19T01:45:36.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.796 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.797 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.798 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-08-19T01:45:36.798 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.798 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.798 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.798 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-08-19T01:45:36.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.799 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:36.800 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.800 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.800 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.800 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-08-19T01:45:36.800 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-08-19T01:45:36.800 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-08-19T01:45:36.801 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.802 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.802 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.802 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-08-19T01:45:36.802 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-08-19T01:45:36.802 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.802 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.803 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.804 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-19T01:45:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-08-19T01:45:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.805 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-19T01:45:36.806 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.807 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.807 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.807 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-08-19T01:45:36.807 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.807 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.807 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.808 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.809 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.809 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-08-19T01:45:36.809 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.809 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.809 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.809 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-08-19T01:45:36.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-08-19T01:45:36.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.810 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.810 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.810 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-08-19T01:45:36.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-08-19T01:45:36.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.812 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.812 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.812 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:36.812 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.812 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.812 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-08-19T01:45:36.813 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.814 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.814 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.814 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-08-19T01:45:36.814 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.814 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.814 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.814 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-19T01:45:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-08-19T01:45:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-08-19T01:45:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-19T01:45:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-08-19T01:45:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-08-19T01:45:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.817 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-08-19T01:45:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-08-19T01:45:36.818 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.819 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.820 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.821 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-08-19T01:45:36.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-08-19T01:45:36.821 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.821 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.821 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.821 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:36.822 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.822 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.822 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.822 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-08-19T01:45:36.822 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.822 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.822 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-19T01:45:36.823 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.824 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-08-19T01:45:36.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-08-19T01:45:36.825 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.825 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.825 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.825 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:36.825 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.825 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-08-19T01:45:36.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-08-19T01:45:36.827 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.827 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.827 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.827 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.827 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.827 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-08-19T01:45:36.828 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.829 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.829 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.829 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.829 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.830 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.831 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.831 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.831 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.831 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.831 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.831 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.831 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.832 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.832 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.832 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.832 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.832 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.832 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.832 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.833 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-08-19T01:45:36.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-08-19T01:45:36.833 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.833 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.834 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.834 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.834 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.834 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.834 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.834 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-19T01:45:36.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.835 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.835 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-19T01:45:36.836 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.837 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.838 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.838 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.838 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.838 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.838 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.838 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.838 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.839 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.840 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.841 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.841 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.841 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.841 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.841 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.841 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.842 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-19T01:45:36.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.845 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.846 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.846 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.846 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.846 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.846 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-08-19T01:45:36.846 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.846 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.847 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.847 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-08-19T01:45:36.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-08-19T01:45:36.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.847 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.847 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.849 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-08-19T01:45:36.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-08-19T01:45:36.849 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.849 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.849 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-08-19T01:45:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.851 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-08-19T01:45:36.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-08-19T01:45:36.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.851 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.851 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.852 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-08-19T01:45:36.853 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.853 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.853 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.853 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-08-19T01:45:36.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-08-19T01:45:36.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.854 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.854 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.854 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-08-19T01:45:36.854 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.854 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.854 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-08-19T01:45:36.855 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.856 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.856 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.856 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-19T01:45:36.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.856 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.856 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.856 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.857 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.858 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.858 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.858 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.858 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.858 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-19T01:45:36.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.859 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.859 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-08-19T01:45:36.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-08-19T01:45:36.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-19T01:45:36.860 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.861 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.861 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.861 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-08-19T01:45:36.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-08-19T01:45:36.861 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.861 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.862 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.862 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.862 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.862 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.862 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-19T01:45:36.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.863 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.863 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.863 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-08-19T01:45:36.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-08-19T01:45:36.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.865 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.865 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-08-19T01:45:36.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-08-19T01:45:36.865 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.865 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.865 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.866 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.866 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.866 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.866 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.866 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-19T01:45:36.866 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.866 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-08-19T01:45:36.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.868 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.869 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.870 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-19T01:45:36.870 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.870 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.870 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.870 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-08-19T01:45:36.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.871 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-19T01:45:36.872 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.872 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.872 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.872 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-08-19T01:45:36.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-08-19T01:45:36.872 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-19T01:45:36.873 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.874 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.874 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.874 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-08-19T01:45:36.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-08-19T01:45:36.874 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.874 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.876 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.876 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-08-19T01:45:36.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-08-19T01:45:36.876 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.876 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.876 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.877 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.878 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-08-19T01:45:36.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-08-19T01:45:36.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.878 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.878 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.878 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.879 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.879 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.879 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.879 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-19T01:45:36.879 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.879 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.879 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.880 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.881 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.881 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.881 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.881 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.881 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.881 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.882 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.882 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.882 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.882 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.882 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.882 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.882 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.883 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.884 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.884 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.884 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.884 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.884 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.884 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.886 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.886 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.886 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.886 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.886 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.886 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.886 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.887 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.887 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.887 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.887 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.887 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.887 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.887 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.888 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.889 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.889 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.889 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.889 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.889 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.890 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-08-19T01:45:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-19T01:45:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.892 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-08-19T01:45:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-19T01:45:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-19T01:45:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-19T01:45:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.897 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-08-19T01:45:36.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-19T01:45:36.897 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.897 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.897 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.897 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.897 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.898 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.898 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.898 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.898 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.898 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.898 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.899 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-08-19T01:45:36.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-08-19T01:45:36.901 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.901 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.901 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.901 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.901 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.901 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.901 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.902 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.902 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.902 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.902 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.902 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-19T01:45:36.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-19T01:45:36.903 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.903 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.903 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.903 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.903 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.903 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-19T01:45:36.904 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.905 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.905 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.905 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.905 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.905 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.905 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.905 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.906 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.906 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.906 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.906 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-19T01:45:36.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-19T01:45:36.906 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-19T01:45:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-19T01:45:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.909 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.910 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.910 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-19T01:45:36.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-19T01:45:36.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.910 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.910 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.911 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.911 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.911 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.912 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.912 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-19T01:45:36.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-19T01:45:36.912 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.912 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.912 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.913 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.914 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-19T01:45:36.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-19T01:45:36.914 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.914 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.914 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.914 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.915 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.915 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.915 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.916 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-19T01:45:36.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-19T01:45:36.916 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.916 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.916 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.918 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-19T01:45:36.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-19T01:45:36.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.918 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.918 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.918 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.918 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.919 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.920 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-19T01:45:36.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-19T01:45:36.920 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.920 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.920 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.920 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.921 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.921 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.921 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.921 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.921 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.921 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.922 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-19T01:45:36.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.924 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-08-19T01:45:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-19T01:45:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.927 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-19T01:45:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-19T01:45:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.930 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-19T01:45:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-19T01:45:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.931 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.932 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.933 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.935 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.936 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.937 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-19T01:45:36.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-19T01:45:36.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.939 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.939 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.939 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.939 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.939 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.940 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.940 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.940 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.940 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.940 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-19T01:45:36.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.941 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-19T01:45:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-19T01:45:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.943 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.944 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.945 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.946 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-19T01:45:36.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-19T01:45:36.946 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.946 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.946 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.948 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-19T01:45:36.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.951 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.952 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.953 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-08-19T01:45:36.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-19T01:45:36.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.953 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.955 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.955 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-19T01:45:36.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-19T01:45:36.955 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.955 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.955 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.956 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-19T01:45:36.956 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.956 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.956 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.956 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.956 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.957 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.957 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-19T01:45:36.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-19T01:45:36.957 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.957 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.957 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.959 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.959 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-19T01:45:36.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-19T01:45:36.959 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.959 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.959 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.960 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.961 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-19T01:45:36.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-19T01:45:36.961 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.961 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.961 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.961 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-08-19T01:45:36.962 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.962 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.962 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.962 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.962 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.962 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.962 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.964 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.964 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.964 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.964 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.964 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.964 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.966 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.966 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.966 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.966 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.966 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.966 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.968 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.968 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.968 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.968 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.968 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-08-19T01:45:36.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-19T01:45:36.969 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.969 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.969 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.969 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:36.969 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.969 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.969 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.970 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.970 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.970 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.970 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.970 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-19T01:45:36.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.972 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.972 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-19T01:45:36.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-19T01:45:36.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.973 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.973 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.973 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:36.973 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.973 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.973 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.973 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.974 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.974 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.974 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.974 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-19T01:45:36.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-19T01:45:36.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.976 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.976 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.976 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-08-19T01:45:36.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-08-19T01:45:36.976 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.976 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.977 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.977 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.977 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.977 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.977 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.977 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.977 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.978 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.979 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.979 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.979 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.979 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.979 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.979 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.979 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.980 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.980 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-08-19T01:45:36.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-08-19T01:45:36.980 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.980 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.980 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.981 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-08-19T01:45:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-08-19T01:45:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.982 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.984 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-08-19T01:45:36.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-08-19T01:45:36.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.984 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.984 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.984 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.984 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.985 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.985 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.985 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.985 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.985 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.985 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.986 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:36.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:36.986 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.986 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.986 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.986 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.986 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.987 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.987 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.987 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.987 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.987 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.987 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.987 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.988 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:36.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:36.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.989 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.989 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.989 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.989 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.990 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.990 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.990 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.990 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.991 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:36.992 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:36.992 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:36.992 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:36.992 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:36.992 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:36.992 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:36.992 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:36.993 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.994 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.994 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:36.994 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.994 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.994 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.995 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:36.996 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.996 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.996 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.996 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-08-19T01:45:36.996 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-08-19T01:45:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-08-19T01:45:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:36.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:36.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:36.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:36.998 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:36.998 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:36.998 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:36.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-08-19T01:45:37.000 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.000 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.000 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.000 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.000 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.000 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.001 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-08-19T01:45:37.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.002 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.002 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.002 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.003 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.004 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-08-19T01:45:37.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-08-19T01:45:37.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-08-19T01:45:37.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.005 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.006 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.007 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-08-19T01:45:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-08-19T01:45:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-08-19T01:45:37.009 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.010 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-08-19T01:45:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-08-19T01:45:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.011 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-08-19T01:45:37.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.014 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.014 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.014 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.014 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.014 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.014 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-08-19T01:45:37.015 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.015 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.015 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.015 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.016 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.017 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.017 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.017 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.017 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.017 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.018 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.018 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.018 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.019 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.019 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.019 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.019 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.019 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.019 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.021 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.021 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.021 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.021 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.021 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.021 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.021 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.022 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.022 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.022 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.022 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.022 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.022 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.022 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.023 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.024 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-08-19T01:45:37.025 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.026 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.026 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-19T01:45:37.026 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.026 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.026 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.027 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.028 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.028 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-19T01:45:37.028 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.028 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.028 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.028 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.029 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.030 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-08-19T01:45:37.030 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.030 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.030 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-08-19T01:45:37.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-08-19T01:45:37.031 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.031 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.031 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.031 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-08-19T01:45:37.031 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.031 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.031 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.032 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-19T01:45:37.032 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.032 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.032 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.032 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-08-19T01:45:37.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-08-19T01:45:37.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.033 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.033 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.033 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.033 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.033 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.034 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-19T01:45:37.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.034 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.034 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.034 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:37.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.035 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-08-19T01:45:37.036 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.036 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.036 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.036 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-08-19T01:45:37.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-08-19T01:45:37.036 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-19T01:45:37.037 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.038 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.038 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.040 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.042 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.044 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.045 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.045 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.045 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.045 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.045 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.045 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.045 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.047 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.048 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.049 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-08-19T01:45:37.049 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.049 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.049 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.049 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.050 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-08-19T01:45:37.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.051 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.053 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.054 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.054 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.054 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.054 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.055 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.056 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-08-19T01:45:37.057 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.058 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.058 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-19T01:45:37.058 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.058 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.058 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.059 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.060 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.060 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.060 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-08-19T01:45:37.060 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.060 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.061 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:37.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:37.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.061 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-08-19T01:45:37.061 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-08-19T01:45:37.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.062 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-19T01:45:37.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.062 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.064 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.065 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.066 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.067 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.067 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.067 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.067 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.068 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.069 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.070 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.070 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.070 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.070 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.070 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.071 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.071 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.071 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.071 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.073 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-08-19T01:45:37.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-08-19T01:45:37.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.074 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.074 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.074 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-08-19T01:45:37.074 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.074 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-08-19T01:45:37.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.075 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-08-19T01:45:37.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-08-19T01:45:37.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-08-19T01:45:37.076 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-08-19T01:45:37.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-08-19T01:45:37.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.078 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.078 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.078 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:37.078 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:37.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-08-19T01:45:37.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-08-19T01:45:37.080 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-08-19T01:45:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-08-19T01:45:37.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-08-19T01:45:37.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-08-19T01:45:37.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-19T01:45:37.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.083 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.083 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.084 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.085 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.086 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.086 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.086 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.086 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.086 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.087 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.087 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.087 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.089 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.089 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.089 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.090 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.091 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.091 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.091 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.091 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.091 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.091 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.091 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.092 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.092 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.093 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.094 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.094 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-08-19T01:45:37.094 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.094 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.094 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.095 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.096 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.097 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.097 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.098 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.099 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.099 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.100 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.100 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.100 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.100 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.101 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.101 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.101 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.101 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.101 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.101 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.102 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.102 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.102 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.102 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.103 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.103 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.103 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.103 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.103 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-08-19T01:45:37.105 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.105 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.105 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.105 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-08-19T01:45:37.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.106 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-19T01:45:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-08-19T01:45:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-08-19T01:45:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-19T01:45:37.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.110 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.110 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.110 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.111 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.112 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.113 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.113 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.114 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.115 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.116 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.118 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.118 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.119 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-08-19T01:45:37.119 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.119 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.119 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.119 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-08-19T01:45:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-19T01:45:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.124 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.126 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.126 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.127 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.128 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.128 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.128 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.128 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.128 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.128 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.129 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-08-19T01:45:37.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-19T01:45:37.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.133 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-08-19T01:45:37.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-08-19T01:45:37.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.137 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-08-19T01:45:37.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-08-19T01:45:37.137 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.137 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-08-19T01:45:37.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.139 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-08-19T01:45:37.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-08-19T01:45:37.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-08-19T01:45:37.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.140 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-19T01:45:37.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-19T01:45:37.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.141 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.141 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-08-19T01:45:37.141 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.141 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.141 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.142 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.143 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.144 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.144 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.148 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.148 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.148 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.148 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.148 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.149 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.151 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.151 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.152 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-08-19T01:45:37.152 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.152 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.152 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.152 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.154 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.154 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.154 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.154 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.154 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.154 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.154 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.155 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.155 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.156 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.156 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.157 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.158 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.159 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.159 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.159 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.159 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.159 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.159 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.159 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.162 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.162 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.163 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-08-19T01:45:37.163 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.163 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.163 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.163 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.164 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.168 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.169 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.170 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.170 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-08-19T01:45:37.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.176 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.177 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.177 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.177 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.177 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.178 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.180 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.180 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.181 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.181 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.181 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.181 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.181 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.181 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.182 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.182 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.182 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.183 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.184 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.184 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.185 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-08-19T01:45:37.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-08-19T01:45:37.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.185 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.186 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-08-19T01:45:37.186 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-08-19T01:45:37.186 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-08-19T01:45:37.186 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.189 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.189 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.189 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.190 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.190 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.190 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.190 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.191 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.192 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.194 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.194 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.194 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.194 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.196 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.198 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.198 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-08-19T01:45:37.198 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.198 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.198 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.199 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.200 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.200 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.200 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.200 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.200 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.201 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.201 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.202 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.203 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.206 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.210 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.210 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.210 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.210 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.210 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.214 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.216 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.216 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.218 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:37.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:37.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:37.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-08-19T01:45:37.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-08-19T01:45:37.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-19T01:45:37.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.222 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-08-19T01:45:37.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-19T01:45:37.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.225 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-08-19T01:45:37.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-08-19T01:45:37.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.226 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.226 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.226 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.226 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.228 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.228 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.228 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.230 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.231 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.231 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.232 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.233 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.234 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.234 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-08-19T01:45:37.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-08-19T01:45:37.237 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-08-19T01:45:37.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-08-19T01:45:37.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-08-19T01:45:37.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-08-19T01:45:37.241 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.241 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.241 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.241 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-08-19T01:45:37.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-08-19T01:45:37.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.243 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-08-19T01:45:37.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-08-19T01:45:37.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-19T01:45:37.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.246 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.247 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.247 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.247 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.248 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.250 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.251 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.252 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-08-19T01:45:37.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-08-19T01:45:37.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-19T01:45:37.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-08-19T01:45:37.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:37.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.260 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:37.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:37.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.262 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:37.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:37.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.262 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-08-19T01:45:37.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-19T01:45:37.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-08-19T01:45:37.264 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.265 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-08-19T01:45:37.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-08-19T01:45:37.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-08-19T01:45:37.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.266 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-08-19T01:45:37.267 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-08-19T01:45:37.267 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-08-19T01:45:37.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-08-19T01:45:37.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-08-19T01:45:37.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.270 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.270 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-08-19T01:45:37.270 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.272 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.272 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.272 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.272 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.272 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.274 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.278 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.282 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-08-19T01:45:37.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-08-19T01:45:37.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.284 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-08-19T01:45:37.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-08-19T01:45:37.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.286 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.286 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.286 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-08-19T01:45:37.286 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.286 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.288 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-08-19T01:45:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-08-19T01:45:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.290 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.290 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.292 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.293 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.294 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.298 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.300 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.300 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-08-19T01:45:37.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-08-19T01:45:37.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.301 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.302 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-08-19T01:45:37.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-08-19T01:45:37.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.303 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-08-19T01:45:37.304 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.305 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.305 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.305 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.306 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.307 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.307 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.307 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.307 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.308 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.310 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.310 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.310 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.311 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.311 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.311 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.311 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.312 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.312 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.312 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.312 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.315 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.316 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.316 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.316 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.316 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.316 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.317 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.318 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.318 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.318 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.320 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.321 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.322 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.322 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.323 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.323 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.323 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.323 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.323 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.324 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.326 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-08-19T01:45:37.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-08-19T01:45:37.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.326 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-08-19T01:45:37.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.327 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-19T01:45:37.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.328 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.328 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.328 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-08-19T01:45:37.328 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.329 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.329 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.329 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.329 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.329 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.330 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.331 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.331 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.331 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.331 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.332 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.333 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.334 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.335 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.335 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.335 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.335 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.335 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.335 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.335 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.336 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.338 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.338 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.338 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.338 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.338 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-08-19T01:45:37.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-08-19T01:45:37.339 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.339 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.339 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.339 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.339 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.339 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.340 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-08-19T01:45:37.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.342 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.342 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.342 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-08-19T01:45:37.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-08-19T01:45:37.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.344 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.345 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.346 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-08-19T01:45:37.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-08-19T01:45:37.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-08-19T01:45:37.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.349 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-08-19T01:45:37.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-08-19T01:45:37.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.351 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.353 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-08-19T01:45:37.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-08-19T01:45:37.354 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.354 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-08-19T01:45:37.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-08-19T01:45:37.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.355 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.355 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.355 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.355 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-08-19T01:45:37.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-08-19T01:45:37.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.356 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-08-19T01:45:37.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.358 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.359 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.359 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-08-19T01:45:37.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-08-19T01:45:37.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.360 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.360 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.360 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.361 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.361 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.361 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-08-19T01:45:37.362 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.362 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.362 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.362 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-08-19T01:45:37.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-08-19T01:45:37.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.363 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-08-19T01:45:37.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.364 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.364 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.365 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-08-19T01:45:37.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.366 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.369 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.371 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.371 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.371 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.371 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.372 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.372 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.372 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.372 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.373 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.373 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.373 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.373 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.373 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.373 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.374 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.374 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.375 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.375 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.378 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.378 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.379 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.380 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.381 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.382 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.382 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.382 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.383 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.383 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.383 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.385 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.386 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.386 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.387 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.387 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.387 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-08-19T01:45:37.387 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.387 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.388 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.389 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.390 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.390 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.391 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.392 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.394 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.394 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.394 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.394 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.394 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.394 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.394 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.396 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.397 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.397 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-08-19T01:45:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-08-19T01:45:37.399 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.399 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.399 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-19T01:45:37.399 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.400 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:37.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:37.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.401 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.401 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-08-19T01:45:37.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.403 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.403 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.403 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.404 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.405 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.405 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.405 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.407 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.408 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.410 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.410 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.412 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.412 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.412 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-08-19T01:45:37.412 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.412 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.414 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-19T01:45:37.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-08-19T01:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-19T01:45:37.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.420 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.421 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-19T01:45:37.421 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.421 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.422 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.424 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-08-19T01:45:37.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-08-19T01:45:37.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.424 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.424 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.426 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-08-19T01:45:37.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-08-19T01:45:37.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-19T01:45:37.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.427 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.427 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.427 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.427 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.429 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.429 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:37.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:37.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-08-19T01:45:37.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-08-19T01:45:37.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-08-19T01:45:37.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-08-19T01:45:37.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-08-19T01:45:37.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.432 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-08-19T01:45:37.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.436 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.437 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.438 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.438 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.438 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.440 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.442 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.443 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.443 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.443 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.443 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.443 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.444 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.444 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.448 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.448 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.450 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-08-19T01:45:37.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-08-19T01:45:37.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-19T01:45:37.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.451 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.451 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.452 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-08-19T01:45:37.453 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.453 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.453 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.453 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.454 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.454 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.454 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.455 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.455 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.455 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.455 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.455 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.455 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.456 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.456 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.457 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.457 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.457 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.457 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.457 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.458 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.459 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.461 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.461 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.461 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.461 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.462 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.463 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.465 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.466 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.466 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.468 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.470 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.470 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.470 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.470 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.470 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.470 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.471 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.471 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.472 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.472 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.472 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.472 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-08-19T01:45:37.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-08-19T01:45:37.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.474 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.474 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.474 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-08-19T01:45:37.474 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.474 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-08-19T01:45:37.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-08-19T01:45:37.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.476 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.476 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.476 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.476 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.476 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.476 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-08-19T01:45:37.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.477 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-08-19T01:45:37.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-08-19T01:45:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-08-19T01:45:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-08-19T01:45:37.479 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.480 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.480 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-08-19T01:45:37.480 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.480 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.480 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-08-19T01:45:37.482 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-08-19T01:45:37.482 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-08-19T01:45:37.482 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-08-19T01:45:37.482 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-08-19T01:45:37.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-08-19T01:45:37.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-08-19T01:45:37.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-08-19T01:45:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-08-19T01:45:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.484 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.485 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.486 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-08-19T01:45:37.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-08-19T01:45:37.486 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-08-19T01:45:37.486 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.486 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.486 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.488 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-08-19T01:45:37.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-08-19T01:45:37.488 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.488 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.488 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.488 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.488 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.489 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.490 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.490 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-08-19T01:45:37.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-08-19T01:45:37.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.491 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.491 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.491 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.491 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.491 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-08-19T01:45:37.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-08-19T01:45:37.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.493 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-08-19T01:45:37.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-08-19T01:45:37.494 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-08-19T01:45:37.494 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-08-19T01:45:37.494 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-08-19T01:45:37.494 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-08-19T01:45:37.494 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-08-19T01:45:37.494 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-08-19T01:45:37.494 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-08-19T01:45:37.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-08-19T01:45:37.496 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.496 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-08-19T01:45:37.496 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-08-19T01:45:37.496 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.497 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.497 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.497 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-08-19T01:45:37.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-08-19T01:45:37.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-08-19T01:45:37.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.498 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-08-19T01:45:37.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.500 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-08-19T01:45:37.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.501 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.501 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-08-19T01:45:37.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-08-19T01:45:37.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-08-19T01:45:37.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-08-19T01:45:37.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-08-19T01:45:37.502 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-19T01:45:37.502 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-08-19T01:45:37.502 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-08-19T01:45:37.502 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-08-19T01:45:37.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-08-19T01:45:37.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: "services": { 2024-08-19T01:45:37.503 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard": "https://172.21.15.103:8443/" 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: "always_on_modules": { 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: "octopus": [ 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-08-19T01:45:37.504 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: "pacific": [ 2024-08-19T01:45:37.505 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-08-19T01:45:37.506 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-08-19T01:45:37.506 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-08-19T01:45:37.506 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-08-19T01:45:37.506 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-08-19T01:45:37.506 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-08-19T01:45:37.506 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-08-19T01:45:37.506 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: "quincy": [ 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-08-19T01:45:37.507 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.508 INFO:teuthology.orchestra.run.smithi103.stdout: "reef": [ 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-08-19T01:45:37.509 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-08-19T01:45:37.510 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-08-19T01:45:37.510 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-08-19T01:45:37.510 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-08-19T01:45:37.510 INFO:teuthology.orchestra.run.smithi103.stdout: "squid": [ 2024-08-19T01:45:37.510 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-08-19T01:45:37.510 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-08-19T01:45:37.510 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-08-19T01:45:37.511 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:37.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.512 INFO:teuthology.orchestra.run.smithi103.stdout: "last_failure_osd_epoch": 3, 2024-08-19T01:45:37.512 INFO:teuthology.orchestra.run.smithi103.stdout: "active_clients": [ 2024-08-19T01:45:37.512 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-08-19T01:45:37.512 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-08-19T01:45:37.512 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-08-19T01:45:37.513 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-08-19T01:45:37.513 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 1423964731 2024-08-19T01:45:37.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.513 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:37.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.513 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "libcephsqlite", 2024-08-19T01:45:37.514 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-08-19T01:45:37.514 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-08-19T01:45:37.514 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-08-19T01:45:37.514 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 927767623 2024-08-19T01:45:37.514 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-08-19T01:45:37.515 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-08-19T01:45:37.516 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3263898905 2024-08-19T01:45:37.516 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.516 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:37.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:37.516 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-08-19T01:45:37.516 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-08-19T01:45:37.517 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-08-19T01:45:37.517 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-08-19T01:45:37.517 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-08-19T01:45:37.517 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3433232620 2024-08-19T01:45:37.517 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.517 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:37.517 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:37.518 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-08-19T01:45:37.518 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:45:37.519 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:35 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1474192429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.519 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:35 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1793933086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.520 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:35 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/528991508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.520 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:35 smithi146 ceph-mon[22098]: pgmap v633: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 1.65k op/s 2024-08-19T01:45:37.520 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:35 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1035963782' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-19T01:45:37.520 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:35 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1474192429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.520 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:35 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1793933086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.520 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:35 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/528991508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.520 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:35 smithi190 ceph-mon[22032]: pgmap v633: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 1.65k op/s 2024-08-19T01:45:37.521 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:35 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1035963782' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-19T01:45:37.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb38c0a2b90 msgr2=0x7fb38c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:37.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb38c0a2b90 0x7fb38c0a5050 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb388005fd0 tx=0x7fb388005850 comp rx=0 tx=0).stop 2024-08-19T01:45:37.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c100760 msgr2=0x7fb39c06aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:45:37.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c100760 0x7fb39c06aca0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c970 tx=0x7fb39000ce30 comp rx=0 tx=0).stop 2024-08-19T01:45:37.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 shutdown_connections 2024-08-19T01:45:37.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fb38c0a2b90 0x7fb38c0a5050 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:37.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb39c102310 0x7fb39c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:37.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb39c101950 0x7fb39c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:37.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 --2- 172.21.15.103:0/1035963782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb39c100760 0x7fb39c06aca0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:45:37.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 >> 172.21.15.103:0/1035963782 conn(0x7fb39c0fbee0 msgr2=0x7fb39c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:45:37.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.054+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 shutdown_connections 2024-08-19T01:45:37.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:45:35.055+0000 7fb3a30fc640 1 -- 172.21.15.103:0/1035963782 wait complete. 2024-08-19T01:45:37.530 DEBUG:tasks.check_counter:Getting stats from a 2024-08-19T01:45:37.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-19T01:45:37.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:35 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1474192429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:35 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1793933086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:35 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/528991508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-19T01:45:37.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:35 smithi103 ceph-mon[18436]: pgmap v633: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 1.65k op/s 2024-08-19T01:45:37.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:35 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1035963782' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-19T01:45:37.799 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:37.922 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:37 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:45:37.922 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:37 smithi103 ceph-mon[18436]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 974 op/s 2024-08-19T01:45:37.950 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:37 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:45:37.951 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:37 smithi146 ceph-mon[22098]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 974 op/s 2024-08-19T01:45:38.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:37 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:45:38.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:37 smithi190 ceph-mon[22032]: pgmap v634: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.8 MiB/s wr, 974 op/s 2024-08-19T01:45:38.360 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:45:38.360 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:38.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1513, 2024-08-19T01:45:38.360 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1515, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 784131, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 481135, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.113940766, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.037825147, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.030049317, 2024-08-19T01:45:38.361 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.036196884, 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1513, 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.043495994, 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000028748 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 230, 2024-08-19T01:45:38.363 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 212, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 593294, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 126270, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.026513091, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.012415199, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.018149695, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000064651, 2024-08-19T01:45:38.364 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 211, 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.009867091, 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000046763 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 591904, 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 125248 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 625, 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 775, 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 155646, 2024-08-19T01:45:38.366 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 544648, 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.057617320, 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.020411013, 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.013832733, 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.018738973, 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:38.367 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 774, 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.024488090, 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000031638 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 347488 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-08-19T01:45:38.369 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3564, 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.079422268, 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000022284 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:38.370 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-08-19T01:45:38.371 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-08-19T01:45:38.372 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 39, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 39, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:38.373 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 39, 2024-08-19T01:45:38.374 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-08-19T01:45:38.374 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-08-19T01:45:38.374 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-08-19T01:45:38.374 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 4, 2024-08-19T01:45:38.374 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-08-19T01:45:38.374 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-08-19T01:45:38.374 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-08-19T01:45:38.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-08-19T01:45:38.376 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-08-19T01:45:38.377 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:45:38.378 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-08-19T01:45:38.379 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:38.380 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:38.381 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-08-19T01:45:38.382 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 272, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-08-19T01:45:38.383 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4529670, 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4528588, 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-08-19T01:45:38.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 272 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 42, 2024-08-19T01:45:38.385 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 42, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 11, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 11, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 39, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 39, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-08-19T01:45:38.386 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 51268, 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-08-19T01:45:38.387 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-08-19T01:45:38.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.393 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.394 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.395 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-08-19T01:45:38.395 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.397 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-08-19T01:45:38.398 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.399 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.399 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.401 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:38.401 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.401 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.402 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.402 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:38.402 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.402 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.403 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.405 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.406 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.407 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.408 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 1, 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 0, 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-08-19T01:45:38.410 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1494, 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:38.411 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:38.412 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:38.413 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:38.414 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:38.414 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:38.414 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:38.414 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:38.414 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:38.414 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:38.414 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:38.415 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 197503, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 55, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-08-19T01:45:38.416 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 153, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:38.417 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 131461, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 403, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:38.418 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 2136, 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 2136, 2024-08-19T01:45:38.419 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2136, 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.544971114, 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000723301 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2136, 2024-08-19T01:45:38.420 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2137 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 2136, 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 2136, 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 1341, 2024-08-19T01:45:38.421 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-08-19T01:45:38.422 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 793, 2024-08-19T01:45:38.422 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-08-19T01:45:38.422 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-08-19T01:45:38.422 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-08-19T01:45:38.422 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-08-19T01:45:38.422 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-08-19T01:45:38.422 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:38.423 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-08-19T01:45:38.424 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-08-19T01:45:38.425 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-08-19T01:45:38.426 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 86, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 2, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 6, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 6, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-08-19T01:45:38.427 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-08-19T01:45:38.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:38.429 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:38.430 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-08-19T01:45:38.430 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:38.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.430 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:38.430 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:38.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-08-19T01:45:38.430 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2364, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1305774, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2364, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2364, 2024-08-19T01:45:38.431 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1305774, 2024-08-19T01:45:38.432 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:38.432 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.432 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.432 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.432 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.432 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2136, 2024-08-19T01:45:38.433 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 335371, 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2136, 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 335371, 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:38.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2136, 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2136, 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2136, 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2136, 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.436 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:38.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:38.438 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:38.438 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:38.438 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-08-19T01:45:38.438 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-08-19T01:45:38.438 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:38.438 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.438 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55880eb96b40": { 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:38.439 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-08-19T01:45:38.440 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:38.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:38.441 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:38.441 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:38.441 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.441 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:38.441 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:45:39.216 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:39 smithi103 ceph-mon[18436]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 452 op/s 2024-08-19T01:45:39.243 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-08-19T01:45:39.243 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-08-19T01:45:39.243 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-08-19T01:45:39.244 DEBUG:tasks.check_counter:Getting stats from d 2024-08-19T01:45:39.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-19T01:45:39.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:39 smithi190 ceph-mon[22032]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 452 op/s 2024-08-19T01:45:39.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:39 smithi146 ceph-mon[22098]: pgmap v635: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 452 op/s 2024-08-19T01:45:39.499 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:40.060 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:45:40.060 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:40.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-08-19T01:45:40.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4, 2024-08-19T01:45:40.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2944, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 448, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.003107841, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000040094, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002212674, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:45:40.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.004120820, 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001373606 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:40.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 158, 2024-08-19T01:45:40.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 105936, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.010296556, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007914994, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001186862, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:45:40.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 156, 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.005158364, 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000033066 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:40.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:45:40.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 105936 2024-08-19T01:45:40.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.066 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:40.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 210, 2024-08-19T01:45:40.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 207, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 475680, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 127264, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.024482695, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.011949808, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.015729999, 2024-08-19T01:45:40.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 206, 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.007099037, 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000034461 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:40.068 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 475680, 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 127264 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-08-19T01:45:40.069 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:40.070 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:40.071 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:40.072 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:40.073 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148984, 2024-08-19T01:45:40.074 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:40.075 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:40.076 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-08-19T01:45:40.076 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-08-19T01:45:40.076 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:40.076 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:40.076 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:40.076 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-08-19T01:45:40.076 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:40.077 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 211, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 446019, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 211, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:40.078 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 211, 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 446019, 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:40.079 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:40.080 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:45:40.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:40 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:45:40.378 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:40 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:45:40.379 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:40 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-19T01:45:40.912 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-19T01:45:40.912 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-19T01:45:40.912 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-08-19T01:45:40.912 DEBUG:tasks.check_counter:Getting stats from g 2024-08-19T01:45:40.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-19T01:45:41.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:41 smithi103 ceph-mon[18436]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 352 KiB/s wr, 102 op/s 2024-08-19T01:45:41.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:41 smithi190 ceph-mon[22032]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 352 KiB/s wr, 102 op/s 2024-08-19T01:45:41.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:41 smithi146 ceph-mon[22098]: pgmap v636: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 352 KiB/s wr, 102 op/s 2024-08-19T01:45:41.801 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:42.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.238 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:42 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.346 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:45:42.346 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:42.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 732, 2024-08-19T01:45:42.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 732, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 247497, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 231953, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.071186672, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.026958911, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.018392892, 2024-08-19T01:45:42.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.019424819, 2024-08-19T01:45:42.348 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:42.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 731, 2024-08-19T01:45:42.348 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.029200074, 2024-08-19T01:45:42.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000039945 2024-08-19T01:45:42.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.348 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:42.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1480, 2024-08-19T01:45:42.349 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1634, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 883695, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 821003, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.092971820, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.031348791, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.022580338, 2024-08-19T01:45:42.350 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.032095357, 2024-08-19T01:45:42.351 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:42.351 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1632, 2024-08-19T01:45:42.351 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.043381602, 2024-08-19T01:45:42.351 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000026581 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:45:42.352 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 350624 2024-08-19T01:45:42.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.353 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:42.353 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 232, 2024-08-19T01:45:42.353 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 214, 2024-08-19T01:45:42.353 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 593796, 2024-08-19T01:45:42.353 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 127493, 2024-08-19T01:45:42.353 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.026772819, 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.011904558, 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.019158405, 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000263719, 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 213, 2024-08-19T01:45:42.354 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.008564749, 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000040210 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.355 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 592448, 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 126528 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-08-19T01:45:42.356 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3749, 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.082838337, 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000022096 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-08-19T01:45:42.357 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-08-19T01:45:42.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.358 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.358 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.358 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-08-19T01:45:42.358 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-08-19T01:45:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 244, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 90, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 154, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:42.360 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 117, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 7, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-08-19T01:45:42.361 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-08-19T01:45:42.362 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-08-19T01:45:42.363 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:45:42.364 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:42.365 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-08-19T01:45:42.366 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:42.367 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:42.368 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-08-19T01:45:42.369 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-08-19T01:45:42.370 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-08-19T01:45:42.370 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 316, 2024-08-19T01:45:42.370 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-08-19T01:45:42.370 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-08-19T01:45:42.370 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-08-19T01:45:42.370 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-08-19T01:45:42.370 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4702309, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4699549, 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-08-19T01:45:42.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 316 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 187, 2024-08-19T01:45:42.372 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 187, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 30, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 30, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 244, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 244, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-08-19T01:45:42.373 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 52148, 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-08-19T01:45:42.374 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.376 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-08-19T01:45:42.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.378 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.378 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:42.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.378 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-08-19T01:45:42.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.381 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.382 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.382 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-08-19T01:45:42.382 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.382 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.382 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.383 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.384 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-08-19T01:45:42.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.384 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.384 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.384 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.385 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:42.386 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.388 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:42.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.390 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.390 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:42.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.390 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-08-19T01:45:42.391 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:42.393 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-08-19T01:45:42.394 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 2, 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1500, 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:42.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:42.396 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:42.397 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:42.398 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:42.399 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 197463, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 54, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:42.400 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 153, 2024-08-19T01:45:42.401 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:42.402 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:42.402 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:42.402 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-08-19T01:45:42.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 600344, 2024-08-19T01:45:42.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 1826, 2024-08-19T01:45:42.402 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:42.402 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 2210, 2024-08-19T01:45:42.403 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-08-19T01:45:42.404 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-08-19T01:45:42.404 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 2210, 2024-08-19T01:45:42.404 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-08-19T01:45:42.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2210, 2024-08-19T01:45:42.404 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.212977241, 2024-08-19T01:45:42.404 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000548858 2024-08-19T01:45:42.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2210, 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2211 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 2210, 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 2210, 2024-08-19T01:45:42.405 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-08-19T01:45:42.406 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-08-19T01:45:42.406 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-08-19T01:45:42.406 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 1368, 2024-08-19T01:45:42.406 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-08-19T01:45:42.406 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 840, 2024-08-19T01:45:42.406 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-08-19T01:45:42.406 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:42.407 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-08-19T01:45:42.408 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-08-19T01:45:42.409 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-08-19T01:45:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-08-19T01:45:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-08-19T01:45:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-08-19T01:45:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-08-19T01:45:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-08-19T01:45:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-08-19T01:45:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 86, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 2, 2024-08-19T01:45:42.411 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 6, 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 6, 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-08-19T01:45:42.412 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.413 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:42.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2440, 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1490747, 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:42.415 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2440, 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2440, 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1490747, 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.416 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:42.417 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2210, 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 508010, 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2210, 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 508010, 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.418 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:42.419 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 2210, 2024-08-19T01:45:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2210, 2024-08-19T01:45:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2210, 2024-08-19T01:45:42.420 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2210, 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:42.421 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:42.422 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-08-19T01:45:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-08-19T01:45:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.423 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55cca6374b40": { 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:42.424 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:42.425 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:42.426 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:45:42.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:42 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:42.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:42 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:43.211 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-08-19T01:45:43.211 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-08-19T01:45:43.211 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-08-19T01:45:43.211 DEBUG:tasks.check_counter:Getting stats from j 2024-08-19T01:45:43.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-19T01:45:43.236 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:43 smithi103 ceph-mon[18436]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-08-19T01:45:43.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:43 smithi190 ceph-mon[22032]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-08-19T01:45:43.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:43 smithi146 ceph-mon[22098]: pgmap v637: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-08-19T01:45:43.479 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:45:44.024 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-08-19T01:45:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:44.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.001125302, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-08-19T01:45:44.025 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000235942, 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:44.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 5, 2024-08-19T01:45:44.027 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 161, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3104, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 106560, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.013213751, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.008915821, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.003070578, 2024-08-19T01:45:44.028 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 160, 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.009498503, 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000059365 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:44.029 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 106560 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 212, 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 209, 2024-08-19T01:45:44.030 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 476224, 2024-08-19T01:45:44.031 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 128544, 2024-08-19T01:45:44.031 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-08-19T01:45:44.031 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-08-19T01:45:44.031 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.026476511, 2024-08-19T01:45:44.031 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.012335609, 2024-08-19T01:45:44.031 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.017795222, 2024-08-19T01:45:44.031 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 208, 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.007683530, 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000036940 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:44.032 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 476224, 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 128544 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-08-19T01:45:44.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:44.034 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:44.035 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:44.036 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-08-19T01:45:44.037 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148984, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:44.038 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-08-19T01:45:44.039 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:44.040 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 213, 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 446279, 2024-08-19T01:45:44.041 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 213, 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 213, 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 446279, 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-08-19T01:45:44.042 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-08-19T01:45:44.043 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-08-19T01:45:44.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:44.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-08-19T01:45:44.043 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-08-19T01:45:44.700 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:45:44.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:45:44.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:44 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:45:44.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:45:44.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:44 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-19T01:45:44.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-19T01:45:44.881 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:44 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' 2024-08-19T01:45:44.902 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-19T01:45:44.903 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-19T01:45:44.903 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-08-19T01:45:44.903 DEBUG:tasks.check_counter:Getting stats from b 2024-08-19T01:45:44.903 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-19T01:45:45.168 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:45:45.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:45 smithi146 ceph-mon[22098]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-08-19T01:45:45.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:45 smithi103 ceph-mon[18436]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-08-19T01:45:45.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:45 smithi190 ceph-mon[22032]: pgmap v638: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-08-19T01:45:46.642 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-19T01:45:46.643 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:46.643 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 3178, 2024-08-19T01:45:46.643 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 3005, 2024-08-19T01:45:46.643 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 1171877, 2024-08-19T01:45:46.643 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 9327121, 2024-08-19T01:45:46.643 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 13, 2024-08-19T01:45:46.643 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 2, 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.252465353, 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.100659603, 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.116411850, 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.036751144, 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3006, 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.251471461, 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000083656 2024-08-19T01:45:46.644 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2886, 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000631224, 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000218 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:46.645 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 7242, 2024-08-19T01:45:46.646 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 8829, 2024-08-19T01:45:46.646 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 2536031, 2024-08-19T01:45:46.646 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 6440935, 2024-08-19T01:45:46.646 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 15, 2024-08-19T01:45:46.646 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 8, 2024-08-19T01:45:46.646 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.466662602, 2024-08-19T01:45:46.646 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.173607437, 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.262732701, 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.064657421, 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 8824, 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.325755400, 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000036916 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:46.647 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 13311, 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.002306799, 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000173 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 55536, 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 387296 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 2855, 2024-08-19T01:45:46.648 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 2717, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 1679892, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 3539697, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 191, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.279702980, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.095609849, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.190001596, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.017854659, 2024-08-19T01:45:46.649 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2717, 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.505715407, 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000186130 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3263, 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000682383, 2024-08-19T01:45:46.650 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000209 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 632016, 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 147104 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.651 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2201, 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.724217062, 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000783378 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:46.652 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 914, 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.040308385, 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000044101 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-08-19T01:45:46.653 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 2741, 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 2720, 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2720, 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 8.223034253, 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.003023174 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-08-19T01:45:46.654 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 21, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 10, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 4, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 318, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 58, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 6, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 41, 2024-08-19T01:45:46.655 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 11, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 49, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 347, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 32, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 32, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 5, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 4177, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 3904, 2024-08-19T01:45:46.656 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 21, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 4, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 52, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 459, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 6, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 341, 2024-08-19T01:45:46.657 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 1, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 28, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 572, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 325, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 13424230473, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-08-19T01:45:46.658 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 572, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 180, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 12, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 168, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 392, 2024-08-19T01:45:46.659 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 997, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 1351, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 6, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 2761, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 882, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 16, 2024-08-19T01:45:46.660 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 1944 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 4, 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 6, 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 28, 2024-08-19T01:45:46.661 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-08-19T01:45:46.662 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:46.662 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:46.662 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 530, 2024-08-19T01:45:46.662 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 469, 2024-08-19T01:45:46.662 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 61, 2024-08-19T01:45:46.663 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-08-19T01:45:46.663 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:46.663 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:46.663 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:46.663 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-08-19T01:45:46.663 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-08-19T01:45:46.663 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 12, 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 6, 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-08-19T01:45:46.664 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 3701, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 3679, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 3679, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 22, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 31, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 50, 2024-08-19T01:45:46.665 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 49, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 49, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 1, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 1, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 1, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 12128774, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 12164843, 2024-08-19T01:45:46.666 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 655, 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 7.967908508, 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.012164745 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 61, 2024-08-19T01:45:46.667 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 826, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 765, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 24, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 125, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 101, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 58, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 1065, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 1007, 2024-08-19T01:45:46.668 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 32, 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 792, 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 760, 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 60888, 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 231712 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 3300, 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 1, 2024-08-19T01:45:46.669 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 2741, 2024-08-19T01:45:46.670 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 21, 2024-08-19T01:45:46.670 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 2, 2024-08-19T01:45:46.670 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-08-19T01:45:46.670 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 641, 2024-08-19T01:45:46.670 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.612944453, 2024-08-19T01:45:46.670 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000956231 2024-08-19T01:45:46.670 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 726, 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.168515378, 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000232114 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.671 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 61, 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.116612908, 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001911687 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-08-19T01:45:46.672 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 247, 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.221474270, 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000896656 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.673 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-08-19T01:45:46.674 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.674 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.674 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.674 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.674 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-08-19T01:45:46.674 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.674 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:46.675 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.676 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.676 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.676 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.676 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-08-19T01:45:46.676 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.676 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.676 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 105, 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.099759075, 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000950086 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.677 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-08-19T01:45:46.678 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 144, 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.098600652, 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000684726 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 158, 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.508210039, 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.003216519 2024-08-19T01:45:46.679 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 113, 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 5.650961745, 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.050008511 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.680 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-08-19T01:45:46.681 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 16, 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.103731184, 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.006483199 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.682 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4, 2024-08-19T01:45:46.683 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.005729201, 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001432300 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:46.684 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4, 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001809766, 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000452441 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.685 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 7, 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.018388691, 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002626955 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 13, 2024-08-19T01:45:46.686 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.009707093, 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000746699 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 481, 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.606589798, 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001261101 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-08-19T01:45:46.688 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 56, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 56, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 628, 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:46.689 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:46.690 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:46.691 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:46.692 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-08-19T01:45:46.693 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 306254, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 116, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:46.694 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 153, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:46.695 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 222506, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 1571, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:46.696 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 3374, 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 8759002, 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 37, 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 3374, 2024-08-19T01:45:46.697 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3374, 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 37.329485036, 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.011063866 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3374, 2024-08-19T01:45:46.698 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 4968 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 3374, 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 641, 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 2733, 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 219, 2024-08-19T01:45:46.699 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 433, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 43, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 686, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 168, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 983, 2024-08-19T01:45:46.700 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 598, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 866, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:46.701 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 6, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-08-19T01:45:46.702 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-08-19T01:45:46.703 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 86, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-08-19T01:45:46.704 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 12, 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 814, 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 28, 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 124 2024-08-19T01:45:46.705 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-08-19T01:45:46.706 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 6, 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 6, 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 469, 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 469, 2024-08-19T01:45:46.707 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 13271, 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 4169544, 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:46.708 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 13271, 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 13271, 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 4169544, 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.709 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:46.710 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 3374, 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 8806195, 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 3374, 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 8806195, 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.711 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:46.712 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 3374, 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 3374, 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 3374, 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 3374, 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.713 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 469, 2024-08-19T01:45:46.714 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 47369, 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 469, 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 31, 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 47369, 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:46.715 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55a48dc98960": { 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:46.716 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 3701, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 7970539, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 3701, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 655, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 7970539, 2024-08-19T01:45:46.717 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:46.718 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:46.718 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:46.718 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:46.718 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.718 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:46.718 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-19T01:45:47.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:47 smithi146 ceph-mon[22098]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:45:47.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:47 smithi103 ceph-mon[18436]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:45:47.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:47 smithi190 ceph-mon[22032]: pgmap v639: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:45:47.518 INFO:tasks.check_counter:Daemon mds.b mds.exported=6 2024-08-19T01:45:47.518 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-08-19T01:45:47.518 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-08-19T01:45:47.518 DEBUG:tasks.check_counter:Getting stats from e 2024-08-19T01:45:47.519 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-19T01:45:47.770 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:45:48.314 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-19T01:45:48.326 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:48.326 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 952, 2024-08-19T01:45:48.326 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 954, 2024-08-19T01:45:48.326 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 4498069, 2024-08-19T01:45:48.326 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 302746, 2024-08-19T01:45:48.326 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 6, 2024-08-19T01:45:48.326 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 2, 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.097139167, 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.037852252, 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.023702301, 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.025753217, 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 952, 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.046597535, 2024-08-19T01:45:48.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000048946 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:48.328 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 779, 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 761, 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 728464, 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 302946, 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:48.329 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:48.330 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.078980633, 2024-08-19T01:45:48.330 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.026289720, 2024-08-19T01:45:48.330 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.036564148, 2024-08-19T01:45:48.330 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.018821983, 2024-08-19T01:45:48.330 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:48.330 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 760, 2024-08-19T01:45:48.330 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.029641350, 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000039001 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.331 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 593968, 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 129888 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 875, 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1031, 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 4002246, 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 630053, 2024-08-19T01:45:48.332 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.077845068, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.025358513, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.023695200, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.021844044, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1030, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.034574098, 2024-08-19T01:45:48.333 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000033567 2024-08-19T01:45:48.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.334 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:48.334 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.334 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.334 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.334 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 353392 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-08-19T01:45:48.335 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3845, 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.102631418, 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000026692 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:48.336 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-08-19T01:45:48.337 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-08-19T01:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 579, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 376, 2024-08-19T01:45:48.339 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 203, 2024-08-19T01:45:48.340 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:48.340 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 27, 2024-08-19T01:45:48.340 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-08-19T01:45:48.340 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-08-19T01:45:48.340 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 4, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:48.341 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:48.342 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 325, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 13424230473, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:48.343 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:48.344 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-08-19T01:45:48.345 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-08-19T01:45:48.346 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:48.347 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.348 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 3701, 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-08-19T01:45:48.349 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-08-19T01:45:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-08-19T01:45:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-08-19T01:45:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-08-19T01:45:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 47, 2024-08-19T01:45:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-08-19T01:45:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-08-19T01:45:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 12128774, 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 12164843, 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 12160542, 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.351 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 3701 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 50, 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 799, 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 749, 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 17, 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 116, 2024-08-19T01:45:48.352 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 99, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 579, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 1042, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 463, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 50596, 2024-08-19T01:45:48.353 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215328 2024-08-19T01:45:48.354 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.354 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-08-19T01:45:48.354 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-08-19T01:45:48.354 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-08-19T01:45:48.354 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-08-19T01:45:48.354 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-08-19T01:45:48.354 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.355 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.356 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.356 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.356 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:48.356 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.356 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.356 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-08-19T01:45:48.357 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.358 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.358 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.358 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.358 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:48.358 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.358 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.358 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-08-19T01:45:48.359 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.360 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.360 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.360 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.360 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:48.360 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.360 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.360 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.361 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:48.361 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.361 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.361 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.361 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.361 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-08-19T01:45:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-08-19T01:45:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.362 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.363 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.364 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.365 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.366 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.367 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.368 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:48.368 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.368 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.368 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.368 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.369 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:48.369 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.369 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.369 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.369 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.369 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.370 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-08-19T01:45:48.371 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.372 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.373 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.374 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-08-19T01:45:48.375 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 1533, 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:48.376 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:48.377 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:48.378 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:48.378 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:48.378 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:48.378 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:48.378 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:48.378 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:48.378 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:48.379 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:48.380 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 189383, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 50, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-08-19T01:45:48.381 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 153, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:48.382 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:48.383 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-08-19T01:45:48.383 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 536354, 2024-08-19T01:45:48.383 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 2685, 2024-08-19T01:45:48.383 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:48.383 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-08-19T01:45:48.383 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:48.383 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 2366, 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 26, 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 4, 2024-08-19T01:45:48.384 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 2366, 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2366, 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.169983645, 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000494498 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-08-19T01:45:48.385 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2366, 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 2367 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 2366, 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 2366, 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-08-19T01:45:48.386 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 1457, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 906, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-08-19T01:45:48.387 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 2, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:48.388 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:48.389 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-08-19T01:45:48.390 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-08-19T01:45:48.391 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 86, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 5, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 5, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-08-19T01:45:48.392 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-08-19T01:45:48.393 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:48.394 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:48.395 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 2602, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 8979652, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 2602, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:48.396 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2602, 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 8979652, 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.397 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:48.398 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:48.398 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:48.398 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:48.398 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:48.398 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:48.398 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:48.398 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 2366, 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 7970549, 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2366, 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 7970549, 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.399 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:48.400 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 2366, 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 2366, 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2366, 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 2366, 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.401 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:48.402 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-08-19T01:45:48.403 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55aced6b8b40": { 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:48.404 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-08-19T01:45:48.405 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-08-19T01:45:48.406 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:48.406 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:48.406 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:48.406 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:48.406 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.406 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:48.406 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-19T01:45:49.057 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-19T01:45:49.057 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-19T01:45:49.057 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-08-19T01:45:49.057 DEBUG:tasks.check_counter:Getting stats from h 2024-08-19T01:45:49.057 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-19T01:45:49.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:49 smithi146 ceph-mon[22098]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:45:49.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:49 smithi103 ceph-mon[18436]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:45:49.308 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:45:49.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:49 smithi190 ceph-mon[22032]: pgmap v640: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-19T01:45:50.512 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1661, 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1863, 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 1310134, 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 1345943, 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 10, 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-08-19T01:45:50.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.159235456, 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.070334155, 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.078268209, 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.014792303, 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1862, 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.075243880, 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000040410 2024-08-19T01:45:50.514 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1871, 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000426511, 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000227 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 607424, 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 138912 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.515 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 723, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1136, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 319974, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 908103, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 25, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.066946208, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.025397372, 2024-08-19T01:45:50.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.032235090, 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.008157209, 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1132, 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.039914875, 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000035260 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:50.517 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 619, 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000169705, 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000274 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 356336 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:50.518 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 892, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 939, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 632757, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 664924, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 24, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.064969590, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.021146275, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.039686575, 2024-08-19T01:45:50.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.007279006, 2024-08-19T01:45:50.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:50.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 940, 2024-08-19T01:45:50.520 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.048757810, 2024-08-19T01:45:50.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000051870 2024-08-19T01:45:50.520 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1007, 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000216635, 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000215 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 448 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-08-19T01:45:50.521 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-08-19T01:45:50.522 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-08-19T01:45:50.522 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.522 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-08-19T01:45:50.522 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:50.522 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:50.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1217, 2024-08-19T01:45:50.522 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.209876009, 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000172453 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-08-19T01:45:50.523 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000014157, 2024-08-19T01:45:50.524 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000007078 2024-08-19T01:45:50.524 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.524 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.524 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-08-19T01:45:50.524 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 115, 2024-08-19T01:45:50.524 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 114, 2024-08-19T01:45:50.524 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 114, 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.081342888, 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000713534 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 1, 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:50.525 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 11, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 254, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 28, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 226, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:50.526 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 214, 2024-08-19T01:45:50.527 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-08-19T01:45:50.527 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 180, 2024-08-19T01:45:50.527 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 180, 2024-08-19T01:45:50.527 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 7, 2024-08-19T01:45:50.527 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 168, 2024-08-19T01:45:50.527 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 157, 2024-08-19T01:45:50.527 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-08-19T01:45:50.528 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 3, 2024-08-19T01:45:50.528 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:50.528 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:50.528 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-08-19T01:45:50.528 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 451, 2024-08-19T01:45:50.528 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-08-19T01:45:50.528 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 7, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 238, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 1065, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 325, 2024-08-19T01:45:50.529 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 13424230473, 2024-08-19T01:45:50.530 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-08-19T01:45:50.530 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 1065, 2024-08-19T01:45:50.530 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:50.530 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:50.530 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:50.531 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 140, 2024-08-19T01:45:50.531 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:50.531 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 140, 2024-08-19T01:45:50.531 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 925, 2024-08-19T01:45:50.531 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 8, 2024-08-19T01:45:50.531 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 8, 2024-08-19T01:45:50.531 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 83, 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 112, 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 97, 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 2 2024-08-19T01:45:50.532 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 3, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 2, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 17, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 6, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:50.533 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:50.534 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-08-19T01:45:50.534 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-08-19T01:45:50.534 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:50.534 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-08-19T01:45:50.534 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:50.534 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:50.534 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-08-19T01:45:50.535 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 317, 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-08-19T01:45:50.536 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 317, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 2, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-08-19T01:45:50.537 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-08-19T01:45:50.538 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-08-19T01:45:50.538 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-08-19T01:45:50.538 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-08-19T01:45:50.538 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4702883, 2024-08-19T01:45:50.538 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-08-19T01:45:50.538 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-08-19T01:45:50.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 189, 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.478826957, 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002533475 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 197, 2024-08-19T01:45:50.539 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 197, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 0, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 40, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 41, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 1, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 254, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 254, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 0, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 180, 2024-08-19T01:45:50.540 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 181, 2024-08-19T01:45:50.541 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 1, 2024-08-19T01:45:50.541 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 53628, 2024-08-19T01:45:50.541 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 223520 2024-08-19T01:45:50.541 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.541 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-08-19T01:45:50.541 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 139, 2024-08-19T01:45:50.541 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 4, 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 115, 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 13, 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 16, 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1, 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.005362360, 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005362360 2024-08-19T01:45:50.542 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1, 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000168458, 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000168458 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.543 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-08-19T01:45:50.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 47, 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.033467917, 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000712083 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.545 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.546 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-08-19T01:45:50.546 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.546 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.546 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.546 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.546 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-08-19T01:45:50.546 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.547 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.548 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.549 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.549 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-08-19T01:45:50.549 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.549 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.549 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.549 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.550 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.551 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.551 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-08-19T01:45:50.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 65, 2024-08-19T01:45:50.551 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.042344153, 2024-08-19T01:45:50.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000651448 2024-08-19T01:45:50.551 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.552 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:50.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.554 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.555 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:50.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.557 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.558 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.559 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:50.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.561 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.561 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.561 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.561 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-08-19T01:45:50.561 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-08-19T01:45:50.561 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 66, 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 66, 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 631, 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:50.562 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:50.563 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:50.564 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:50.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:50.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 304646, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 110, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 153, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 725760, 2024-08-19T01:45:50.568 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 4099, 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-08-19T01:45:50.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 1373, 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 538022, 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 28, 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 1373, 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1373, 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.303981725, 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000949731 2024-08-19T01:45:50.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1373, 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1470 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 1373, 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 1065, 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 308, 2024-08-19T01:45:50.571 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 10, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 0, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 189, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 30, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-08-19T01:45:50.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 6, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1065, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 20, 2024-08-19T01:45:50.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-08-19T01:45:50.574 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-08-19T01:45:50.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 86, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-08-19T01:45:50.576 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 12, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 117, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 0, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 33 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.577 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-08-19T01:45:50.578 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-08-19T01:45:50.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 3272, 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1951444, 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:50.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 3272, 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 3272, 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1951444, 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.581 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:50.582 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 1373, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 538022, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1373, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 538022, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.583 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:50.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 1373, 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 1373, 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1373, 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1373, 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:50.585 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:50.586 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-08-19T01:45:50.587 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55b9e22d4960": { 2024-08-19T01:45:50.588 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 574, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 317, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 508579, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 317, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:50.589 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:50.590 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 189, 2024-08-19T01:45:50.590 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 508005, 2024-08-19T01:45:50.590 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:50.590 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:50.590 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:50.590 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:50.590 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.591 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:50.591 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-19T01:45:51.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:51 smithi103 ceph-mon[18436]: pgmap v641: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-08-19T01:45:51.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:51 smithi146 ceph-mon[22098]: pgmap v641: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-08-19T01:45:51.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:51 smithi190 ceph-mon[22032]: pgmap v641: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-08-19T01:45:51.682 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-08-19T01:45:51.682 INFO:tasks.check_counter:Daemon mds.h mds.imported=7 2024-08-19T01:45:51.682 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-08-19T01:45:51.682 DEBUG:tasks.check_counter:Getting stats from k 2024-08-19T01:45:51.682 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-19T01:45:51.940 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.b/config 2024-08-19T01:45:52.477 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-19T01:45:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 238, 2024-08-19T01:45:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 220, 2024-08-19T01:45:52.477 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 595458, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 131972, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 1, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.033353732, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.014690255, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.022265453, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000017206, 2024-08-19T01:45:52.478 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 219, 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.013611420, 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000062152 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.479 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 594624, 2024-08-19T01:45:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 131648 2024-08-19T01:45:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 658, 2024-08-19T01:45:52.480 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 816, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 163764, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 561403, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.066941813, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.024178001, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.015228911, 2024-08-19T01:45:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.020614208, 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 815, 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.038488038, 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000047224 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.482 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 353744 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1538, 2024-08-19T01:45:52.483 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1540, 2024-08-19T01:45:52.484 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 768373, 2024-08-19T01:45:52.484 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 489142, 2024-08-19T01:45:52.484 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:52.484 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.140098783, 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.049728166, 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.036193449, 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.041921647, 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1538, 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.059445378, 2024-08-19T01:45:52.485 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000038651 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:52.486 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:52.487 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3639, 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.091657471, 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000025187 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-08-19T01:45:52.488 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-08-19T01:45:52.489 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.490 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.490 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.490 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.490 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-08-19T01:45:52.490 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-08-19T01:45:52.490 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:52.490 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 156, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 45, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 111, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 92, 2024-08-19T01:45:52.491 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 5, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-08-19T01:45:52.492 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-08-19T01:45:52.493 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-08-19T01:45:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:45:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 0, 2024-08-19T01:45:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 0, 2024-08-19T01:45:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-08-19T01:45:52.494 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-08-19T01:45:52.495 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:45:52.496 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-08-19T01:45:52.497 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-08-19T01:45:52.498 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-08-19T01:45:52.498 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:52.498 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:52.498 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-08-19T01:45:52.498 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-08-19T01:45:52.498 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:52.498 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:52.499 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-08-19T01:45:52.500 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 179, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-08-19T01:45:52.501 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4506108, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 4505534, 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-08-19T01:45:52.502 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 179 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 158, 2024-08-19T01:45:52.503 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 158, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 0, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 73, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 73, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 0, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 156, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 156, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 0, 2024-08-19T01:45:52.504 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 49588, 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215328 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-08-19T01:45:52.505 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.506 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.507 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.508 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.508 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.508 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-08-19T01:45:52.508 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.508 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.508 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.508 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.509 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-08-19T01:45:52.509 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.509 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.509 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.509 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.509 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:52.510 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.510 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.510 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.510 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.510 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-08-19T01:45:52.510 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.510 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.511 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.511 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.511 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-08-19T01:45:52.511 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.511 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.511 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.512 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.512 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:52.512 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.512 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.512 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.512 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.512 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.513 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.514 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.514 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-08-19T01:45:52.514 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.515 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.515 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.515 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.515 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-08-19T01:45:52.515 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.515 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-08-19T01:45:52.516 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.517 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.518 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-08-19T01:45:52.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.518 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.518 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.518 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:52.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.519 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.521 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.522 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.522 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:52.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.522 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.522 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.523 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.525 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.526 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.527 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-08-19T01:45:52.527 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-08-19T01:45:52.527 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-08-19T01:45:52.527 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-08-19T01:45:52.527 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-08-19T01:45:52.527 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-08-19T01:45:52.527 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 1540, 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:52.528 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:52.529 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:52.529 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:52.529 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:52.529 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:52.529 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:52.529 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:52.529 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:52.530 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:52.530 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:52.530 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:52.530 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:52.530 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:52.530 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:52.530 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:52.531 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:52.532 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:52.532 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:52.532 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:52.532 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:52.532 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:52.532 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:52.533 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-08-19T01:45:52.533 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:52.533 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:52.533 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:52.533 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:52.533 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 189383, 2024-08-19T01:45:52.533 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 50, 2024-08-19T01:45:52.534 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:52.534 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:52.534 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-08-19T01:45:52.534 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-08-19T01:45:52.534 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:52.534 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:52.534 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:52.535 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:52.535 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:52.535 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 153, 2024-08-19T01:45:52.535 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:52.535 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:52.535 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:52.535 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-08-19T01:45:52.536 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 546507, 2024-08-19T01:45:52.536 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 9861, 2024-08-19T01:45:52.536 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:52.536 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-08-19T01:45:52.536 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:52.536 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-08-19T01:45:52.536 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.537 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-08-19T01:45:52.537 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-08-19T01:45:52.537 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-08-19T01:45:52.537 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 2192, 2024-08-19T01:45:52.537 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 21, 2024-08-19T01:45:52.537 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 3, 2024-08-19T01:45:52.537 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 2192, 2024-08-19T01:45:52.538 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-08-19T01:45:52.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2192, 2024-08-19T01:45:52.538 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.108984471, 2024-08-19T01:45:52.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000505923 2024-08-19T01:45:52.538 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.538 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-08-19T01:45:52.539 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-08-19T01:45:52.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2192, 2024-08-19T01:45:52.539 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 2193 2024-08-19T01:45:52.541 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.542 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 2192, 2024-08-19T01:45:52.542 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 2192, 2024-08-19T01:45:52.542 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-08-19T01:45:52.542 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-08-19T01:45:52.542 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-08-19T01:45:52.542 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 1388, 2024-08-19T01:45:52.542 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 802, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-08-19T01:45:52.543 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:52.544 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-08-19T01:45:52.545 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-08-19T01:45:52.546 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 86, 2024-08-19T01:45:52.547 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 5, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 5, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-08-19T01:45:52.548 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-08-19T01:45:52.549 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:52.550 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 2430, 2024-08-19T01:45:52.551 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1293838, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 2430, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2430, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1293838, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.552 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:52.553 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 2192, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 311809, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2192, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 311809, 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:52.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-08-19T01:45:52.555 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 2192, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 2192, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2192, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 2192, 2024-08-19T01:45:52.556 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:52.557 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-08-19T01:45:52.558 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x561931732b40": { 2024-08-19T01:45:52.559 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-08-19T01:45:52.560 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-08-19T01:45:52.561 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-08-19T01:45:52.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-08-19T01:45:52.561 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-08-19T01:45:52.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-08-19T01:45:52.561 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.561 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-19T01:45:52.561 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-19T01:45:53.200 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:53 smithi146 ceph-mon[22098]: pgmap v642: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-19T01:45:53.300 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-08-19T01:45:53.300 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-08-19T01:45:53.300 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-08-19T01:45:53.301 DEBUG:tasks.check_counter:Getting stats from c 2024-08-19T01:45:53.301 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-19T01:45:53.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:53 smithi103 ceph-mon[18436]: pgmap v642: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-19T01:45:53.329 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:53 smithi190 ceph-mon[22032]: pgmap v642: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-19T01:45:53.566 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:45:54.099 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-19T01:45:54.099 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:54.099 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1135, 2024-08-19T01:45:54.099 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1139, 2024-08-19T01:45:54.099 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 571056, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 877132, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 11, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.101315542, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.037105139, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.049984096, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.013057875, 2024-08-19T01:45:54.100 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:54.101 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1139, 2024-08-19T01:45:54.101 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.073288192, 2024-08-19T01:45:54.101 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000064344 2024-08-19T01:45:54.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.101 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:54.101 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1404, 2024-08-19T01:45:54.101 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000472629, 2024-08-19T01:45:54.102 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000336 2024-08-19T01:45:54.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.102 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:54.102 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:54.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.102 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:54.102 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 845, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 948, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 743688, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 806539, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 11, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.065862060, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.028366075, 2024-08-19T01:45:54.103 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.032730281, 2024-08-19T01:45:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.007268708, 2024-08-19T01:45:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 948, 2024-08-19T01:45:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.047545269, 2024-08-19T01:45:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000050153 2024-08-19T01:45:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.104 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 765, 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000177920, 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000232 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 355296 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.105 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:54.106 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 851, 2024-08-19T01:45:54.106 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1026, 2024-08-19T01:45:54.106 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 909297, 2024-08-19T01:45:54.106 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 765533, 2024-08-19T01:45:54.106 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 37, 2024-08-19T01:45:54.106 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 7, 2024-08-19T01:45:54.106 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.083289952, 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.028823307, 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.056659596, 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.005449707, 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1022, 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.037034411, 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000036237 2024-08-19T01:45:54.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 502, 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000145282, 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000289 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 624304, 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 143008 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.108 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 866, 2024-08-19T01:45:54.109 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.157906883, 2024-08-19T01:45:54.110 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000182340 2024-08-19T01:45:54.110 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.110 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:54.110 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:54.110 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000012715, 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000006357 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 86, 2024-08-19T01:45:54.111 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 86, 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 86, 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.056654842, 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000658777 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:54.112 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:54.113 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-08-19T01:45:54.113 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-08-19T01:45:54.113 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-08-19T01:45:54.113 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 166, 2024-08-19T01:45:54.113 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-08-19T01:45:54.113 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 166, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 165, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 147, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 147, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 7, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 130, 2024-08-19T01:45:54.114 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 122, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 8, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 1193, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 1, 2024-08-19T01:45:54.115 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 3, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 145, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 990, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 325, 2024-08-19T01:45:54.116 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 13424230473, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 990, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 472, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 472, 2024-08-19T01:45:54.117 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 518, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 12, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 6, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 58, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 84, 2024-08-19T01:45:54.118 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 73, 2024-08-19T01:45:54.119 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:54.119 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-08-19T01:45:54.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.119 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-08-19T01:45:54.119 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-08-19T01:45:54.119 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 3, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 2, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 12, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 1, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-08-19T01:45:54.120 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:54.121 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:54.122 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:54.122 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-08-19T01:45:54.122 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:54.122 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:54.122 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 179, 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 179, 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-08-19T01:45:54.123 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 2, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-08-19T01:45:54.124 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 4506108, 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 87, 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.178452022, 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002051172 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.125 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 168, 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 168, 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 0, 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 83, 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 84, 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 1, 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 166, 2024-08-19T01:45:54.126 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 166, 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 0, 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 147, 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 147, 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 52964, 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 223520 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.127 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 98, 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 86, 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 13, 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 4, 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.128 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000598632, 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000598632 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.129 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:54.130 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.130 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.130 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.130 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-08-19T01:45:54.130 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.131 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.131 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.131 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-08-19T01:45:54.131 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 35, 2024-08-19T01:45:54.131 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.025667063, 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000733344 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.132 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-08-19T01:45:54.133 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.133 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.133 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.133 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-08-19T01:45:54.133 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.133 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.134 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.136 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-08-19T01:45:54.137 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 50, 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.030389147, 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000607782 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.138 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.139 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-08-19T01:45:54.140 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.142 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:54.142 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.142 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.142 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.142 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-08-19T01:45:54.142 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.143 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.145 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.146 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-08-19T01:45:54.147 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 53, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 53, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 633, 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:54.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:54.149 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:54.150 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:54.150 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:54.150 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:54.150 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:54.150 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:54.150 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:54.150 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:54.151 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:54.151 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:54.151 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:54.151 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:54.151 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:54.151 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:54.151 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:54.152 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 291126, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 117, 2024-08-19T01:45:54.153 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:54.154 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 153, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 657123, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 11722, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:54.155 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 1179, 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 339802, 2024-08-19T01:45:54.156 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 1179, 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1179, 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.851975697, 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000722625 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1179, 2024-08-19T01:45:54.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1264 2024-08-19T01:45:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 1179, 2024-08-19T01:45:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 990, 2024-08-19T01:45:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 189, 2024-08-19T01:45:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-08-19T01:45:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-08-19T01:45:54.158 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 87, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 17, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 6, 2024-08-19T01:45:54.159 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 990, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-08-19T01:45:54.160 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-08-19T01:45:54.161 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-08-19T01:45:54.162 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 86, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-08-19T01:45:54.163 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 104, 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 30 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-08-19T01:45:54.164 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:54.165 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:54.166 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2827, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1952211, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2827, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2827, 2024-08-19T01:45:54.167 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1952211, 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:54.168 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 1179, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 339802, 2024-08-19T01:45:54.169 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1179, 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 339802, 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.170 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:54.171 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:54.171 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-08-19T01:45:54.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:54.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:54.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:54.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:54.171 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 1179, 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 1179, 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1179, 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1179, 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.172 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:54.173 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:54.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55a13e536960": { 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:54.175 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:54.177 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 179, 2024-08-19T01:45:54.177 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 311804, 2024-08-19T01:45:54.177 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:54.178 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 179, 2024-08-19T01:45:54.178 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:54.178 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:54.178 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 87, 2024-08-19T01:45:54.178 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 311804, 2024-08-19T01:45:54.178 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:54.178 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:54.179 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:54.179 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:54.179 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.179 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:54.179 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-19T01:45:54.898 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-19T01:45:54.899 INFO:tasks.check_counter:Daemon mds.c mds.imported=3 2024-08-19T01:45:54.899 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-08-19T01:45:54.899 DEBUG:tasks.check_counter:Getting stats from f 2024-08-19T01:45:54.899 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-19T01:45:55.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:55 smithi190 ceph-mon[22032]: pgmap v643: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-19T01:45:55.212 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:45:55.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:55 smithi103 ceph-mon[18436]: pgmap v643: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-19T01:45:55.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:55 smithi146 ceph-mon[22098]: pgmap v643: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-19T01:45:55.778 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-19T01:45:55.779 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:55.779 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 547, 2024-08-19T01:45:55.779 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 668, 2024-08-19T01:45:55.779 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 243261, 2024-08-19T01:45:55.779 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 753124, 2024-08-19T01:45:55.779 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 16, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.051831753, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.017515296, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.030860542, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.004575316, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 666, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.040373463, 2024-08-19T01:45:55.780 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000060620 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 788, 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000184640, 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000234 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:55.781 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 491, 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 433, 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 760366, 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 290690, 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 9, 2024-08-19T01:45:55.782 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.053274979, 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.020677718, 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.037023067, 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.003035091, 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 433, 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.039614598, 2024-08-19T01:45:55.783 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000091488 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 182, 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000040036, 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000219 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 615568, 2024-08-19T01:45:55.784 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 136320 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 931, 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1053, 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 483864, 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 1117507, 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 33, 2024-08-19T01:45:55.785 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 9, 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.103583808, 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.042198884, 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.061560732, 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.004768552, 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1051, 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.057095694, 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000054325 2024-08-19T01:45:55.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1716, 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000420905, 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000245 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 356480 2024-08-19T01:45:55.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:55.788 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 398, 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.049630753, 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000124700 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:55.789 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000012656, 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000006328 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 145, 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 144, 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-08-19T01:45:55.790 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 144, 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.107856825, 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000749005 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 1, 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:55.791 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 11, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 50, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 50, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 49, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-08-19T01:45:55.792 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 34, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 34, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 6, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 220, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 195, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 5, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:55.793 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 1625, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 1, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 2, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 28, 2024-08-19T01:45:55.794 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 59, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 325, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 13424230473, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 59, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:55.795 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:55.796 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:55.796 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:45:55.796 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:55.796 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:45:55.796 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 59, 2024-08-19T01:45:55.796 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:45:55.796 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 4, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 101, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 153, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 137, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:55.797 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 4 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 3, 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 2, 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 8, 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-08-19T01:45:55.798 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:55.799 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:55.800 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 274, 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-08-19T01:45:55.801 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 274, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 2, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-08-19T01:45:55.802 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 4530818, 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 132, 2024-08-19T01:45:55.803 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.264042650, 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002000323 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 53, 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 53, 2024-08-19T01:45:55.804 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 0, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 22, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 23, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 1, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 50, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 50, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 0, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 34, 2024-08-19T01:45:55.805 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 34, 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 54076, 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 223520 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 167, 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-08-19T01:45:55.806 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 145, 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 9, 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 8, 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3, 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.008163858, 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002721286 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.807 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000870626, 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000870626 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 64, 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.045636514, 2024-08-19T01:45:55.809 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000713070 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-08-19T01:45:55.810 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-08-19T01:45:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-08-19T01:45:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-08-19T01:45:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 76, 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.053185827, 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000699813 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-08-19T01:45:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 79, 2024-08-19T01:45:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 79, 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 636, 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 316012, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 134, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 153, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 193829, 2024-08-19T01:45:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 1144, 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-08-19T01:45:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 298, 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 357762, 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 27, 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 298, 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 298, 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.868975204, 2024-08-19T01:45:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002916024 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 298, 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 400 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 298, 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 59, 2024-08-19T01:45:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 239, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 132, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 16, 2024-08-19T01:45:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 6, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 59, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-08-19T01:45:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-08-19T01:45:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-08-19T01:45:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 86, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-08-19T01:45:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 117, 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 40 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-08-19T01:45:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-08-19T01:45:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1965, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1294238, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1965, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1965, 2024-08-19T01:45:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1294238, 2024-08-19T01:45:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 298, 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 357762, 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 298, 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 357762, 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 298, 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 298, 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 298, 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 298, 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-19T01:45:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x563bba386960": { 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 274, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 336514, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 274, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 132, 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 336514, 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:55.859 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-19T01:45:56.576 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-19T01:45:56.577 INFO:tasks.check_counter:Daemon mds.f mds.imported=2 2024-08-19T01:45:56.577 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-08-19T01:45:56.577 DEBUG:tasks.check_counter:Getting stats from i 2024-08-19T01:45:56.577 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-19T01:45:56.826 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:45:57.251 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:57 smithi190 ceph-mon[22032]: pgmap v644: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:45:57.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:57 smithi103 ceph-mon[18436]: pgmap v644: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:45:57.368 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-19T01:45:57.368 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:57.368 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 3149, 2024-08-19T01:45:57.368 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 4868, 2024-08-19T01:45:57.368 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 972077, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 4906240, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 23, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 9, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.314545503, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.122148630, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.169331222, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.038832562, 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:57.369 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4859, 2024-08-19T01:45:57.370 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.127018288, 2024-08-19T01:45:57.370 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000026140 2024-08-19T01:45:57.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.370 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:57.370 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4725, 2024-08-19T01:45:57.370 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001177124, 2024-08-19T01:45:57.370 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000249 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3872, 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 1184 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 2115, 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 2190, 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 1619761, 2024-08-19T01:45:57.371 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 1414778, 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 8, 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.174539799, 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.072944984, 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.109359849, 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.009388602, 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:57.372 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2195, 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.103751974, 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000047267 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3201, 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000766976, 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000239 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.373 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 624720, 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 145216 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 737, 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 930, 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 315053, 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 1068661, 2024-08-19T01:45:57.374 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 28, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 9, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.081490258, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.033368464, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.029321196, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.013919178, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 930, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.048579623, 2024-08-19T01:45:57.375 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000052236 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 424, 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000111426, 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000262 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 358560 2024-08-19T01:45:57.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3148, 2024-08-19T01:45:57.377 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.531014346, 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000168683 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-08-19T01:45:57.378 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000013195, 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000006597 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 354, 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 273, 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 273, 2024-08-19T01:45:57.379 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.251631304, 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000921726 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 81, 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 19, 2024-08-19T01:45:57.380 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 1, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 44, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 25, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 19, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 43, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 307, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 28, 2024-08-19T01:45:57.381 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 28, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 11, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 464, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 308, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 81, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 7, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-08-19T01:45:57.382 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 66, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 12, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 307, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 5, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 209, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 796, 2024-08-19T01:45:57.383 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 325, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 13424230473, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 796, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 119, 2024-08-19T01:45:57.384 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 119, 2024-08-19T01:45:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 677, 2024-08-19T01:45:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 9, 2024-08-19T01:45:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 26, 2024-08-19T01:45:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 1144, 2024-08-19T01:45:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 1214, 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 1162, 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 122 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-08-19T01:45:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 3, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 2, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 9, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 17, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:57.387 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-08-19T01:45:57.388 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 12, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 1, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:57.389 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 1456, 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 1456, 2024-08-19T01:45:57.390 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 3, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 3, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-08-19T01:45:57.391 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 7200526, 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1111, 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2.264540677, 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002038290 2024-08-19T01:45:57.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 47, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 333, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 286, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 30, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 108, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 78, 2024-08-19T01:45:57.393 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 44, 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 351, 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 307, 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 28, 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 314, 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 286, 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 53156, 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 223520 2024-08-19T01:45:57.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.395 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-08-19T01:45:57.395 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 379, 2024-08-19T01:45:57.395 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 20, 2024-08-19T01:45:57.395 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 354, 2024-08-19T01:45:57.395 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 25, 2024-08-19T01:45:57.395 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 45, 2024-08-19T01:45:57.395 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 120, 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.140404741, 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001170039 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2, 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001992542, 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000996271 2024-08-19T01:45:57.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.397 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 89, 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.058007646, 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000651771 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.398 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:57.399 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.399 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.399 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.399 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-08-19T01:45:57.399 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.399 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:57.400 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.403 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.403 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-08-19T01:45:57.403 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.403 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.403 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 62, 2024-08-19T01:45:57.404 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.051226375, 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000826231 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.405 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.406 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-08-19T01:45:57.407 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.409 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:57.410 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.411 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.411 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.411 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-08-19T01:45:57.411 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.411 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.411 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-08-19T01:45:57.412 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.414 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:57.414 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:57.414 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:57.414 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.414 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.414 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.414 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-08-19T01:45:57.415 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 81, 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 81, 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 639, 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:57.416 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:57.417 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:57.418 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:57.419 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:57.420 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:57.421 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 282643, 2024-08-19T01:45:57.421 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 113, 2024-08-19T01:45:57.421 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:57.421 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:57.421 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-19T01:45:57.421 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-19T01:45:57.421 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:57.422 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:57.422 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:57.422 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:57.422 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:57.422 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 153, 2024-08-19T01:45:57.422 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:57.422 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 186309, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 1062, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-19T01:45:57.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 2294, 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 3118396, 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 2294, 2024-08-19T01:45:57.424 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2294, 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3.452901349, 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001505188 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2294, 2024-08-19T01:45:57.425 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2646 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 2294, 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 796, 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 1498, 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-08-19T01:45:57.426 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 0, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 1111, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 43, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 1, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 6, 2024-08-19T01:45:57.427 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 796, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:57.428 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-08-19T01:45:57.429 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-08-19T01:45:57.430 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 86, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-08-19T01:45:57.431 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 507, 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 152 2024-08-19T01:45:57.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-08-19T01:45:57.433 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-08-19T01:45:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-08-19T01:45:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-08-19T01:45:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:57.434 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 5997, 2024-08-19T01:45:57.435 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 2349587, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 5997, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 5997, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2349587, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.436 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:57.437 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:57.438 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:57.438 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:57.438 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2294, 2024-08-19T01:45:57.438 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3118396, 2024-08-19T01:45:57.438 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2294, 2024-08-19T01:45:57.438 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3118396, 2024-08-19T01:45:57.438 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-08-19T01:45:57.439 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2294, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2294, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2294, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2294, 2024-08-19T01:45:57.440 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:57.441 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-19T01:45:57.442 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x5575d7608960": { 2024-08-19T01:45:57.443 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1456, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 3006222, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1456, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:57.444 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1111, 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3006222, 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.445 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:57.446 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-19T01:45:57.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:57 smithi146 ceph-mon[22098]: pgmap v644: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:45:58.209 INFO:tasks.check_counter:Daemon mds.i mds.exported=12 2024-08-19T01:45:58.210 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-08-19T01:45:58.210 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-08-19T01:45:58.210 DEBUG:tasks.check_counter:Getting stats from l 2024-08-19T01:45:58.210 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-19T01:45:58.459 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.c/config 2024-08-19T01:45:59.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:45:59 smithi190 ceph-mon[22032]: pgmap v645: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:45:59.146 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-08-19T01:45:59.147 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-08-19T01:45:59.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 986, 2024-08-19T01:45:59.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 966, 2024-08-19T01:45:59.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 847760, 2024-08-19T01:45:59.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 369650, 2024-08-19T01:45:59.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-08-19T01:45:59.147 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 1, 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.121528418, 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.042142523, 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.046698508, 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.028465584, 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 966, 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.058789385, 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000060858 2024-08-19T01:45:59.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 598112, 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 133376 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.149 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-08-19T01:45:59.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 3, 2024-08-19T01:45:59.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 165, 2024-08-19T01:45:59.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 716, 2024-08-19T01:45:59.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 355962, 2024-08-19T01:45:59.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:59.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:59.150 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.009719707, 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.006826883, 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.001237572, 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000059385, 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 163, 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.010142301, 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000062222 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.151 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 355264 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.152 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1874, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1876, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3479759, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 595939, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.123849890, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.040879500, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.033221156, 2024-08-19T01:45:59.153 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.040966561, 2024-08-19T01:45:59.154 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-08-19T01:45:59.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1874, 2024-08-19T01:45:59.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.057520961, 2024-08-19T01:45:59.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000030694 2024-08-19T01:45:59.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.154 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-08-19T01:45:59.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-08-19T01:45:59.155 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4573, 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.109605685, 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000023968 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.157 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-08-19T01:45:59.158 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 29, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 29, 2024-08-19T01:45:59.159 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 28, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 10, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-08-19T01:45:59.160 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-08-19T01:45:59.161 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-08-19T01:45:59.162 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-08-19T01:45:59.163 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.164 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-08-19T01:45:59.165 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-08-19T01:45:59.166 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.167 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 1456, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-08-19T01:45:59.168 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 3, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 2, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 7200526, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 7199952, 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-08-19T01:45:59.169 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 1456 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 32, 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 323, 2024-08-19T01:45:59.170 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 291, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 18, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 97, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 79, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 29, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 341, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 312, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-08-19T01:45:59.171 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 50332, 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 215328 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-08-19T01:45:59.172 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-08-19T01:45:59.173 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-08-19T01:45:59.176 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.177 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-08-19T01:45:59.178 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.178 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.178 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.178 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-08-19T01:45:59.178 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.178 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-08-19T01:45:59.179 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.181 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-08-19T01:45:59.181 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.181 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.181 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.181 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-08-19T01:45:59.181 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.182 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.183 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.184 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.185 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.185 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-08-19T01:45:59.185 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.185 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.185 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.186 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.187 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-08-19T01:45:59.187 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.187 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.187 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.187 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.189 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-08-19T01:45:59.190 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-08-19T01:45:59.191 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 1549, 2024-08-19T01:45:59.192 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-08-19T01:45:59.193 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-08-19T01:45:59.194 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-08-19T01:45:59.195 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-08-19T01:45:59.196 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 181360, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 48, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-08-19T01:45:59.197 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7408, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 153, 2024-08-19T01:45:59.198 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 112997, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 351, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-08-19T01:45:59.199 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 2619, 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 21, 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 3, 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 2619, 2024-08-19T01:45:59.200 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2619, 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.226964943, 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000468486 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2619, 2024-08-19T01:45:59.201 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2620 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 2619, 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 2619, 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 1400, 2024-08-19T01:45:59.202 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1217, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-08-19T01:45:59.203 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-08-19T01:45:59.204 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-08-19T01:45:59.205 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-08-19T01:45:59.206 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 86, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-08-19T01:45:59.207 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 2, 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 4, 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 4, 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-08-19T01:45:59.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.209 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-08-19T01:45:59.209 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2024-08-19T01:45:59.209 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-08-19T01:45:59.209 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-08-19T01:45:59.209 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-08-19T01:45:59.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-08-19T01:45:59.210 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:59.211 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2859, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 4055764, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2859, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2859, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4055764, 2024-08-19T01:45:59.212 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2619, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3006227, 2024-08-19T01:45:59.214 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2619, 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3006227, 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.215 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:59.216 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2619, 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2619, 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2619, 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2619, 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.217 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:59.218 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.219 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55ee91af8b40": { 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-08-19T01:45:59.220 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-08-19T01:45:59.221 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-08-19T01:45:59.222 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-08-19T01:45:59.222 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-08-19T01:45:59.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-08-19T01:45:59.222 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-08-19T01:45:59.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:45:59 smithi103 ceph-mon[18436]: pgmap v645: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:45:59.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:45:59 smithi146 ceph-mon[22098]: pgmap v645: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-19T01:45:59.905 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-08-19T01:45:59.905 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-08-19T01:45:59.905 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-08-19T01:45:59.906 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-08-19T01:45:59.916 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-08-19T01:45:59.917 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:45:59.917 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-19T01:45:59.940 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:45:59.940 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-19T01:46:00.000 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-19T01:46:00.306 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:00.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 -- 172.21.15.103:0/2978222359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce410bea0 msgr2=0x7f8ce410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:00.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/2978222359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce410bea0 0x7f8ce410e340 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009ca0 tx=0x7f8cdc02a950 comp rx=0 tx=0).stop 2024-08-19T01:46:00.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 -- 172.21.15.103:0/2978222359 shutdown_connections 2024-08-19T01:46:00.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/2978222359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce410bea0 0x7f8ce410e340 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/2978222359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce4103390 0x7f8ce410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/2978222359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ce4102a70 0x7f8ce4102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 -- 172.21.15.103:0/2978222359 >> 172.21.15.103:0/2978222359 conn(0x7f8ce40fc4d0 msgr2=0x7f8ce40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 -- 172.21.15.103:0/2978222359 shutdown_connections 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.809+0000 7f8cea1e1640 1 -- 172.21.15.103:0/2978222359 wait complete. 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 Processor -- start 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 -- start start 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ce4102a70 0x7f8ce4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:00.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce4103390 0x7f8ce4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce410bea0 0x7f8ce419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4111b90 con 0x7f8ce4103390 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4111a10 con 0x7f8ce4102a70 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8cea1e1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4111d10 con 0x7f8ce410bea0 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.810+0000 7f8ce2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce4103390 0x7f8ce4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce4103390 0x7f8ce4198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40064/0 (socket says 172.21.15.103:40064) 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 -- 172.21.15.103:0/274540024 learned_addr learned my addr 172.21.15.103:0/274540024 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:00.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce37fe640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ce4102a70 0x7f8ce4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce3fff640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce410bea0 0x7f8ce419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 -- 172.21.15.103:0/274540024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce410bea0 msgr2=0x7f8ce419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce410bea0 0x7f8ce419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 -- 172.21.15.103:0/274540024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ce4102a70 msgr2=0x7f8ce4198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ce4102a70 0x7f8ce4198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 -- 172.21.15.103:0/274540024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce419d0d0 con 0x7f8ce4103390 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce3fff640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce410bea0 0x7f8ce419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:46:00.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce2ffd640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce4103390 0x7f8ce4198660 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800b3f0 tx=0x7f8cd800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce0ff9640 1 -- 172.21.15.103:0/274540024 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8015020 con 0x7f8ce4103390 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce419d3c0 con 0x7f8ce4103390 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce0ff9640 1 -- 172.21.15.103:0/274540024 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cd8011ce0 con 0x7f8ce4103390 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce41a4ca0 con 0x7f8ce4103390 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.811+0000 7f8ce0ff9640 1 -- 172.21.15.103:0/274540024 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8010d80 con 0x7f8ce4103390 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.812+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce4102e50 con 0x7f8ce4103390 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.813+0000 7f8ce0ff9640 1 -- 172.21.15.103:0/274540024 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8cd8018020 con 0x7f8ce4103390 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.814+0000 7f8ce0ff9640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ccc0a2b80 0x7f8ccc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:00.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.814+0000 7f8ce37fe640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ccc0a2b80 0x7f8ccc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:00.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.814+0000 7f8ce0ff9640 1 -- 172.21.15.103:0/274540024 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f8cd80ceeb0 con 0x7f8ce4103390 2024-08-19T01:46:00.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.814+0000 7f8ce37fe640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ccc0a2b80 0x7f8ccc0a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40058c0 tx=0x7f8cd400a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:00.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.817+0000 7f8ce0ff9640 1 -- 172.21.15.103:0/274540024 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8cd80fddf0 con 0x7f8ce4103390 2024-08-19T01:46:00.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.996+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8ce4106f70 con 0x7f8ce4103390 2024-08-19T01:46:00.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.997+0000 7f8ce0ff9640 1 -- 172.21.15.103:0/274540024 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+600 (secure 0 0 0) 0x7f8cd801f020 con 0x7f8ce4103390 2024-08-19T01:46:00.999 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2698864603 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:00.999 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3003368150 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:00.999 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2881014674 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:01.000 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2881014674 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:01.000 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2169969902 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:01.000 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/3969074886 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:01.000 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/975765434 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:01.000 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/3969074886 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:01.000 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3082166873 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:01.000 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/948089109 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:01.001 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-08-19T01:46:01.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ccc0a2b80 msgr2=0x7f8ccc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:01.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ccc0a2b80 0x7f8ccc0a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40058c0 tx=0x7f8cd400a4c0 comp rx=0 tx=0).stop 2024-08-19T01:46:01.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce4103390 msgr2=0x7f8ce4198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce4103390 0x7f8ce4198660 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800b3f0 tx=0x7f8cd800b8b0 comp rx=0 tx=0).stop 2024-08-19T01:46:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 shutdown_connections 2024-08-19T01:46:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f8ccc0a2b80 0x7f8ccc0a5040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ce410bea0 0x7f8ce419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ce4103390 0x7f8ce4198660 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 --2- 172.21.15.103:0/274540024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ce4102a70 0x7f8ce4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:01.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 >> 172.21.15.103:0/274540024 conn(0x7f8ce40fc4d0 msgr2=0x7f8ce40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 shutdown_connections 2024-08-19T01:46:01.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:00.999+0000 7f8cea1e1640 1 -- 172.21.15.103:0/274540024 wait complete. 2024-08-19T01:46:01.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:01 smithi103 ceph-mon[18436]: pgmap v646: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-08-19T01:46:01.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:01 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/274540024' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:46:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:01 smithi190 ceph-mon[22032]: pgmap v646: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-08-19T01:46:01.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:01 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/274540024' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:46:01.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:01 smithi146 ceph-mon[22098]: pgmap v646: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-08-19T01:46:01.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:01 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/274540024' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:46:01.865 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-08-19T01:46:01.865 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-19T01:46:01.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-19T01:46:02.180 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 -- 172.21.15.103:0/3962066096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074350 msgr2=0x7f941c074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 --2- 172.21.15.103:0/3962066096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074350 0x7f941c074730 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f9418009900 tx=0x7f941802a480 comp rx=0 tx=0).stop 2024-08-19T01:46:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 -- 172.21.15.103:0/3962066096 shutdown_connections 2024-08-19T01:46:02.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 --2- 172.21.15.103:0/3962066096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c10e0f0 0x7f941c110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 --2- 172.21.15.103:0/3962066096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f941c074c70 0x7f941c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 --2- 172.21.15.103:0/3962066096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074350 0x7f941c074730 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 -- 172.21.15.103:0/3962066096 >> 172.21.15.103:0/3962066096 conn(0x7f941c0fd5f0 msgr2=0x7f941c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 -- 172.21.15.103:0/3962066096 shutdown_connections 2024-08-19T01:46:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.727+0000 7f9423bc8640 1 -- 172.21.15.103:0/3962066096 wait complete. 2024-08-19T01:46:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 Processor -- start 2024-08-19T01:46:02.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 -- start start 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f941c074350 0x7f941c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074c70 0x7f941c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c10e0f0 0x7f941c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f941c113d50 con 0x7f941c074c70 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f941c113bd0 con 0x7f941c074350 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.728+0000 7f9423bc8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f941c113ed0 con 0x7f941c10e0f0 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074c70 0x7f941c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074c70 0x7f941c19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46948/0 (socket says 172.21.15.103:46948) 2024-08-19T01:46:02.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 -- 172.21.15.103:0/1630544577 learned_addr learned my addr 172.21.15.103:0/1630544577 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9422165640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c10e0f0 0x7f941c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421964640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f941c074350 0x7f941c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 -- 172.21.15.103:0/1630544577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c10e0f0 msgr2=0x7f941c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c10e0f0 0x7f941c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 -- 172.21.15.103:0/1630544577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f941c074350 msgr2=0x7f941c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f941c074350 0x7f941c19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 -- 172.21.15.103:0/1630544577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c19f290 con 0x7f941c074c70 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421964640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f941c074350 0x7f941c19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:46:02.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9421163640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074c70 0x7f941c19a820 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f940c00e9f0 tx=0x7f940c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9412ffd640 1 -- 172.21.15.103:0/1630544577 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c00cd80 con 0x7f941c074c70 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9412ffd640 1 -- 172.21.15.103:0/1630544577 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f940c0054d0 con 0x7f941c074c70 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941c19f580 con 0x7f941c074c70 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9412ffd640 1 -- 172.21.15.103:0/1630544577 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c010490 con 0x7f941c074c70 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.729+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941c1a6e60 con 0x7f941c074c70 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.730+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941c074730 con 0x7f941c074c70 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.731+0000 7f9412ffd640 1 -- 172.21.15.103:0/1630544577 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f940c005020 con 0x7f941c074c70 2024-08-19T01:46:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.732+0000 7f9412ffd640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f94080a2b80 0x7f94080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.732+0000 7f9421964640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f94080a2b80 0x7f94080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.732+0000 7f9412ffd640 1 -- 172.21.15.103:0/1630544577 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f940c0cf4f0 con 0x7f941c074c70 2024-08-19T01:46:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.732+0000 7f9421964640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f94080a2b80 0x7f94080a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9418031440 tx=0x7f9418005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.736+0000 7f9412ffd640 1 -- 172.21.15.103:0/1630544577 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f940c09c480 con 0x7f941c074c70 2024-08-19T01:46:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.915+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f941c02d5e0 con 0x7f941c074c70 2024-08-19T01:46:02.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.915+0000 7f9412ffd640 1 -- 172.21.15.103:0/1630544577 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+600 (secure 0 0 0) 0x7f940c09bd80 con 0x7f941c074c70 2024-08-19T01:46:02.917 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2698864603 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:02.918 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3003368150 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:02.918 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2881014674 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:02.918 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2881014674 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:02.918 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2169969902 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:02.918 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/3969074886 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:02.918 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/975765434 2024-08-20T01:27:00.697852+0000 2024-08-19T01:46:02.918 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/3969074886 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:02.919 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3082166873 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:02.919 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/948089109 2024-08-20T01:26:36.589386+0000 2024-08-19T01:46:02.919 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-08-19T01:46:02.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f94080a2b80 msgr2=0x7f94080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:02.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f94080a2b80 0x7f94080a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9418031440 tx=0x7f9418005a20 comp rx=0 tx=0).stop 2024-08-19T01:46:02.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074c70 msgr2=0x7f941c19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:02.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074c70 0x7f941c19a820 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f940c00e9f0 tx=0x7f940c00edf0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 shutdown_connections 2024-08-19T01:46:02.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f94080a2b80 0x7f94080a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f941c10e0f0 0x7f941c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f941c074c70 0x7f941c19a820 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 --2- 172.21.15.103:0/1630544577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f941c074350 0x7f941c19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 >> 172.21.15.103:0/1630544577 conn(0x7f941c0fd5f0 msgr2=0x7f941c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 shutdown_connections 2024-08-19T01:46:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:02.918+0000 7f9423bc8640 1 -- 172.21.15.103:0/1630544577 wait complete. 2024-08-19T01:46:03.295 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:03 smithi103 ceph-mon[18436]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:46:03.295 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:03 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/1630544577' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:46:03.295 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:03 smithi103 ceph-mon[18436]: pgmap v647: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-08-19T01:46:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:03 smithi190 ceph-mon[22032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:46:03.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:03 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/1630544577' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:46:03.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:03 smithi190 ceph-mon[22032]: pgmap v647: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-08-19T01:46:03.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:03 smithi146 ceph-mon[22098]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-19T01:46:03.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:03 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/1630544577' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-19T01:46:03.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:03 smithi146 ceph-mon[22098]: pgmap v647: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-08-19T01:46:03.688 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi103.front.sepia.ceph.com... 2024-08-19T01:46:03.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:46:03.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-19T01:46:03.736 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi103.stderr:ceph-fuse[69453]: fuse finished with error 0 and tester_r 0 2024-08-19T01:46:04.528 INFO:teuthology.orchestra.run:waiting for 300 2024-08-19T01:46:04.537 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:46:04.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:46:04.537 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-19T01:46:04.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:46:04.595 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-08-19T01:46:04.663 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-19T01:46:04.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:46:04.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-19T01:46:04.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-19T01:46:04.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-08-19T01:46:04.814 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-19T01:46:04.826 INFO:tasks.cephadm:Teardown begin 2024-08-19T01:46:04.827 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-19T01:46:04.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:46:05.085 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:05.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:05 smithi103 ceph-mon[18436]: pgmap v648: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-19T01:46:05.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:05 smithi190 ceph-mon[22032]: pgmap v648: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-19T01:46:05.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:05 smithi146 ceph-mon[22098]: pgmap v648: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-19T01:46:05.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 -- 172.21.15.103:0/1464854868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564105c70 msgr2=0x7f1564106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:05.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 --2- 172.21.15.103:0/1464854868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564105c70 0x7f1564106100 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f155c009ca0 tx=0x7f155c02a940 comp rx=0 tx=0).stop 2024-08-19T01:46:05.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 -- 172.21.15.103:0/1464854868 shutdown_connections 2024-08-19T01:46:05.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 --2- 172.21.15.103:0/1464854868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564105c70 0x7f1564106100 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 --2- 172.21.15.103:0/1464854868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564101180 0x7f1564105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 --2- 172.21.15.103:0/1464854868 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1564100860 0x7f1564100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 -- 172.21.15.103:0/1464854868 >> 172.21.15.103:0/1464854868 conn(0x7f15640fc4d0 msgr2=0x7f15640fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 -- 172.21.15.103:0/1464854868 shutdown_connections 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.599+0000 7f156b1e5640 1 -- 172.21.15.103:0/1464854868 wait complete. 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 Processor -- start 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 -- start start 2024-08-19T01:46:05.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564100860 0x7f156419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:05.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564101180 0x7f156419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:05.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1564105c70 0x7f15641a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:05.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1564118160 con 0x7f1564100860 2024-08-19T01:46:05.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1564117fe0 con 0x7f1564105c70 2024-08-19T01:46:05.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f156b1e5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15641182e0 con 0x7f1564101180 2024-08-19T01:46:05.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f1568f81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564100860 0x7f156419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:05.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f1568f81640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564100860 0x7f156419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46970/0 (socket says 172.21.15.103:46970) 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f1568f81640 1 -- 172.21.15.103:0/3610421845 learned_addr learned my addr 172.21.15.103:0/3610421845 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.600+0000 7f1563fff640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564101180 0x7f156419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1569782640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1564105c70 0x7f15641a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1568f81640 1 -- 172.21.15.103:0/3610421845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564101180 msgr2=0x7f156419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1568f81640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564101180 0x7f156419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1568f81640 1 -- 172.21.15.103:0/3610421845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1564105c70 msgr2=0x7f15641a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1568f81640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1564105c70 0x7f15641a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1568f81640 1 -- 172.21.15.103:0/3610421845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15641a26b0 con 0x7f1564100860 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1569782640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1564105c70 0x7f15641a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1563fff640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564101180 0x7f156419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1568f81640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564100860 0x7f156419e760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f155800d950 tx=0x7f155800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1561ffb640 1 -- 172.21.15.103:0/3610421845 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558017070 con 0x7f1564100860 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15641a29a0 con 0x7f1564100860 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1561ffb640 1 -- 172.21.15.103:0/3610421845 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1558009c30 con 0x7f1564100860 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f1561ffb640 1 -- 172.21.15.103:0/3610421845 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155800f620 con 0x7f1564100860 2024-08-19T01:46:05.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.601+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641ab310 con 0x7f1564100860 2024-08-19T01:46:05.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.602+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1564105650 con 0x7f1564100860 2024-08-19T01:46:05.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.603+0000 7f1561ffb640 1 -- 172.21.15.103:0/3610421845 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f15580109a0 con 0x7f1564100860 2024-08-19T01:46:05.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.604+0000 7f1561ffb640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15540a2b80 0x7f15540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:05.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.604+0000 7f1563fff640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15540a2b80 0x7f15540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:05.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.604+0000 7f1561ffb640 1 -- 172.21.15.103:0/3610421845 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f15580ced10 con 0x7f1564100860 2024-08-19T01:46:05.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.604+0000 7f1563fff640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15540a2b80 0x7f15540a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f15640ff9e0 tx=0x7f155000b610 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:05.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.608+0000 7f1561ffb640 1 -- 172.21.15.103:0/3610421845 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1558009de0 con 0x7f1564100860 2024-08-19T01:46:05.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.810+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f156419fac0 con 0x7f1564100860 2024-08-19T01:46:05.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.811+0000 7f1561ffb640 1 -- 172.21.15.103:0/3610421845 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11990 (secure 0 0 0) 0x7f155809bca0 con 0x7f1564100860 2024-08-19T01:46:05.814 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:46:05.814 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":42,"btime":"2024-08-19T01:46:02:050100+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:46:01.577100+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:46:05.816 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 42 2024-08-19T01:46:05.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15540a2b80 msgr2=0x7f15540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:05.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15540a2b80 0x7f15540a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f15640ff9e0 tx=0x7f155000b610 comp rx=0 tx=0).stop 2024-08-19T01:46:05.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564100860 msgr2=0x7f156419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:05.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564100860 0x7f156419e760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f155800d950 tx=0x7f155800de10 comp rx=0 tx=0).stop 2024-08-19T01:46:05.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 shutdown_connections 2024-08-19T01:46:05.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f15540a2b80 0x7f15540a5040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1564105c70 0x7f15641a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1564101180 0x7f156419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 --2- 172.21.15.103:0/3610421845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1564100860 0x7f156419e760 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:05.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 >> 172.21.15.103:0/3610421845 conn(0x7f15640fc4d0 msgr2=0x7f156410f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:05.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 shutdown_connections 2024-08-19T01:46:05.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:05.813+0000 7f156b1e5640 1 -- 172.21.15.103:0/3610421845 wait complete. 2024-08-19T01:46:06.236 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:06 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3610421845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:46:06.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:06 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3610421845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:46:06.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:06 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3610421845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:46:06.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-19T01:46:06.840 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:07.364 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:07 smithi103 ceph-mon[18436]: pgmap v649: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-19T01:46:07.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:07 smithi190 ceph-mon[22032]: pgmap v649: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-19T01:46:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 -- 172.21.15.103:0/1794487073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c102c20 msgr2=0x7f1b7c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 --2- 172.21.15.103:0/1794487073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c102c20 0x7f1b7c10d770 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009ca0 tx=0x7f1b7402a950 comp rx=0 tx=0).stop 2024-08-19T01:46:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 -- 172.21.15.103:0/1794487073 shutdown_connections 2024-08-19T01:46:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 --2- 172.21.15.103:0/1794487073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c102c20 0x7f1b7c10d770 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 --2- 172.21.15.103:0/1794487073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b7c1022a0 0x7f1b7c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 --2- 172.21.15.103:0/1794487073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b7c100b50 0x7f1b7c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 -- 172.21.15.103:0/1794487073 >> 172.21.15.103:0/1794487073 conn(0x7f1b7c0fc500 msgr2=0x7f1b7c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 -- 172.21.15.103:0/1794487073 shutdown_connections 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.361+0000 7f1b846d8640 1 -- 172.21.15.103:0/1794487073 wait complete. 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 Processor -- start 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 -- start start 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c100b50 0x7f1b7c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b7c1022a0 0x7f1b7c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b7c102c20 0x7f1b7c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c110b60 con 0x7f1b7c100b50 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1109e0 con 0x7f1b7c1022a0 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b846d8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c110ce0 con 0x7f1b7c102c20 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b82474640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c100b50 0x7f1b7c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b82474640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c100b50 0x7f1b7c1970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46988/0 (socket says 172.21.15.103:46988) 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.362+0000 7f1b82474640 1 -- 172.21.15.103:0/838619436 learned_addr learned my addr 172.21.15.103:0/838619436 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b81c73640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b7c1022a0 0x7f1b7c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b82c75640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b7c102c20 0x7f1b7c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b82474640 1 -- 172.21.15.103:0/838619436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b7c102c20 msgr2=0x7f1b7c19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b82474640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b7c102c20 0x7f1b7c19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b82474640 1 -- 172.21.15.103:0/838619436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b7c1022a0 msgr2=0x7f1b7c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b82474640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b7c1022a0 0x7f1b7c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b82474640 1 -- 172.21.15.103:0/838619436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c19c060 con 0x7f1b7c100b50 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b82474640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c100b50 0x7f1b7c1970b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800d950 tx=0x7f1b7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b737fe640 1 -- 172.21.15.103:0/838619436 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78017030 con 0x7f1b7c100b50 2024-08-19T01:46:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b737fe640 1 -- 172.21.15.103:0/838619436 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b78004020 con 0x7f1b7c100b50 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c19c350 con 0x7f1b7c100b50 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b737fe640 1 -- 172.21.15.103:0/838619436 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780047a0 con 0x7f1b7c100b50 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.363+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c069ca0 con 0x7f1b7c100b50 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.364+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c100f30 con 0x7f1b7c100b50 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.367+0000 7f1b737fe640 1 -- 172.21.15.103:0/838619436 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1b780041d0 con 0x7f1b7c100b50 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.367+0000 7f1b737fe640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1b680a2b80 0x7f1b680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.368+0000 7f1b81c73640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1b680a2b80 0x7f1b680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.368+0000 7f1b737fe640 1 -- 172.21.15.103:0/838619436 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7f1b780cf120 con 0x7f1b7c100b50 2024-08-19T01:46:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.368+0000 7f1b81c73640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1b680a2b80 0x7f1b680a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0ffc10 tx=0x7f1b6c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.370+0000 7f1b737fe640 1 -- 172.21.15.103:0/838619436 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1b7809c0b0 con 0x7f1b7c100b50 2024-08-19T01:46:07.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:07 smithi146 ceph-mon[22098]: pgmap v649: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-19T01:46:07.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.579+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b7c198410 con 0x7f1b7c100b50 2024-08-19T01:46:07.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.580+0000 7f1b737fe640 1 -- 172.21.15.103:0/838619436 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11990 (secure 0 0 0) 0x7f1b7c198410 con 0x7f1b7c100b50 2024-08-19T01:46:07.582 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:46:07.582 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":42,"btime":"2024-08-19T01:46:02:050100+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14514,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3930645278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3930645278},{"type":"v1","addr":"172.21.15.103:6835","nonce":3930645278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3245550347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3245550347},{"type":"v1","addr":"172.21.15.103:6837","nonce":3245550347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-19T01:33:23.991203+0000","modified":"2024-08-19T01:46:01.577100+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24485,"mds_1":24478,"mds_2":24466,"mds_3":24454,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14580":{"gid":14580,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/2158943857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2158943857},{"type":"v1","addr":"172.21.15.103:6839","nonce":2158943857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14592":{"gid":14592,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1914348964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1914348964},{"type":"v1","addr":"172.21.15.103:6841","nonce":1914348964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/1682379250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1682379250},{"type":"v1","addr":"172.21.15.146:6833","nonce":1682379250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/1574351198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1574351198},{"type":"v1","addr":"172.21.15.190:6833","nonce":1574351198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/3609998196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":3609998196},{"type":"v1","addr":"172.21.15.146:6835","nonce":3609998196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.146:6837/3498679319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":3498679319},{"type":"v1","addr":"172.21.15.146:6837","nonce":3498679319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.190:6835/794893065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":794893065},{"type":"v1","addr":"172.21.15.190:6835","nonce":794893065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.190:6837/66432346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":66432346},{"type":"v1","addr":"172.21.15.190:6837","nonce":66432346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.190:6839/639745782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":639745782},{"type":"v1","addr":"172.21.15.190:6839","nonce":639745782}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.146:6839/1128555603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":1128555603},{"type":"v1","addr":"172.21.15.146:6839","nonce":1128555603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485,24466,24454,24478,24452]},"id":1}]} 2024-08-19T01:46:07.583 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 42 2024-08-19T01:46:07.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1b680a2b80 msgr2=0x7f1b680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:07.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1b680a2b80 0x7f1b680a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0ffc10 tx=0x7f1b6c00b390 comp rx=0 tx=0).stop 2024-08-19T01:46:07.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c100b50 msgr2=0x7f1b7c1970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:07.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c100b50 0x7f1b7c1970b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800d950 tx=0x7f1b7800de10 comp rx=0 tx=0).stop 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 shutdown_connections 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f1b680a2b80 0x7f1b680a5040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b7c102c20 0x7f1b7c19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1b7c1022a0 0x7f1b7c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 --2- 172.21.15.103:0/838619436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7c100b50 0x7f1b7c1970b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 >> 172.21.15.103:0/838619436 conn(0x7f1b7c0fc500 msgr2=0x7f1b7c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.582+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 shutdown_connections 2024-08-19T01:46:07.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:07.583+0000 7f1b846d8640 1 -- 172.21.15.103:0/838619436 wait complete. 2024-08-19T01:46:08.343 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-19T01:46:08.368 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:08 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/838619436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:46:08.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:08 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/838619436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:46:08.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:08 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/838619436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-19T01:46:08.598 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 -- 172.21.15.103:0/1735025435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 msgr2=0x7fe1a41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 --2- 172.21.15.103:0/1735025435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a41026e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe190009900 tx=0x7fe19002a480 comp rx=0 tx=0).stop 2024-08-19T01:46:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 -- 172.21.15.103:0/1735025435 shutdown_connections 2024-08-19T01:46:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 --2- 172.21.15.103:0/1735025435 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4102c20 0x7fe1a410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 --2- 172.21.15.103:0/1735025435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a41026e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 --2- 172.21.15.103:0/1735025435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a4100b50 0x7fe1a4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 -- 172.21.15.103:0/1735025435 >> 172.21.15.103:0/1735025435 conn(0x7fe1a40fc500 msgr2=0x7fe1a40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 -- 172.21.15.103:0/1735025435 shutdown_connections 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.132+0000 7fe1aae89640 1 -- 172.21.15.103:0/1735025435 wait complete. 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 Processor -- start 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 -- start start 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4100b50 0x7fe1a4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a4102c20 0x7fe1a41a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:09.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4110b80 con 0x7fe1a41022a0 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4110a00 con 0x7fe1a4100b50 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1aae89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4110d00 con 0x7fe1a4102c20 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a4199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46998/0 (socket says 172.21.15.103:46998) 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 -- 172.21.15.103:0/539615766 learned_addr learned my addr 172.21.15.103:0/539615766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a9426640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a4102c20 0x7fe1a41a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a8c25640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4100b50 0x7fe1a4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:09.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 -- 172.21.15.103:0/539615766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a4102c20 msgr2=0x7fe1a41a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a4102c20 0x7fe1a41a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 -- 172.21.15.103:0/539615766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4100b50 msgr2=0x7fe1a4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4100b50 0x7fe1a4199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a3fff640 1 -- 172.21.15.103:0/539615766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a41a45f0 con 0x7fe1a41022a0 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.133+0000 7fe1a8c25640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4100b50 0x7fe1a4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.134+0000 7fe1a3fff640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a4199570 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe190006010 tx=0x7fe1900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.134+0000 7fe1a1ffb640 1 -- 172.21.15.103:0/539615766 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19003f070 con 0x7fe1a41022a0 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.134+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a41a47c0 con 0x7fe1a41022a0 2024-08-19T01:46:09.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.134+0000 7fe1a1ffb640 1 -- 172.21.15.103:0/539615766 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe190003aa0 con 0x7fe1a41022a0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.134+0000 7fe1a1ffb640 1 -- 172.21.15.103:0/539615766 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190035030 con 0x7fe1a41022a0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.134+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a41a4c10 con 0x7fe1a41022a0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.135+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a4100f30 con 0x7fe1a41022a0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.136+0000 7fe1a1ffb640 1 -- 172.21.15.103:0/539615766 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe190042070 con 0x7fe1a41022a0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.137+0000 7fe1a1ffb640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1940a2b80 0x7fe1940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.137+0000 7fe1a8c25640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1940a2b80 0x7fe1940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.137+0000 7fe1a1ffb640 1 -- 172.21.15.103:0/539615766 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7fe1900efa10 con 0x7fe1a41022a0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.137+0000 7fe1a8c25640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1940a2b80 0x7fe1940a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe198006fd0 tx=0x7fe198008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:09.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.141+0000 7fe1a1ffb640 1 -- 172.21.15.103:0/539615766 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe1900bc9a0 con 0x7fe1a41022a0 2024-08-19T01:46:09.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:09 smithi103 ceph-mon[18436]: pgmap v650: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-08-19T01:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.312+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1a4108dc0 con 0x7fe1a41022a0 2024-08-19T01:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.313+0000 7fe1a1ffb640 1 -- 172.21.15.103:0/539615766 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22704 (secure 0 0 0) 0x7fe1900bc2a0 con 0x7fe1a41022a0 2024-08-19T01:46:09.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-08-19T01:46:09.316 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":86,"fsid":"e721016a-5dc9-11ef-bccf-c7b262605968","created":"2024-08-19T01:26:05.629697+0000","modified":"2024-08-19T01:34:01.874407+0000","last_up_change":"2024-08-19T01:32:29.240554+0000","last_in_change":"2024-08-19T01:32:10.480499+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-19T01:29:05.010792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-19T01:33:19.641931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-19T01:33:22.107240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0157e941-2ba7-4376-b473-9bcdaace0ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6803","nonce":2060644281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6805","nonce":2060644281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6809","nonce":2060644281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2060644281},{"type":"v1","addr":"172.21.15.103:6807","nonce":2060644281}]},"public_addr":"172.21.15.103:6803/2060644281","cluster_addr":"172.21.15.103:6805/2060644281","heartbeat_back_addr":"172.21.15.103:6809/2060644281","heartbeat_front_addr":"172.21.15.103:6807/2060644281","state":["exists","up"]},{"osd":1,"uuid":"bc4bc49c-cf46-48e4-98ed-60ad89a2812a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6801","nonce":351745310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6803","nonce":351745310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6807","nonce":351745310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":351745310},{"type":"v1","addr":"172.21.15.146:6805","nonce":351745310}]},"public_addr":"172.21.15.146:6801/351745310","cluster_addr":"172.21.15.146:6803/351745310","heartbeat_back_addr":"172.21.15.146:6807/351745310","heartbeat_front_addr":"172.21.15.146:6805/351745310","state":["exists","up"]},{"osd":2,"uuid":"5253afd0-394e-4565-b605-fad40ff30406","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6801","nonce":2941836968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6803","nonce":2941836968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6807","nonce":2941836968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2941836968},{"type":"v1","addr":"172.21.15.190:6805","nonce":2941836968}]},"public_addr":"172.21.15.190:6801/2941836968","cluster_addr":"172.21.15.190:6803/2941836968","heartbeat_back_addr":"172.21.15.190:6807/2941836968","heartbeat_front_addr":"172.21.15.190:6805/2941836968","state":["exists","up"]},{"osd":3,"uuid":"e89d7c5c-8df2-4130-bc7d-1fb7ddd0960b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6811","nonce":2225524779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6813","nonce":2225524779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6817","nonce":2225524779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2225524779},{"type":"v1","addr":"172.21.15.103:6815","nonce":2225524779}]},"public_addr":"172.21.15.103:6811/2225524779","cluster_addr":"172.21.15.103:6813/2225524779","heartbeat_back_addr":"172.21.15.103:6817/2225524779","heartbeat_front_addr":"172.21.15.103:6815/2225524779","state":["exists","up"]},{"osd":4,"uuid":"2d80a218-c821-4a4c-a133-ec98bf694692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6809","nonce":3000848263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6811","nonce":3000848263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6815","nonce":3000848263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3000848263},{"type":"v1","addr":"172.21.15.146:6813","nonce":3000848263}]},"public_addr":"172.21.15.146:6809/3000848263","cluster_addr":"172.21.15.146:6811/3000848263","heartbeat_back_addr":"172.21.15.146:6815/3000848263","heartbeat_front_addr":"172.21.15.146:6813/3000848263","state":["exists","up"]},{"osd":5,"uuid":"a36b0adc-8fec-4313-8298-55fc4236a809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6809","nonce":1607717439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6811","nonce":1607717439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6815","nonce":1607717439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1607717439},{"type":"v1","addr":"172.21.15.190:6813","nonce":1607717439}]},"public_addr":"172.21.15.190:6809/1607717439","cluster_addr":"172.21.15.190:6811/1607717439","heartbeat_back_addr":"172.21.15.190:6815/1607717439","heartbeat_front_addr":"172.21.15.190:6813/1607717439","state":["exists","up"]},{"osd":6,"uuid":"243118b8-58e2-40a3-833c-008ec861cab9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6819","nonce":3250967398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6821","nonce":3250967398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6825","nonce":3250967398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3250967398},{"type":"v1","addr":"172.21.15.103:6823","nonce":3250967398}]},"public_addr":"172.21.15.103:6819/3250967398","cluster_addr":"172.21.15.103:6821/3250967398","heartbeat_back_addr":"172.21.15.103:6825/3250967398","heartbeat_front_addr":"172.21.15.103:6823/3250967398","state":["exists","up"]},{"osd":7,"uuid":"01bec701-16f3-4c16-87f7-59daf836ba88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6817","nonce":2549773085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6819","nonce":2549773085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6823","nonce":2549773085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2549773085},{"type":"v1","addr":"172.21.15.146:6821","nonce":2549773085}]},"public_addr":"172.21.15.146:6817/2549773085","cluster_addr":"172.21.15.146:6819/2549773085","heartbeat_back_addr":"172.21.15.146:6823/2549773085","heartbeat_front_addr":"172.21.15.146:6821/2549773085","state":["exists","up"]},{"osd":8,"uuid":"581ea912-28e7-4dec-8e01-18301ee980cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6817","nonce":2035395456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6819","nonce":2035395456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6823","nonce":2035395456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2035395456},{"type":"v1","addr":"172.21.15.190:6821","nonce":2035395456}]},"public_addr":"172.21.15.190:6817/2035395456","cluster_addr":"172.21.15.190:6819/2035395456","heartbeat_back_addr":"172.21.15.190:6823/2035395456","heartbeat_front_addr":"172.21.15.190:6821/2035395456","state":["exists","up"]},{"osd":9,"uuid":"f8ae349f-b007-407a-b7e9-0f9b9ba66aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6827","nonce":1538313167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6829","nonce":1538313167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6833","nonce":1538313167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1538313167},{"type":"v1","addr":"172.21.15.103:6831","nonce":1538313167}]},"public_addr":"172.21.15.103:6827/1538313167","cluster_addr":"172.21.15.103:6829/1538313167","heartbeat_back_addr":"172.21.15.103:6833/1538313167","heartbeat_front_addr":"172.21.15.103:6831/1538313167","state":["exists","up"]},{"osd":10,"uuid":"5d90c65e-e820-4352-b4bd-58204606e5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6825","nonce":1357662473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6827","nonce":1357662473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6831","nonce":1357662473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":1357662473},{"type":"v1","addr":"172.21.15.146:6829","nonce":1357662473}]},"public_addr":"172.21.15.146:6825/1357662473","cluster_addr":"172.21.15.146:6827/1357662473","heartbeat_back_addr":"172.21.15.146:6831/1357662473","heartbeat_front_addr":"172.21.15.146:6829/1357662473","state":["exists","up"]},{"osd":11,"uuid":"22d60bfc-b9ac-46e7-b4cf-dc08b2a41f97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6825","nonce":2863234594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6827","nonce":2863234594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6831","nonce":2863234594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2863234594},{"type":"v1","addr":"172.21.15.190:6829","nonce":2863234594}]},"public_addr":"172.21.15.190:6825/2863234594","cluster_addr":"172.21.15.190:6827/2863234594","heartbeat_back_addr":"172.21.15.190:6831/2863234594","heartbeat_front_addr":"172.21.15.190:6829/2863234594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:20.067760+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:28:41.471223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:02.211031+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:25.360314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:29:47.622391+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:10.198419+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:33.002202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:30:55.687816+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:18.018723+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:31:41.839342+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:05.207538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-19T01:32:27.781250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/2698864603":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/3003368150":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6801/2881014674":"2024-08-20T01:27:00.697852+0000","172.21.15.103:0/2169969902":"2024-08-20T01:26:36.589386+0000","172.21.15.103:6801/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/975765434":"2024-08-20T01:27:00.697852+0000","172.21.15.103:6800/3969074886":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/3082166873":"2024-08-20T01:26:36.589386+0000","172.21.15.103:0/948089109":"2024-08-20T01:26:36.589386+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-19T01:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.315+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1940a2b80 msgr2=0x7fe1940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.315+0000 7fe1aae89640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1940a2b80 0x7fe1940a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe198006fd0 tx=0x7fe198008040 comp rx=0 tx=0).stop 2024-08-19T01:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 msgr2=0x7fe1a4199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a4199570 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe190006010 tx=0x7fe1900042f0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 shutdown_connections 2024-08-19T01:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fe1940a2b80 0x7fe1940a5040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1a4102c20 0x7fe1a41a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1a41022a0 0x7fe1a4199570 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 --2- 172.21.15.103:0/539615766 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe1a4100b50 0x7fe1a4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 >> 172.21.15.103:0/539615766 conn(0x7fe1a40fc500 msgr2=0x7fe1a40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 shutdown_connections 2024-08-19T01:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:09.316+0000 7fe1aae89640 1 -- 172.21.15.103:0/539615766 wait complete. 2024-08-19T01:46:09.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:09 smithi190 ceph-mon[22032]: pgmap v650: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-08-19T01:46:09.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:09 smithi146 ceph-mon[22098]: pgmap v650: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-08-19T01:46:10.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs fail cephfs 2024-08-19T01:46:10.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:10 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/539615766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:46:10.348 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:10.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:10 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/539615766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:46:10.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:10 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/539615766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-19T01:46:10.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 -- 172.21.15.103:0/2757700471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41007b0 msgr2=0x7efce4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:10.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 --2- 172.21.15.103:0/2757700471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41007b0 0x7efce4100c10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efccc009930 tx=0x7efccc02a420 comp rx=0 tx=0).stop 2024-08-19T01:46:10.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 -- 172.21.15.103:0/2757700471 shutdown_connections 2024-08-19T01:46:10.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 --2- 172.21.15.103:0/2757700471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce4101150 0x7efce410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 --2- 172.21.15.103:0/2757700471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41007b0 0x7efce4100c10 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 --2- 172.21.15.103:0/2757700471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce41067b0 0x7efce4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 -- 172.21.15.103:0/2757700471 >> 172.21.15.103:0/2757700471 conn(0x7efce40fc4d0 msgr2=0x7efce40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 -- 172.21.15.103:0/2757700471 shutdown_connections 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.858+0000 7efcea684640 1 -- 172.21.15.103:0/2757700471 wait complete. 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 Processor -- start 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 -- start start 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce41007b0 0x7efce4197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce4101150 0x7efce4197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41067b0 0x7efce419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efce4110a60 con 0x7efce41067b0 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efce41108e0 con 0x7efce41007b0 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcea684640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efce4110be0 con 0x7efce4101150 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41067b0 0x7efce419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41067b0 0x7efce419b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47020/0 (socket says 172.21.15.103:47020) 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 -- 172.21.15.103:0/2613993360 learned_addr learned my addr 172.21.15.103:0/2613993360 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce3fff640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce41007b0 0x7efce4197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcdbfff640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce4101150 0x7efce4197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 -- 172.21.15.103:0/2613993360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce4101150 msgr2=0x7efce4197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce4101150 0x7efce4197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 -- 172.21.15.103:0/2613993360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce41007b0 msgr2=0x7efce4197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce41007b0 0x7efce4197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 -- 172.21.15.103:0/2613993360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce419bfb0 con 0x7efce41067b0 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce3fff640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce41007b0 0x7efce4197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-19T01:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efcdbfff640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce4101150 0x7efce4197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.859+0000 7efce8c21640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41067b0 0x7efce419b8d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00eaa0 tx=0x7efcdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.860+0000 7efce1ffb640 1 -- 172.21.15.103:0/2613993360 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cdb0 con 0x7efce41067b0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.860+0000 7efce1ffb640 1 -- 172.21.15.103:0/2613993360 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc005020 con 0x7efce41067b0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.860+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce419c2a0 con 0x7efce41067b0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.860+0000 7efce1ffb640 1 -- 172.21.15.103:0/2613993360 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0116d0 con 0x7efce41067b0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.860+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce41a3b80 con 0x7efce41067b0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.861+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce4100c10 con 0x7efce41067b0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.862+0000 7efce1ffb640 1 -- 172.21.15.103:0/2613993360 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efcdc0051d0 con 0x7efce41067b0 2024-08-19T01:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.863+0000 7efce1ffb640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcd00a2b80 0x7efcd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:10.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.863+0000 7efce3fff640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcd00a2b80 0x7efcd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:10.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.863+0000 7efce1ffb640 1 -- 172.21.15.103:0/2613993360 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8190+0+0 (secure 0 0 0) 0x7efcdc0cf6a0 con 0x7efce41067b0 2024-08-19T01:46:10.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.863+0000 7efce3fff640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcd00a2b80 0x7efcd00a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efcd4005fd0 tx=0x7efcd4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:10.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:10.866+0000 7efce1ffb640 1 -- 172.21.15.103:0/2613993360 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efcdc016030 con 0x7efce41067b0 2024-08-19T01:46:11.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.071+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efce4198360 con 0x7efce41067b0 2024-08-19T01:46:11.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.073+0000 7efce1ffb640 1 -- 172.21.15.103:0/2613993360 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7efce4198360 con 0x7efce41067b0 2024-08-19T01:46:11.075 INFO:teuthology.orchestra.run.smithi103.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-19T01:46:11.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcd00a2b80 msgr2=0x7efcd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:11.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcd00a2b80 0x7efcd00a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efcd4005fd0 tx=0x7efcd4005850 comp rx=0 tx=0).stop 2024-08-19T01:46:11.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41067b0 msgr2=0x7efce419b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:11.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41067b0 0x7efce419b8d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00eaa0 tx=0x7efcdc00ef60 comp rx=0 tx=0).stop 2024-08-19T01:46:11.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 shutdown_connections 2024-08-19T01:46:11.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7efcd00a2b80 0x7efcd00a5040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:11.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce41067b0 0x7efce419b8d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:11.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce4101150 0x7efce4197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:11.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 --2- 172.21.15.103:0/2613993360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efce41007b0 0x7efce4197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:11.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.077+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 >> 172.21.15.103:0/2613993360 conn(0x7efce40fc4d0 msgr2=0x7efce40fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:11.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.078+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 shutdown_connections 2024-08-19T01:46:11.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:11.078+0000 7efcea684640 1 -- 172.21.15.103:0/2613993360 wait complete. 2024-08-19T01:46:11.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mon[22032]: pgmap v651: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-08-19T01:46:11.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2613993360' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:46:11.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mon[22032]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-19T01:46:11.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mon[22032]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:46:11.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2613993360' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-19T01:46:11.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mon[22032]: osdmap e87: 12 total, 12 up, 12 in 2024-08-19T01:46:11.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mon[22032]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-19T01:46:11.381 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l[51604]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.382 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l[51604]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.382 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mds[51608]: -- 172.21.15.190:0/1606596437 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd81157860 con 0x55cd81135400 2024-08-19T01:46:11.382 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l[51604]: starting mds.l at 2024-08-19T01:46:11.382 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f[51916]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.383 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f[51916]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.383 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mds[51920]: -- 172.21.15.190:0/4276490278 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f55d89c20 con 0x563f55d67400 2024-08-19T01:46:11.383 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f[51916]: starting mds.f at 2024-08-19T01:46:11.383 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i[52519]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.383 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i[52519]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.384 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mds[52523]: -- 172.21.15.190:0/4137708556 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578a910fc20 con 0x5578a90ec800 2024-08-19T01:46:11.384 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i[52519]: starting mds.i at 2024-08-19T01:46:11.384 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c[52219]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.384 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c[52219]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.384 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-mds[52223]: -- 172.21.15.190:0/2468634909 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f88431860 con 0x561f8840e800 2024-08-19T01:46:11.384 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:46:11 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c[52219]: starting mds.c at 2024-08-19T01:46:11.450 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b[52184]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.451 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b[52184]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.451 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mds[52188]: -- 172.21.15.146:0/661001403 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9652af860 con 0x55d96528d000 2024-08-19T01:46:11.451 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b[52184]: starting mds.b at 2024-08-19T01:46:11.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mon[22098]: pgmap v651: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-08-19T01:46:11.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2613993360' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:46:11.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mon[22098]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-19T01:46:11.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mon[22098]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:46:11.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2613993360' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-19T01:46:11.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mon[22098]: osdmap e87: 12 total, 12 up, 12 in 2024-08-19T01:46:11.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mon[22098]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-19T01:46:11.452 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e[51575]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.453 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e[51575]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.453 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e[51575]: starting mds.e at 2024-08-19T01:46:11.453 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h[51880]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.453 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h[51880]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.453 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h[51880]: starting mds.h at 2024-08-19T01:46:11.454 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k[51264]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.454 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k[51264]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.454 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-mds[51268]: -- 172.21.15.146:0/919895061 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b7cebdc20 con 0x557b7ce9b400 2024-08-19T01:46:11.454 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:46:11 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k[51264]: starting mds.k at 2024-08-19T01:46:11.553 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g[58252]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.554 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g[58252]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.554 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mds[58256]: -- 172.21.15.103:0/1968666379 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56138c1a1860 con 0x56138c17f400 2024-08-19T01:46:11.554 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g[58252]: starting mds.g at 2024-08-19T01:46:11.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a[18432]: 2024-08-19T01:46:11.072+0000 7f453cd8b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:46:11.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mon[18436]: pgmap v651: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-08-19T01:46:11.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2613993360' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-19T01:46:11.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mon[18436]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-19T01:46:11.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mon[18436]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-19T01:46:11.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2613993360' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-19T01:46:11.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mon[18436]: osdmap e87: 12 total, 12 up, 12 in 2024-08-19T01:46:11.556 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mon[18436]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-19T01:46:11.556 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a[58670]: ignoring --setuser ceph since I am not root 2024-08-19T01:46:11.556 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a[58670]: ignoring --setgroup ceph since I am not root 2024-08-19T01:46:11.556 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-mds[58674]: -- 172.21.15.103:0/1675373293 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5584f320d860 con 0x5584f31ea800 2024-08-19T01:46:11.556 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:46:11 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a[58670]: starting mds.a at 2024-08-19T01:46:11.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-19T01:46:12.170 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:12.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 -- 172.21.15.103:0/651895460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf0100b50 msgr2=0x7f6bf0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:12.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/651895460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf0100b50 0x7f6bf0100f30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009900 tx=0x7f6be002a480 comp rx=0 tx=0).stop 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 -- 172.21.15.103:0/651895460 shutdown_connections 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/651895460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf0102c20 0x7f6bf010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/651895460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bf01022a0 0x7f6bf01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/651895460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf0100b50 0x7f6bf0100f30 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 -- 172.21.15.103:0/651895460 >> 172.21.15.103:0/651895460 conn(0x7f6bf00fc500 msgr2=0x7f6bf00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 -- 172.21.15.103:0/651895460 shutdown_connections 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 -- 172.21.15.103:0/651895460 wait complete. 2024-08-19T01:46:12.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 Processor -- start 2024-08-19T01:46:12.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.789+0000 7f6bf54f2640 1 -- start start 2024-08-19T01:46:12.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bf54f2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bf0100b50 0x7f6bf0198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:12.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bf54f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf01022a0 0x7f6bf0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:12.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bf54f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf0102c20 0x7f6bf01a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:12.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bf54f2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0110b60 con 0x7f6bf01022a0 2024-08-19T01:46:12.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bf54f2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf01109e0 con 0x7f6bf0100b50 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bf54f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0110ce0 con 0x7f6bf0102c20 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf01022a0 0x7f6bf0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf01022a0 0x7f6bf0199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35288/0 (socket says 172.21.15.103:35288) 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 -- 172.21.15.103:0/3343948719 learned_addr learned my addr 172.21.15.103:0/3343948719 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6beeffd640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bf0100b50 0x7f6bf0198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bef7fe640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf0102c20 0x7f6bf01a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 -- 172.21.15.103:0/3343948719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf0102c20 msgr2=0x7f6bf01a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf0102c20 0x7f6bf01a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 -- 172.21.15.103:0/3343948719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bf0100b50 msgr2=0x7f6bf0198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bf0100b50 0x7f6bf0198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 -- 172.21.15.103:0/3343948719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf01a44d0 con 0x7f6bf01022a0 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bee7fc640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf01022a0 0x7f6bf0199450 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800c920 tx=0x7f6bd800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6be7fff640 1 -- 172.21.15.103:0/3343948719 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8007c70 con 0x7f6bf01022a0 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6be7fff640 1 -- 172.21.15.103:0/3343948719 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bd8007e10 con 0x7f6bf01022a0 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6be7fff640 1 -- 172.21.15.103:0/3343948719 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8018820 con 0x7f6bf01022a0 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.790+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf01a46a0 con 0x7f6bf01022a0 2024-08-19T01:46:12.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.791+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf01a4bb0 con 0x7f6bf01022a0 2024-08-19T01:46:12.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.791+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf0100f30 con 0x7f6bf01022a0 2024-08-19T01:46:12.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.793+0000 7f6be7fff640 1 -- 172.21.15.103:0/3343948719 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6bd80189c0 con 0x7f6bf01022a0 2024-08-19T01:46:12.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.793+0000 7f6be7fff640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6bdc0a2b80 0x7f6bdc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:12.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.793+0000 7f6beeffd640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6bdc0a2b80 0x7f6bdc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:12.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.793+0000 7f6be7fff640 1 -- 172.21.15.103:0/3343948719 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8620+0+0 (secure 0 0 0) 0x7f6bd80cf2f0 con 0x7f6bf01022a0 2024-08-19T01:46:12.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.794+0000 7f6beeffd640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6bdc0a2b80 0x7f6bdc0a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009230 tx=0x7f6be00058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:12.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:12.797+0000 7f6be7fff640 1 -- 172.21.15.103:0/3343948719 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6bd8014030 con 0x7f6bf01022a0 2024-08-19T01:46:13.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.000+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6bf0108dc0 con 0x7f6bf01022a0 2024-08-19T01:46:13.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.136+0000 7f6be7fff640 1 -- 172.21.15.103:0/3343948719 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f6bd809c0d0 con 0x7f6bf01022a0 2024-08-19T01:46:13.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6bdc0a2b80 msgr2=0x7f6bdc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:13.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6bdc0a2b80 0x7f6bdc0a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009230 tx=0x7f6be00058d0 comp rx=0 tx=0).stop 2024-08-19T01:46:13.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf01022a0 msgr2=0x7f6bf0199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:13.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf01022a0 0x7f6bf0199450 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800c920 tx=0x7f6bd800cde0 comp rx=0 tx=0).stop 2024-08-19T01:46:13.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 shutdown_connections 2024-08-19T01:46:13.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f6bdc0a2b80 0x7f6bdc0a5040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:13.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6bf0102c20 0x7f6bf01a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:13.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bf01022a0 0x7f6bf0199450 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:13.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 --2- 172.21.15.103:0/3343948719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6bf0100b50 0x7f6bf0198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:13.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 >> 172.21.15.103:0/3343948719 conn(0x7f6bf00fc500 msgr2=0x7f6bf00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:13.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 shutdown_connections 2024-08-19T01:46:13.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:13.139+0000 7f6bf54f2640 1 -- 172.21.15.103:0/3343948719 wait complete. 2024-08-19T01:46:13.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6832/899565472,v1:172.21.15.146:6833/899565472] up:boot 2024-08-19T01:46:13.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6834/2989860970,v1:172.21.15.146:6835/2989860970] up:boot 2024-08-19T01:46:13.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6834/2850300588,v1:172.21.15.190:6835/2850300588] up:boot 2024-08-19T01:46:13.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6832/1204057806,v1:172.21.15.190:6833/1204057806] up:boot 2024-08-19T01:46:13.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6836/2534290017,v1:172.21.15.190:6838/2534290017] up:boot 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6836/3119594483,v1:172.21.15.146:6839/3119594483] up:boot 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.146:6837/611466139,v1:172.21.15.146:6838/611466139] up:boot 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.190:6837/2552134941,v1:172.21.15.190:6839/2552134941] up:boot 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.103:6838/1178427035,v1:172.21.15.103:6840/1178427035] up:boot 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: mds.? [v2:172.21.15.103:6839/3756239501,v1:172.21.15.103:6841/3756239501] up:boot 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:46:13.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:46:13.306 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:46:13.307 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: pgmap v653: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-08-19T01:46:13.307 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:13 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3343948719' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:46:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6832/899565472,v1:172.21.15.146:6833/899565472] up:boot 2024-08-19T01:46:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6834/2989860970,v1:172.21.15.146:6835/2989860970] up:boot 2024-08-19T01:46:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6834/2850300588,v1:172.21.15.190:6835/2850300588] up:boot 2024-08-19T01:46:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6832/1204057806,v1:172.21.15.190:6833/1204057806] up:boot 2024-08-19T01:46:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6836/2534290017,v1:172.21.15.190:6838/2534290017] up:boot 2024-08-19T01:46:13.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6836/3119594483,v1:172.21.15.146:6839/3119594483] up:boot 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.146:6837/611466139,v1:172.21.15.146:6838/611466139] up:boot 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.190:6837/2552134941,v1:172.21.15.190:6839/2552134941] up:boot 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.103:6838/1178427035,v1:172.21.15.103:6840/1178427035] up:boot 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: mds.? [v2:172.21.15.103:6839/3756239501,v1:172.21.15.103:6841/3756239501] up:boot 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:46:13.452 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:46:13.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:46:13.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:46:13.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:46:13.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:46:13.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:46:13.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: pgmap v653: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-08-19T01:46:13.453 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:13 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3343948719' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:46:13.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6832/899565472,v1:172.21.15.146:6833/899565472] up:boot 2024-08-19T01:46:13.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6834/2989860970,v1:172.21.15.146:6835/2989860970] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6834/2850300588,v1:172.21.15.190:6835/2850300588] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6832/1204057806,v1:172.21.15.190:6833/1204057806] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6836/2534290017,v1:172.21.15.190:6838/2534290017] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6836/3119594483,v1:172.21.15.146:6839/3119594483] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.146:6837/611466139,v1:172.21.15.146:6838/611466139] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.190:6837/2552134941,v1:172.21.15.190:6839/2552134941] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.103:6838/1178427035,v1:172.21.15.103:6840/1178427035] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: mds.? [v2:172.21.15.103:6839/3756239501,v1:172.21.15.103:6841/3756239501] up:boot 2024-08-19T01:46:13.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-19T01:46:13.632 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-19T01:46:13.633 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='mgr.14148 172.21.15.103:0/2210387883' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-19T01:46:13.633 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: pgmap v653: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-08-19T01:46:13.633 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:13 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3343948719' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-19T01:46:13.868 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-19T01:46:14.121 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:14.408 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:14 smithi103 ceph-mon[18436]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-19T01:46:14.408 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:14 smithi103 ceph-mon[18436]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-19T01:46:14.408 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:14 smithi103 ceph-mon[18436]: Cluster is now healthy 2024-08-19T01:46:14.408 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:14 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3343948719' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:46:14.408 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:14 smithi103 ceph-mon[18436]: fsmap 12 up:standby 2024-08-19T01:46:14.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:14 smithi146 ceph-mon[22098]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-19T01:46:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:14 smithi146 ceph-mon[22098]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-19T01:46:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:14 smithi146 ceph-mon[22098]: Cluster is now healthy 2024-08-19T01:46:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:14 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3343948719' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:46:14.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:14 smithi146 ceph-mon[22098]: fsmap 12 up:standby 2024-08-19T01:46:14.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:14 smithi190 ceph-mon[22032]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-19T01:46:14.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:14 smithi190 ceph-mon[22032]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-19T01:46:14.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:14 smithi190 ceph-mon[22032]: Cluster is now healthy 2024-08-19T01:46:14.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:14 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3343948719' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-19T01:46:14.631 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:14 smithi190 ceph-mon[22032]: fsmap 12 up:standby 2024-08-19T01:46:14.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 -- 172.21.15.103:0/1318006648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc541078b0 msgr2=0x7fdc54107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:14.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/1318006648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc541078b0 0x7fdc54107c90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009930 tx=0x7fdc5002a420 comp rx=0 tx=0).stop 2024-08-19T01:46:14.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 -- 172.21.15.103:0/1318006648 shutdown_connections 2024-08-19T01:46:14.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/1318006648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc54102250 0x7fdc5410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:14.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/1318006648 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc541018b0 0x7fdc54101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:14.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/1318006648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc541078b0 0x7fdc54107c90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:14.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 -- 172.21.15.103:0/1318006648 >> 172.21.15.103:0/1318006648 conn(0x7fdc540fd5f0 msgr2=0x7fdc540ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 -- 172.21.15.103:0/1318006648 shutdown_connections 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.625+0000 7fdc5c38f640 1 -- 172.21.15.103:0/1318006648 wait complete. 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 Processor -- start 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 -- start start 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc541018b0 0x7fdc54074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc54102250 0x7fdc54074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc541078b0 0x7fdc54074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54111b20 con 0x7fdc54102250 2024-08-19T01:46:14.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdc541119a0 con 0x7fdc541078b0 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5c38f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54111ca0 con 0x7fdc541018b0 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc54102250 0x7fdc54074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc54102250 0x7fdc54074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35296/0 (socket says 172.21.15.103:35296) 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 -- 172.21.15.103:0/2494582341 learned_addr learned my addr 172.21.15.103:0/2494582341 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5a12b640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc541018b0 0x7fdc54074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5a92c640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc541078b0 0x7fdc54074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 -- 172.21.15.103:0/2494582341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc541018b0 msgr2=0x7fdc54074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc541018b0 0x7fdc54074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:14.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 -- 172.21.15.103:0/2494582341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc541078b0 msgr2=0x7fdc54074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc541078b0 0x7fdc54074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.626+0000 7fdc5992a640 1 -- 172.21.15.103:0/2494582341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc541a6490 con 0x7fdc54102250 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.627+0000 7fdc5992a640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc54102250 0x7fdc54074760 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c920 tx=0x7fdc4400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.627+0000 7fdc4b7fe640 1 -- 172.21.15.103:0/2494582341 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44007c70 con 0x7fdc54102250 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.627+0000 7fdc4b7fe640 1 -- 172.21.15.103:0/2494582341 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc44007e10 con 0x7fdc54102250 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.627+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc541a6780 con 0x7fdc54102250 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.627+0000 7fdc4b7fe640 1 -- 172.21.15.103:0/2494582341 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44018820 con 0x7fdc54102250 2024-08-19T01:46:14.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.627+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc541a6d10 con 0x7fdc54102250 2024-08-19T01:46:14.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.628+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc54101d10 con 0x7fdc54102250 2024-08-19T01:46:14.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.629+0000 7fdc4b7fe640 1 -- 172.21.15.103:0/2494582341 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdc440189c0 con 0x7fdc54102250 2024-08-19T01:46:14.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.630+0000 7fdc4b7fe640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc400a2c00 0x7fdc400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:14.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.630+0000 7fdc5a12b640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc400a2c00 0x7fdc400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:14.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.630+0000 7fdc4b7fe640 1 -- 172.21.15.103:0/2494582341 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8620+0+0 (secure 0 0 0) 0x7fdc440cf140 con 0x7fdc54102250 2024-08-19T01:46:14.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.630+0000 7fdc5a12b640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc400a2c00 0x7fdc400a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006010 tx=0x7fdc50000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:14.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.634+0000 7fdc4b7fe640 1 -- 172.21.15.103:0/2494582341 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc44014030 con 0x7fdc54102250 2024-08-19T01:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:14.801+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdc54109e20 con 0x7fdc54102250 2024-08-19T01:46:15.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.143+0000 7fdc4b7fe640 1 -- 172.21.15.103:0/2494582341 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) ==== 181+0+0 (secure 0 0 0) 0x7fdc4409bfc0 con 0x7fdc54102250 2024-08-19T01:46:15.145 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' removed 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc400a2c00 msgr2=0x7fdc400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc400a2c00 0x7fdc400a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006010 tx=0x7fdc50000bc0 comp rx=0 tx=0).stop 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc54102250 msgr2=0x7fdc54074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc54102250 0x7fdc54074760 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c920 tx=0x7fdc4400cde0 comp rx=0 tx=0).stop 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 shutdown_connections 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7fdc400a2c00 0x7fdc400a50c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdc541078b0 0x7fdc54074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:15.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc54102250 0x7fdc54074760 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:15.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 --2- 172.21.15.103:0/2494582341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc541018b0 0x7fdc54074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:15.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 >> 172.21.15.103:0/2494582341 conn(0x7fdc540fd5f0 msgr2=0x7fdc540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:15.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.147+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 shutdown_connections 2024-08-19T01:46:15.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:15.148+0000 7fdc5c38f640 1 -- 172.21.15.103:0/2494582341 wait complete. 2024-08-19T01:46:15.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:15 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2494582341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:46:15.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:15 smithi146 ceph-mon[22098]: pgmap v654: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-08-19T01:46:15.523 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:15 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2494582341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:46:15.523 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:15 smithi103 ceph-mon[18436]: pgmap v654: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-08-19T01:46:15.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:15 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2494582341' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:46:15.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:15 smithi190 ceph-mon[22032]: pgmap v654: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2024-08-19T01:46:15.888 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid e721016a-5dc9-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-19T01:46:16.138 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/mon.a/config 2024-08-19T01:46:16.405 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:16 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/2494582341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:46:16.405 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:16 smithi103 ceph-mon[18436]: osdmap e88: 12 total, 12 up, 12 in 2024-08-19T01:46:16.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:16 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/2494582341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:46:16.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:16 smithi146 ceph-mon[22098]: osdmap e88: 12 total, 12 up, 12 in 2024-08-19T01:46:16.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:16 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/2494582341' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:46:16.630 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:16 smithi190 ceph-mon[22032]: osdmap e88: 12 total, 12 up, 12 in 2024-08-19T01:46:16.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 -- 172.21.15.103:0/1659364629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581067b0 msgr2=0x7f4758106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:16.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 --2- 172.21.15.103:0/1659364629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581067b0 0x7f4758106b90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4748009930 tx=0x7f474802a420 comp rx=0 tx=0).stop 2024-08-19T01:46:16.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 -- 172.21.15.103:0/1659364629 shutdown_connections 2024-08-19T01:46:16.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 --2- 172.21.15.103:0/1659364629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4758101150 0x7f475810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:16.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 --2- 172.21.15.103:0/1659364629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47581007b0 0x7f4758100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:16.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 --2- 172.21.15.103:0/1659364629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581067b0 0x7f4758106b90 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:16.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 -- 172.21.15.103:0/1659364629 >> 172.21.15.103:0/1659364629 conn(0x7f47580fc4d0 msgr2=0x7f47580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 -- 172.21.15.103:0/1659364629 shutdown_connections 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.627+0000 7f47601c3640 1 -- 172.21.15.103:0/1659364629 wait complete. 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 Processor -- start 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 -- start start 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581007b0 0x7f4758198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4758101150 0x7f47581993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47581067b0 0x7f47581a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:16.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4758110a60 con 0x7f47581007b0 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47581108e0 con 0x7f4758101150 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f47601c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4758110be0 con 0x7f47581067b0 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581007b0 0x7f4758198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581007b0 0x7f4758198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35314/0 (socket says 172.21.15.103:35314) 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 -- 172.21.15.103:0/3962857935 learned_addr learned my addr 172.21.15.103:0/3962857935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475e760640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47581067b0 0x7f47581a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475d75e640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4758101150 0x7f47581993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 -- 172.21.15.103:0/3962857935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47581067b0 msgr2=0x7f47581a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47581067b0 0x7f47581a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 -- 172.21.15.103:0/3962857935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4758101150 msgr2=0x7f47581993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4758101150 0x7f47581993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.628+0000 7f475df5f640 1 -- 172.21.15.103:0/3962857935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47581a4470 con 0x7f47581007b0 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.629+0000 7f475df5f640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581007b0 0x7f4758198eb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4748004770 tx=0x7f47480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.629+0000 7f4756ffd640 1 -- 172.21.15.103:0/3962857935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474803f070 con 0x7f47581007b0 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.629+0000 7f4756ffd640 1 -- 172.21.15.103:0/3962857935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f474803b460 con 0x7f47581007b0 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.629+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47581a4640 con 0x7f47581007b0 2024-08-19T01:46:16.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.629+0000 7f4756ffd640 1 -- 172.21.15.103:0/3962857935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748035070 con 0x7f47581007b0 2024-08-19T01:46:16.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.629+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47581a4af0 con 0x7f47581007b0 2024-08-19T01:46:16.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.630+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4758100c10 con 0x7f47581007b0 2024-08-19T01:46:16.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.631+0000 7f4756ffd640 1 -- 172.21.15.103:0/3962857935 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f474803b610 con 0x7f47581007b0 2024-08-19T01:46:16.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.631+0000 7f4756ffd640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47400a2c00 0x7f47400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-19T01:46:16.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.632+0000 7f475d75e640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47400a2c00 0x7f47400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-19T01:46:16.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.632+0000 7f4756ffd640 1 -- 172.21.15.103:0/3962857935 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8087+0+0 (secure 0 0 0) 0x7f47480ef280 con 0x7f47581007b0 2024-08-19T01:46:16.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.632+0000 7f475d75e640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47400a2c00 0x7f47400a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f474c005fd0 tx=0x7f474c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-19T01:46:16.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.636+0000 7f4756ffd640 1 -- 172.21.15.103:0/3962857935 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f47480bc2c0 con 0x7f47581007b0 2024-08-19T01:46:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:16.809+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4758108d20 con 0x7f47581007b0 2024-08-19T01:46:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.150+0000 7f4756ffd640 1 -- 172.21.15.103:0/3962857935 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) ==== 169+0+0 (secure 0 0 0) 0x7f47480bbbc0 con 0x7f47581007b0 2024-08-19T01:46:17.152 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' removed 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.153+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47400a2c00 msgr2=0x7f47400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.153+0000 7f47601c3640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47400a2c00 0x7f47400a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f474c005fd0 tx=0x7f474c005850 comp rx=0 tx=0).stop 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.153+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581007b0 msgr2=0x7f4758198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581007b0 0x7f4758198eb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4748004770 tx=0x7f47480042f0 comp rx=0 tx=0).stop 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 shutdown_connections 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:6800/2656970648,v1:172.21.15.103:6801/2656970648] conn(0x7f47400a2c00 0x7f47400a50c0 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f474c005fd0 tx=0x7f474c005850 comp rx=0 tx=0).stop 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47581067b0 0x7f47581a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:17.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4758101150 0x7f47581993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:17.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 --2- 172.21.15.103:0/3962857935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47581007b0 0x7f4758198eb0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-19T01:46:17.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 >> 172.21.15.103:0/3962857935 conn(0x7f47580fc4d0 msgr2=0x7f47580fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-19T01:46:17.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 shutdown_connections 2024-08-19T01:46:17.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-08-19T01:46:17.154+0000 7f47601c3640 1 -- 172.21.15.103:0/3962857935 wait complete. 2024-08-19T01:46:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:17 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3962857935' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:46:17.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:17 smithi146 ceph-mon[22098]: pgmap v656: 65 pgs: 65 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:46:17.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:17 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3962857935' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:46:17.532 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:17 smithi190 ceph-mon[22032]: pgmap v656: 65 pgs: 65 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:46:17.551 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:17 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3962857935' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-19T01:46:17.552 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:17 smithi103 ceph-mon[18436]: pgmap v656: 65 pgs: 65 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-19T01:46:17.922 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:46:17.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:46:18.027 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:46:18.079 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-19T01:46:18.079 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:46:18.110 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:46:18.140 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-19T01:46:18.176 INFO:tasks.cephadm:Stopping all daemons... 2024-08-19T01:46:18.176 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-19T01:46:18.176 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a 2024-08-19T01:46:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:18 smithi190 ceph-mon[22032]: from='client.? 172.21.15.103:0/3962857935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:46:18.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:18 smithi190 ceph-mon[22032]: osdmap e89: 12 total, 12 up, 12 in 2024-08-19T01:46:18.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:18 smithi146 ceph-mon[22098]: from='client.? 172.21.15.103:0/3962857935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:46:18.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:18 smithi146 ceph-mon[22098]: osdmap e89: 12 total, 12 up, 12 in 2024-08-19T01:46:18.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:18 smithi103 ceph-mon[18436]: from='client.? 172.21.15.103:0/3962857935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-19T01:46:18.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:18 smithi103 ceph-mon[18436]: osdmap e89: 12 total, 12 up, 12 in 2024-08-19T01:46:18.555 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:18 smithi103 systemd[1]: Stopping Ceph mon.a for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:18.983 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:18 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a[18432]: 2024-08-19T01:46:18.715+0000 7f4542596640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:18.983 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:18 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a[18432]: 2024-08-19T01:46:18.715+0000 7f4542596640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-19T01:46:18.984 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:18 smithi103 podman[107628]: 2024-08-19 01:46:18.759308142 +0000 UTC m=+0.188700110 container died a8f5c18bf93266b7af249d16335fa092d47e692d65fcf542f5bc308827f2e413 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-19T01:46:19.553 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:19 smithi103 podman[107628]: 2024-08-19 01:46:19.284055767 +0000 UTC m=+0.713447732 container cleanup a8f5c18bf93266b7af249d16335fa092d47e692d65fcf542f5bc308827f2e413 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-19T01:46:19.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:19 smithi103 bash[107628]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a 2024-08-19T01:46:19.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Aug 19 01:46:19 smithi103 podman[107638]: 2024-08-19 01:46:19.511346483 +0000 UTC m=+0.744656751 container remove a8f5c18bf93266b7af249d16335fa092d47e692d65fcf542f5bc308827f2e413 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-19T01:46:19.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.a.service' 2024-08-19T01:46:19.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:19.926 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-19T01:46:19.926 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-19T01:46:19.926 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.b 2024-08-19T01:46:20.437 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:19 smithi146 systemd[1]: Stopping Ceph mon.b for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:20.700 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:20 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-b[22094]: 2024-08-19T01:46:20.435+0000 7f0164a61640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:20.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:20 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-b[22094]: 2024-08-19T01:46:20.435+0000 7f0164a61640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-19T01:46:20.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:20 smithi146 podman[56611]: 2024-08-19 01:46:20.507195017 +0000 UTC m=+0.226102199 container died 1bfba6914e0533316429ae5cfa778a08db61b1fd2862c31619f2ef36070faead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-de09dd0) 2024-08-19T01:46:22.700 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:22 smithi146 podman[56611]: 2024-08-19 01:46:22.394348886 +0000 UTC m=+2.113256051 container cleanup 1bfba6914e0533316429ae5cfa778a08db61b1fd2862c31619f2ef36070faead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-19T01:46:22.701 INFO:journalctl@ceph.mon.b.smithi146.stdout:Aug 19 01:46:22 smithi146 bash[56611]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-b 2024-08-19T01:46:25.372 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.b.service' 2024-08-19T01:46:25.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:25.510 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-19T01:46:25.510 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-19T01:46:25.510 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.c 2024-08-19T01:46:25.880 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:25 smithi190 systemd[1]: Stopping Ceph mon.c for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:27.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:26 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-c[22028]: 2024-08-19T01:46:26.797+0000 7fd33674c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:27.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:26 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-c[22028]: 2024-08-19T01:46:26.797+0000 7fd33674c640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-19T01:46:27.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:26 smithi190 podman[57200]: 2024-08-19 01:46:26.818913461 +0000 UTC m=+0.913712230 container died 2255ac56c6eb1d7ec9302d80064acc97af52f67022773bce45967bd70ca100eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-19T01:46:29.130 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:28 smithi190 podman[57200]: 2024-08-19 01:46:28.890637348 +0000 UTC m=+2.985436111 container cleanup 2255ac56c6eb1d7ec9302d80064acc97af52f67022773bce45967bd70ca100eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-08-19T01:46:29.131 INFO:journalctl@ceph.mon.c.smithi190.stdout:Aug 19 01:46:28 smithi190 bash[57200]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mon-c 2024-08-19T01:46:29.328 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mon.c.service' 2024-08-19T01:46:29.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:29.378 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-19T01:46:29.379 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-19T01:46:29.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.x 2024-08-19T01:46:29.803 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:46:29 smithi103 systemd[1]: Stopping Ceph mgr.x for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:31.304 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:46:30 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:46:30.838+0000 7f4ff479b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:31.304 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:46:30 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x[18664]: 2024-08-19T01:46:30.838+0000 7f4ff479b640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-08-19T01:46:31.304 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:46:30 smithi103 podman[107744]: 2024-08-19 01:46:30.880102604 +0000 UTC m=+1.299454228 container died acaf1c3cc543f30c208422422017158e8d3b64bf255cd11c3f2800191007d5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-08-19T01:46:34.553 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:46:34 smithi103 podman[107744]: 2024-08-19 01:46:34.20325272 +0000 UTC m=+4.622604383 container cleanup acaf1c3cc543f30c208422422017158e8d3b64bf255cd11c3f2800191007d5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-19T01:46:34.554 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:46:34 smithi103 bash[107744]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x 2024-08-19T01:46:34.554 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Aug 19 01:46:34 smithi103 podman[107758]: 2024-08-19 01:46:34.447319735 +0000 UTC m=+3.486157585 container remove acaf1c3cc543f30c208422422017158e8d3b64bf255cd11c3f2800191007d5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-19T01:46:34.603 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.x.service' 2024-08-19T01:46:34.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:34.639 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-19T01:46:34.639 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-19T01:46:34.639 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.y 2024-08-19T01:46:34.950 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:46:34 smithi146 systemd[1]: Stopping Ceph mgr.y for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:35.277 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:46:34 smithi146 podman[56731]: 2024-08-19 01:46:34.99576533 +0000 UTC m=+0.152644275 container died 9cc949ff57f8321b5f79659fb6e70f437a960910666c0b7e0bcac78ec25d58e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-19T01:46:35.683 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:46:35 smithi146 podman[56731]: 2024-08-19 01:46:35.375215219 +0000 UTC m=+0.532094164 container cleanup 9cc949ff57f8321b5f79659fb6e70f437a960910666c0b7e0bcac78ec25d58e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-19T01:46:35.684 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Aug 19 01:46:35 smithi146 bash[56731]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-y 2024-08-19T01:46:35.858 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.y.service' 2024-08-19T01:46:35.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:35.893 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-19T01:46:35.893 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-19T01:46:35.893 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.z 2024-08-19T01:46:36.217 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:46:35 smithi190 systemd[1]: Stopping Ceph mgr.z for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:36.497 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:46:36 smithi190 podman[57326]: 2024-08-19 01:46:36.216779225 +0000 UTC m=+0.118772733 container died 1841274b263796dd4a436a523834aa664071dc54d4ba502950a137dd087d3688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-19T01:46:36.880 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:46:36 smithi190 podman[57326]: 2024-08-19 01:46:36.696157513 +0000 UTC m=+0.598151020 container cleanup 1841274b263796dd4a436a523834aa664071dc54d4ba502950a137dd087d3688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:46:36.880 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Aug 19 01:46:36 smithi190 bash[57326]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mgr-z 2024-08-19T01:46:37.135 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mgr.z.service' 2024-08-19T01:46:37.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:37.170 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-19T01:46:37.170 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-19T01:46:37.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.0 2024-08-19T01:46:37.494 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:37 smithi103 systemd[1]: Stopping Ceph osd.0 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:37.803 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:37 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0[31822]: 2024-08-19T01:46:37.493+0000 7f728e324640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:37.803 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:37 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0[31822]: 2024-08-19T01:46:37.493+0000 7f728e324640 -1 osd.0 89 *** Got signal Terminated *** 2024-08-19T01:46:37.804 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:37 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0[31822]: 2024-08-19T01:46:37.493+0000 7f728e324640 -1 osd.0 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:46:43.303 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:42 smithi103 podman[107878]: 2024-08-19 01:46:42.953060702 +0000 UTC m=+5.584617636 container died a588ae4601b56dd63a3ce5cbb3c2bd3ba8f37857b38c399d7e7986ea79e8a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-19T01:46:43.803 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:43 smithi103 podman[107878]: 2024-08-19 01:46:43.510840054 +0000 UTC m=+6.142397032 container cleanup a588ae4601b56dd63a3ce5cbb3c2bd3ba8f37857b38c399d7e7986ea79e8a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-19T01:46:43.804 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:43 smithi103 bash[107878]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0 2024-08-19T01:46:44.199 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:43 smithi103 podman[107898]: 2024-08-19 01:46:43.83809973 +0000 UTC m=+0.881967307 container remove a588ae4601b56dd63a3ce5cbb3c2bd3ba8f37857b38c399d7e7986ea79e8a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-19T01:46:44.492 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:44 smithi103 podman[107983]: 2024-08-19 01:46:44.101984229 +0000 UTC m=+0.018153800 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:46:44.493 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:44 smithi103 podman[107983]: 2024-08-19 01:46:44.222632623 +0000 UTC m=+0.138802241 container create 2d1125dcba3cb30abd238084385f3ffcf36af3a7e8194151f0d01ada1a366ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-19T01:46:44.805 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:44 smithi103 podman[107983]: 2024-08-19 01:46:44.492555867 +0000 UTC m=+0.408725457 container init 2d1125dcba3cb30abd238084385f3ffcf36af3a7e8194151f0d01ada1a366ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-19T01:46:44.805 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:44 smithi103 podman[107983]: 2024-08-19 01:46:44.496098367 +0000 UTC m=+0.412267940 container start 2d1125dcba3cb30abd238084385f3ffcf36af3a7e8194151f0d01ada1a366ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-19T01:46:44.805 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:44 smithi103 podman[107983]: 2024-08-19 01:46:44.609426712 +0000 UTC m=+0.525596298 container attach 2d1125dcba3cb30abd238084385f3ffcf36af3a7e8194151f0d01ada1a366ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:46:45.222 INFO:journalctl@ceph.osd.0.smithi103.stdout:Aug 19 01:46:44 smithi103 podman[107983]: 2024-08-19 01:46:44.910202414 +0000 UTC m=+0.826371989 container died 2d1125dcba3cb30abd238084385f3ffcf36af3a7e8194151f0d01ada1a366ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-19T01:46:45.562 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.0.service' 2024-08-19T01:46:45.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:45.604 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-19T01:46:45.604 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-19T01:46:45.605 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.1 2024-08-19T01:46:45.913 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:45 smithi146 systemd[1]: Stopping Ceph osd.1 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:46.200 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:45 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1[26554]: 2024-08-19T01:46:45.912+0000 7fa0a2b69640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:46.200 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:45 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1[26554]: 2024-08-19T01:46:45.912+0000 7fa0a2b69640 -1 osd.1 89 *** Got signal Terminated *** 2024-08-19T01:46:46.201 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:45 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1[26554]: 2024-08-19T01:46:45.912+0000 7fa0a2b69640 -1 osd.1 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:46:51.605 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:51 smithi146 podman[56853]: 2024-08-19 01:46:51.284861212 +0000 UTC m=+5.480723141 container died 9c47197e15747e780c8fdfa051786d26ecd5ac54af47cf4411ea27cade5edcda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0) 2024-08-19T01:46:51.931 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:51 smithi146 podman[56853]: 2024-08-19 01:46:51.704030399 +0000 UTC m=+5.899892334 container cleanup 9c47197e15747e780c8fdfa051786d26ecd5ac54af47cf4411ea27cade5edcda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-19T01:46:51.931 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:51 smithi146 bash[56853]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1 2024-08-19T01:46:52.200 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:51 smithi146 podman[56870]: 2024-08-19 01:46:51.930390838 +0000 UTC m=+0.649063113 container remove 9c47197e15747e780c8fdfa051786d26ecd5ac54af47cf4411ea27cade5edcda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-19T01:46:52.651 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:52 smithi146 podman[56961]: 2024-08-19 01:46:52.249623861 +0000 UTC m=+0.017126606 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:46:52.651 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:52 smithi146 podman[56961]: 2024-08-19 01:46:52.389549639 +0000 UTC m=+0.157052420 container create 1931d05fecbf437cdd7f61945b4f2e406238db6e4cefa16168616ee2f4dc0b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux ) 2024-08-19T01:46:52.651 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:52 smithi146 podman[56961]: 2024-08-19 01:46:52.65073958 +0000 UTC m=+0.418242316 container init 1931d05fecbf437cdd7f61945b4f2e406238db6e4cefa16168616ee2f4dc0b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:46:52.951 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:52 smithi146 podman[56961]: 2024-08-19 01:46:52.65422042 +0000 UTC m=+0.421723156 container start 1931d05fecbf437cdd7f61945b4f2e406238db6e4cefa16168616ee2f4dc0b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:46:52.951 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:52 smithi146 podman[56961]: 2024-08-19 01:46:52.742381125 +0000 UTC m=+0.509883863 container attach 1931d05fecbf437cdd7f61945b4f2e406238db6e4cefa16168616ee2f4dc0b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-19T01:46:52.951 INFO:journalctl@ceph.osd.1.smithi146.stdout:Aug 19 01:46:52 smithi146 podman[56961]: 2024-08-19 01:46:52.843729153 +0000 UTC m=+0.611231903 container died 1931d05fecbf437cdd7f61945b4f2e406238db6e4cefa16168616ee2f4dc0b7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-19T01:46:53.498 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.1.service' 2024-08-19T01:46:53.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:46:53.541 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-19T01:46:53.542 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-19T01:46:53.542 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.2 2024-08-19T01:46:53.880 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:46:53 smithi190 systemd[1]: Stopping Ceph osd.2 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:46:54.134 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:46:53 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2[27008]: 2024-08-19T01:46:53.966+0000 7f307a14d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:46:54.134 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:46:53 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2[27008]: 2024-08-19T01:46:53.966+0000 7f307a14d640 -1 osd.2 89 *** Got signal Terminated *** 2024-08-19T01:46:54.134 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:46:53 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2[27008]: 2024-08-19T01:46:53.966+0000 7f307a14d640 -1 osd.2 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:46:59.760 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:46:59 smithi190 podman[57448]: 2024-08-19 01:46:59.445140361 +0000 UTC m=+5.699832011 container died c050c8f0bcd1a0088186e90fe1c7917d99e1c9fe05afc3c4456299f211b4df8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-19T01:47:00.127 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:46:59 smithi190 podman[57448]: 2024-08-19 01:46:59.858888742 +0000 UTC m=+6.113580392 container cleanup c050c8f0bcd1a0088186e90fe1c7917d99e1c9fe05afc3c4456299f211b4df8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:47:00.127 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:46:59 smithi190 bash[57448]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2 2024-08-19T01:47:00.380 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:47:00 smithi190 podman[57475]: 2024-08-19 01:47:00.127135807 +0000 UTC m=+0.680152023 container remove c050c8f0bcd1a0088186e90fe1c7917d99e1c9fe05afc3c4456299f211b4df8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-19T01:47:00.749 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:47:00 smithi190 podman[57563]: 2024-08-19 01:47:00.369458118 +0000 UTC m=+0.017098435 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:00.749 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:47:00 smithi190 podman[57563]: 2024-08-19 01:47:00.494604734 +0000 UTC m=+0.142245048 container create d4d105caca1937b73b47a6cb5a0d392e19adcad978804f319f45262a5158cea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-19T01:47:01.130 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:47:00 smithi190 podman[57563]: 2024-08-19 01:47:00.748041659 +0000 UTC m=+0.395681978 container init d4d105caca1937b73b47a6cb5a0d392e19adcad978804f319f45262a5158cea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-19T01:47:01.131 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:47:00 smithi190 podman[57563]: 2024-08-19 01:47:00.751727877 +0000 UTC m=+0.399368188 container start d4d105caca1937b73b47a6cb5a0d392e19adcad978804f319f45262a5158cea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-19T01:47:01.131 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:47:00 smithi190 podman[57563]: 2024-08-19 01:47:00.839395676 +0000 UTC m=+0.487035987 container attach d4d105caca1937b73b47a6cb5a0d392e19adcad978804f319f45262a5158cea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-19T01:47:01.131 INFO:journalctl@ceph.osd.2.smithi190.stdout:Aug 19 01:47:00 smithi190 podman[57563]: 2024-08-19 01:47:00.937720498 +0000 UTC m=+0.585360811 container died d4d105caca1937b73b47a6cb5a0d392e19adcad978804f319f45262a5158cea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:47:01.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.2.service' 2024-08-19T01:47:01.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:01.641 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-19T01:47:01.642 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-19T01:47:01.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.3 2024-08-19T01:47:01.985 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:01 smithi103 systemd[1]: Stopping Ceph osd.3 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:02.303 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:01 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3[38661]: 2024-08-19T01:47:01.983+0000 7f7fa01c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:02.304 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:01 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3[38661]: 2024-08-19T01:47:01.983+0000 7f7fa01c3640 -1 osd.3 89 *** Got signal Terminated *** 2024-08-19T01:47:02.304 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:01 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3[38661]: 2024-08-19T01:47:01.983+0000 7f7fa01c3640 -1 osd.3 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:07.803 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:07 smithi103 podman[108084]: 2024-08-19 01:47:07.514290265 +0000 UTC m=+5.666598202 container died 20d6f2926537290bc23f80748d4f203caee4fb371a3eb4e6520fa5925df0dded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-19T01:47:08.254 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:07 smithi103 podman[108084]: 2024-08-19 01:47:07.969243144 +0000 UTC m=+6.121551072 container cleanup 20d6f2926537290bc23f80748d4f203caee4fb371a3eb4e6520fa5925df0dded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-19T01:47:08.254 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:07 smithi103 bash[108084]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3 2024-08-19T01:47:08.553 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:08 smithi103 podman[108100]: 2024-08-19 01:47:08.253901855 +0000 UTC m=+0.737593053 container remove 20d6f2926537290bc23f80748d4f203caee4fb371a3eb4e6520fa5925df0dded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:47:08.858 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:08 smithi103 podman[108187]: 2024-08-19 01:47:08.47835752 +0000 UTC m=+0.016852461 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:08.858 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:08 smithi103 podman[108187]: 2024-08-19 01:47:08.605055247 +0000 UTC m=+0.143550183 container create e7718c2fc3e5c2f7d63a06a465a24f76de48428bf4adb5e0421942d1d81f0087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-19T01:47:08.858 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:08 smithi103 podman[108187]: 2024-08-19 01:47:08.857891434 +0000 UTC m=+0.396386380 container init e7718c2fc3e5c2f7d63a06a465a24f76de48428bf4adb5e0421942d1d81f0087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:47:09.303 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:08 smithi103 podman[108187]: 2024-08-19 01:47:08.861296227 +0000 UTC m=+0.399791168 container start e7718c2fc3e5c2f7d63a06a465a24f76de48428bf4adb5e0421942d1d81f0087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:47:09.304 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:08 smithi103 podman[108187]: 2024-08-19 01:47:08.966541923 +0000 UTC m=+0.505036858 container attach e7718c2fc3e5c2f7d63a06a465a24f76de48428bf4adb5e0421942d1d81f0087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-19T01:47:09.304 INFO:journalctl@ceph.osd.3.smithi103.stdout:Aug 19 01:47:09 smithi103 podman[108187]: 2024-08-19 01:47:09.045649678 +0000 UTC m=+0.584144615 container died e7718c2fc3e5c2f7d63a06a465a24f76de48428bf4adb5e0421942d1d81f0087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-08-19T01:47:09.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.3.service' 2024-08-19T01:47:09.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:09.738 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-19T01:47:09.739 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-19T01:47:09.739 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.4 2024-08-19T01:47:10.074 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:09 smithi146 systemd[1]: Stopping Ceph osd.4 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:10.450 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:10 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4[33286]: 2024-08-19T01:47:10.073+0000 7fd782003640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:10.451 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:10 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4[33286]: 2024-08-19T01:47:10.073+0000 7fd782003640 -1 osd.4 89 *** Got signal Terminated *** 2024-08-19T01:47:10.451 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:10 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4[33286]: 2024-08-19T01:47:10.073+0000 7fd782003640 -1 osd.4 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:15.849 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:15 smithi146 podman[57077]: 2024-08-19 01:47:15.576800251 +0000 UTC m=+5.627540920 container died 7f161ef1fde566598545e2d727e88bdf9333b4079f38c735245790180a26f1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:47:16.200 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:15 smithi146 podman[57077]: 2024-08-19 01:47:15.940007429 +0000 UTC m=+5.990748095 container cleanup 7f161ef1fde566598545e2d727e88bdf9333b4079f38c735245790180a26f1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-19T01:47:16.201 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:15 smithi146 bash[57077]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4 2024-08-19T01:47:16.620 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:16 smithi146 podman[57094]: 2024-08-19 01:47:16.249695791 +0000 UTC m=+0.668833350 container remove 7f161ef1fde566598545e2d727e88bdf9333b4079f38c735245790180a26f1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-19T01:47:16.945 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:16 smithi146 podman[57185]: 2024-08-19 01:47:16.52274759 +0000 UTC m=+0.017064014 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:16.946 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:16 smithi146 podman[57185]: 2024-08-19 01:47:16.658922243 +0000 UTC m=+0.153238648 container create d2ba0592666852f5c4411ed52802db40248f16b0bf90ca2f0a28feb98f329e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-19T01:47:16.946 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:16 smithi146 podman[57185]: 2024-08-19 01:47:16.861457826 +0000 UTC m=+0.355774245 container init d2ba0592666852f5c4411ed52802db40248f16b0bf90ca2f0a28feb98f329e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0) 2024-08-19T01:47:16.946 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:16 smithi146 podman[57185]: 2024-08-19 01:47:16.865082885 +0000 UTC m=+0.359399290 container start d2ba0592666852f5c4411ed52802db40248f16b0bf90ca2f0a28feb98f329e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-19T01:47:16.946 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:16 smithi146 podman[57185]: 2024-08-19 01:47:16.944576744 +0000 UTC m=+0.438893165 container attach d2ba0592666852f5c4411ed52802db40248f16b0bf90ca2f0a28feb98f329e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0) 2024-08-19T01:47:17.200 INFO:journalctl@ceph.osd.4.smithi146.stdout:Aug 19 01:47:17 smithi146 podman[57185]: 2024-08-19 01:47:17.056667596 +0000 UTC m=+0.550984001 container died d2ba0592666852f5c4411ed52802db40248f16b0bf90ca2f0a28feb98f329e0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-19T01:47:17.671 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.4.service' 2024-08-19T01:47:17.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:17.710 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-19T01:47:17.710 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-19T01:47:17.710 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.5 2024-08-19T01:47:18.038 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:17 smithi190 systemd[1]: Stopping Ceph osd.5 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:18.380 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:18 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5[33435]: 2024-08-19T01:47:18.037+0000 7f2b4cf7b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:18.380 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:18 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5[33435]: 2024-08-19T01:47:18.037+0000 7f2b4cf7b640 -1 osd.5 89 *** Got signal Terminated *** 2024-08-19T01:47:18.380 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:18 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5[33435]: 2024-08-19T01:47:18.037+0000 7f2b4cf7b640 -1 osd.5 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:23.881 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:23 smithi190 podman[57668]: 2024-08-19 01:47:23.583737509 +0000 UTC m=+5.676028456 container died 08ee83d26a5890d4c3eb30479cec0ef2c13ebd88e445d03b43a5ed5e6df3321f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-19T01:47:24.222 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:24 smithi190 podman[57668]: 2024-08-19 01:47:24.003761142 +0000 UTC m=+6.096052080 container cleanup 08ee83d26a5890d4c3eb30479cec0ef2c13ebd88e445d03b43a5ed5e6df3321f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:47:24.222 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:24 smithi190 bash[57668]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5 2024-08-19T01:47:24.630 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:24 smithi190 podman[57685]: 2024-08-19 01:47:24.2470293 +0000 UTC m=+0.661025807 container remove 08ee83d26a5890d4c3eb30479cec0ef2c13ebd88e445d03b43a5ed5e6df3321f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-19T01:47:25.018 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:24 smithi190 podman[57783]: 2024-08-19 01:47:24.616088973 +0000 UTC m=+0.017508100 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:25.018 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:24 smithi190 podman[57783]: 2024-08-19 01:47:24.747776711 +0000 UTC m=+0.149195834 container create 61dda73dc90be291cc1cee597ab5f0a664232ea1bb6b85c3ddc5f953a056c26b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-de09dd0) 2024-08-19T01:47:25.380 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:25 smithi190 podman[57783]: 2024-08-19 01:47:25.017316496 +0000 UTC m=+0.418735614 container init 61dda73dc90be291cc1cee597ab5f0a664232ea1bb6b85c3ddc5f953a056c26b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:47:25.380 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:25 smithi190 podman[57783]: 2024-08-19 01:47:25.021035402 +0000 UTC m=+0.422454527 container start 61dda73dc90be291cc1cee597ab5f0a664232ea1bb6b85c3ddc5f953a056c26b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-19T01:47:25.381 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:25 smithi190 podman[57783]: 2024-08-19 01:47:25.134190526 +0000 UTC m=+0.535609646 container attach 61dda73dc90be291cc1cee597ab5f0a664232ea1bb6b85c3ddc5f953a056c26b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-08-19T01:47:25.381 INFO:journalctl@ceph.osd.5.smithi190.stdout:Aug 19 01:47:25 smithi190 podman[57783]: 2024-08-19 01:47:25.21201663 +0000 UTC m=+0.613435765 container died 61dda73dc90be291cc1cee597ab5f0a664232ea1bb6b85c3ddc5f953a056c26b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True) 2024-08-19T01:47:25.890 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.5.service' 2024-08-19T01:47:25.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:25.931 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-19T01:47:25.931 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-19T01:47:25.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.6 2024-08-19T01:47:26.258 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:25 smithi103 systemd[1]: Stopping Ceph osd.6 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:26.553 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:26 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6[45101]: 2024-08-19T01:47:26.257+0000 7f16088aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:26.554 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:26 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6[45101]: 2024-08-19T01:47:26.257+0000 7f16088aa640 -1 osd.6 89 *** Got signal Terminated *** 2024-08-19T01:47:26.554 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:26 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6[45101]: 2024-08-19T01:47:26.257+0000 7f16088aa640 -1 osd.6 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:32.134 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:31 smithi103 podman[108290]: 2024-08-19 01:47:31.832780774 +0000 UTC m=+5.701624215 container died 7ec074bab484bfd2d17460346190052b69e940fb29704b9cb3f9cb5ddc3e7bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, ceph=True) 2024-08-19T01:47:32.553 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:32 smithi103 podman[108290]: 2024-08-19 01:47:32.316722277 +0000 UTC m=+6.185565770 container cleanup 7ec074bab484bfd2d17460346190052b69e940fb29704b9cb3f9cb5ddc3e7bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=) 2024-08-19T01:47:32.554 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:32 smithi103 bash[108290]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6 2024-08-19T01:47:32.931 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:32 smithi103 podman[108308]: 2024-08-19 01:47:32.609794442 +0000 UTC m=+0.766457314 container remove 7ec074bab484bfd2d17460346190052b69e940fb29704b9cb3f9cb5ddc3e7bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-08-19T01:47:33.184 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:32 smithi103 podman[108395]: 2024-08-19 01:47:32.833059711 +0000 UTC m=+0.017477375 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:33.184 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:32 smithi103 podman[108395]: 2024-08-19 01:47:32.93570663 +0000 UTC m=+0.120124291 container create b061c0305f18411172c4f875cd6a555cf6edea271e14d341dfa11acf32a8c6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-19T01:47:33.184 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:33 smithi103 podman[108395]: 2024-08-19 01:47:33.180344615 +0000 UTC m=+0.364762265 container init b061c0305f18411172c4f875cd6a555cf6edea271e14d341dfa11acf32a8c6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-19T01:47:33.184 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:33 smithi103 podman[108395]: 2024-08-19 01:47:33.183556019 +0000 UTC m=+0.367973670 container start b061c0305f18411172c4f875cd6a555cf6edea271e14d341dfa11acf32a8c6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-19T01:47:33.553 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:33 smithi103 podman[108395]: 2024-08-19 01:47:33.288782793 +0000 UTC m=+0.473200464 container attach b061c0305f18411172c4f875cd6a555cf6edea271e14d341dfa11acf32a8c6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-19T01:47:33.554 INFO:journalctl@ceph.osd.6.smithi103.stdout:Aug 19 01:47:33 smithi103 podman[108395]: 2024-08-19 01:47:33.369726764 +0000 UTC m=+0.554144425 container died b061c0305f18411172c4f875cd6a555cf6edea271e14d341dfa11acf32a8c6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0) 2024-08-19T01:47:34.099 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.6.service' 2024-08-19T01:47:34.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:34.140 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-19T01:47:34.140 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-19T01:47:34.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.7 2024-08-19T01:47:34.451 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:34 smithi146 systemd[1]: Stopping Ceph osd.7 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:34.712 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:34 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7[40169]: 2024-08-19T01:47:34.516+0000 7fae7ece2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:34.712 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:34 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7[40169]: 2024-08-19T01:47:34.516+0000 7fae7ece2640 -1 osd.7 89 *** Got signal Terminated *** 2024-08-19T01:47:34.712 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:34 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7[40169]: 2024-08-19T01:47:34.516+0000 7fae7ece2640 -1 osd.7 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:40.301 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:39 smithi146 podman[57288]: 2024-08-19 01:47:39.992496853 +0000 UTC m=+5.654431100 container died 5a2ac26c5531e4654303c67a1e637472d046ba2cd995c0b1353817b0bc1d2ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-19T01:47:40.659 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:40 smithi146 podman[57288]: 2024-08-19 01:47:40.399218079 +0000 UTC m=+6.061152326 container cleanup 5a2ac26c5531e4654303c67a1e637472d046ba2cd995c0b1353817b0bc1d2ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-19T01:47:40.659 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:40 smithi146 bash[57288]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7 2024-08-19T01:47:40.950 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:40 smithi146 podman[57317]: 2024-08-19 01:47:40.658669903 +0000 UTC m=+0.661207042 container remove 5a2ac26c5531e4654303c67a1e637472d046ba2cd995c0b1353817b0bc1d2ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:47:41.278 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:41 smithi146 podman[57405]: 2024-08-19 01:47:40.926976708 +0000 UTC m=+0.017089282 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:41.279 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:41 smithi146 podman[57405]: 2024-08-19 01:47:41.034462924 +0000 UTC m=+0.124575483 container create 62a9880a3961f700a7109a42e06382e598b277acd356e11794f504bdb334cd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0) 2024-08-19T01:47:41.279 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:41 smithi146 podman[57405]: 2024-08-19 01:47:41.278359063 +0000 UTC m=+0.368471619 container init 62a9880a3961f700a7109a42e06382e598b277acd356e11794f504bdb334cd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0) 2024-08-19T01:47:41.700 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:41 smithi146 podman[57405]: 2024-08-19 01:47:41.281725459 +0000 UTC m=+0.371838025 container start 62a9880a3961f700a7109a42e06382e598b277acd356e11794f504bdb334cd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-08-19T01:47:41.701 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:41 smithi146 podman[57405]: 2024-08-19 01:47:41.378359288 +0000 UTC m=+0.468471845 container attach 62a9880a3961f700a7109a42e06382e598b277acd356e11794f504bdb334cd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0) 2024-08-19T01:47:41.701 INFO:journalctl@ceph.osd.7.smithi146.stdout:Aug 19 01:47:41 smithi146 podman[57405]: 2024-08-19 01:47:41.470354945 +0000 UTC m=+0.560467503 container died 62a9880a3961f700a7109a42e06382e598b277acd356e11794f504bdb334cd71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-7-deactivate, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-19T01:47:42.170 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.7.service' 2024-08-19T01:47:42.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:42.214 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-19T01:47:42.214 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-19T01:47:42.215 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.8 2024-08-19T01:47:42.541 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:42 smithi190 systemd[1]: Stopping Ceph osd.8 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:42.880 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:42 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8[40344]: 2024-08-19T01:47:42.540+0000 7ff7a3da6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:42.880 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:42 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8[40344]: 2024-08-19T01:47:42.540+0000 7ff7a3da6640 -1 osd.8 89 *** Got signal Terminated *** 2024-08-19T01:47:42.880 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:42 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8[40344]: 2024-08-19T01:47:42.540+0000 7ff7a3da6640 -1 osd.8 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:43.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:43 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:43.597+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:44.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:44 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:44.551+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:45.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:45 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:45.584+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:46.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:46 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:46.550+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:46.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:46 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:46.656+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:47.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:47 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:47.515+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:47.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:47 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:47.675+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:48.216 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:47 smithi190 podman[57886]: 2024-08-19 01:47:47.953112381 +0000 UTC m=+5.545019611 container died 9d9827445358b3fa87d639b294da085734c47bfe8a92bc4a61db323843a09a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-08-19T01:47:48.524 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:48 smithi190 podman[57886]: 2024-08-19 01:47:48.314756233 +0000 UTC m=+5.906663462 container cleanup 9d9827445358b3fa87d639b294da085734c47bfe8a92bc4a61db323843a09a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0) 2024-08-19T01:47:48.524 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:48 smithi190 bash[57886]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8 2024-08-19T01:47:48.880 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:48 smithi190 podman[57902]: 2024-08-19 01:47:48.582487265 +0000 UTC m=+0.630204396 container remove 9d9827445358b3fa87d639b294da085734c47bfe8a92bc4a61db323843a09a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0) 2024-08-19T01:47:48.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:48 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:48.522+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:48.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:48 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:48.647+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:49.245 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:48 smithi190 podman[57992]: 2024-08-19 01:47:48.853504042 +0000 UTC m=+0.017138748 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:49.245 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:48 smithi190 podman[57992]: 2024-08-19 01:47:48.958296089 +0000 UTC m=+0.121930781 container create 1fc7a34d0b58aecbf7dfef3b6d9a55da1c7f7c40e559b033f65b92da98c560cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-19T01:47:49.533 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:49 smithi190 podman[57992]: 2024-08-19 01:47:49.244663186 +0000 UTC m=+0.408297884 container init 1fc7a34d0b58aecbf7dfef3b6d9a55da1c7f7c40e559b033f65b92da98c560cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-19T01:47:49.533 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:49 smithi190 podman[57992]: 2024-08-19 01:47:49.248256447 +0000 UTC m=+0.411891145 container start 1fc7a34d0b58aecbf7dfef3b6d9a55da1c7f7c40e559b033f65b92da98c560cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-19T01:47:49.533 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:49 smithi190 podman[57992]: 2024-08-19 01:47:49.335760815 +0000 UTC m=+0.499395508 container attach 1fc7a34d0b58aecbf7dfef3b6d9a55da1c7f7c40e559b033f65b92da98c560cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-19T01:47:49.533 INFO:journalctl@ceph.osd.8.smithi190.stdout:Aug 19 01:47:49 smithi190 podman[57992]: 2024-08-19 01:47:49.449074756 +0000 UTC m=+0.612709464 container died 1fc7a34d0b58aecbf7dfef3b6d9a55da1c7f7c40e559b033f65b92da98c560cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2) 2024-08-19T01:47:49.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:49 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:49.088+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.674771+0000 front 2024-08-19T01:46:41.674977+0000 (oldest deadline 2024-08-19T01:47:48.675504+0000) 2024-08-19T01:47:49.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:49 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:49.531+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:49.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:49 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:49.598+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:50.066 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.8.service' 2024-08-19T01:47:50.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:50.105 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-19T01:47:50.105 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-19T01:47:50.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.9 2024-08-19T01:47:50.422 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:50 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:50.097+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.674771+0000 front 2024-08-19T01:46:41.674977+0000 (oldest deadline 2024-08-19T01:47:48.675504+0000) 2024-08-19T01:47:50.422 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:50 smithi103 systemd[1]: Stopping Ceph osd.9 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:50.422 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:50 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:50.421+0000 7f588c1da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:50.423 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:50 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:50.421+0000 7f588c1da640 -1 osd.9 89 *** Got signal Terminated *** 2024-08-19T01:47:50.423 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:50 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:50.421+0000 7f588c1da640 -1 osd.9 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:50.630 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:50 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:50.540+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:50.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:50 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:50.553+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:51.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:51 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:51.116+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.674771+0000 front 2024-08-19T01:46:41.674977+0000 (oldest deadline 2024-08-19T01:47:48.675504+0000) 2024-08-19T01:47:51.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:51 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:51.568+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:51.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:51 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:51.512+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:52.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:52 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:52.119+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.674771+0000 front 2024-08-19T01:46:41.674977+0000 (oldest deadline 2024-08-19T01:47:48.675504+0000) 2024-08-19T01:47:52.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:52 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:52.119+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.376774+0000 front 2024-08-19T01:46:50.376701+0000 (oldest deadline 2024-08-19T01:47:52.076758+0000) 2024-08-19T01:47:52.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:52 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:52.595+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:52.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:52 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:52.513+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:53.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:53 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:53.159+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.674771+0000 front 2024-08-19T01:46:41.674977+0000 (oldest deadline 2024-08-19T01:47:48.675504+0000) 2024-08-19T01:47:53.554 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:53 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:53.159+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.376774+0000 front 2024-08-19T01:46:50.376701+0000 (oldest deadline 2024-08-19T01:47:52.076758+0000) 2024-08-19T01:47:53.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:53 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:53.547+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:53.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:53 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:53.547+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:47:53.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:53 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:53.559+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:53.951 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:53 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:53.559+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:47:54.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:54 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:54.192+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.674771+0000 front 2024-08-19T01:46:41.674977+0000 (oldest deadline 2024-08-19T01:47:48.675504+0000) 2024-08-19T01:47:54.554 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:54 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:54.192+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.376774+0000 front 2024-08-19T01:46:50.376701+0000 (oldest deadline 2024-08-19T01:47:52.076758+0000) 2024-08-19T01:47:54.795 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:54 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:54.535+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:54.795 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:54 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:54.535+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:47:54.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:54 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:54.568+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:54.951 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:54 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:54.568+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:47:55.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:55 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:55.203+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.674771+0000 front 2024-08-19T01:46:41.674977+0000 (oldest deadline 2024-08-19T01:47:48.675504+0000) 2024-08-19T01:47:55.554 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:55 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9[52039]: 2024-08-19T01:47:55.203+0000 7f58847e8640 -1 osd.9 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.376774+0000 front 2024-08-19T01:46:50.376701+0000 (oldest deadline 2024-08-19T01:47:52.076758+0000) 2024-08-19T01:47:55.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:55 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:55.580+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:55.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:55 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:55.580+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:47:55.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:55 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:55.520+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:55.951 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:55 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:55.520+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:47:56.048 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:55 smithi103 podman[108522]: 2024-08-19 01:47:55.775327989 +0000 UTC m=+5.463979607 container died 5a8d15fa4ec31e0fee82f260886b0660cedd86101f644aba7a561b8b17c23cc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-08-19T01:47:56.303 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:56 smithi103 podman[108522]: 2024-08-19 01:47:56.2637557 +0000 UTC m=+5.952407362 container cleanup 5a8d15fa4ec31e0fee82f260886b0660cedd86101f644aba7a561b8b17c23cc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-19T01:47:56.303 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:56 smithi103 bash[108522]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9 2024-08-19T01:47:56.803 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:56 smithi103 podman[108538]: 2024-08-19 01:47:56.531925865 +0000 UTC m=+0.757386556 container remove 5a8d15fa4ec31e0fee82f260886b0660cedd86101f644aba7a561b8b17c23cc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-08-19T01:47:56.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:56.618+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:56.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:56.618+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:47:56.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:56 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:56.517+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:56.951 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:56 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:56.517+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:47:57.186 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:56 smithi103 podman[108624]: 2024-08-19 01:47:56.793864841 +0000 UTC m=+0.017404179 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:47:57.187 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:56 smithi103 podman[108624]: 2024-08-19 01:47:56.891196244 +0000 UTC m=+0.114735582 container create 7c59b86d60e59f685f0272b7d29643584e02b0bf7d2d3e7544cf286a9dcb789a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-de09dd0) 2024-08-19T01:47:57.187 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:57 smithi103 podman[108624]: 2024-08-19 01:47:57.102190512 +0000 UTC m=+0.325729860 container init 7c59b86d60e59f685f0272b7d29643584e02b0bf7d2d3e7544cf286a9dcb789a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:47:57.187 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:57 smithi103 podman[108624]: 2024-08-19 01:47:57.105346289 +0000 UTC m=+0.328885629 container start 7c59b86d60e59f685f0272b7d29643584e02b0bf7d2d3e7544cf286a9dcb789a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=) 2024-08-19T01:47:57.187 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:57 smithi103 podman[108624]: 2024-08-19 01:47:57.185449195 +0000 UTC m=+0.408988549 container attach 7c59b86d60e59f685f0272b7d29643584e02b0bf7d2d3e7544cf286a9dcb789a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-19T01:47:57.553 INFO:journalctl@ceph.osd.9.smithi103.stdout:Aug 19 01:47:57 smithi103 podman[108624]: 2024-08-19 01:47:57.307687513 +0000 UTC m=+0.531226852 container died 7c59b86d60e59f685f0272b7d29643584e02b0bf7d2d3e7544cf286a9dcb789a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-19T01:47:57.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:57.583+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:57.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:57 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:57.583+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:47:57.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.9.service' 2024-08-19T01:47:57.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:57 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:57.469+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:57.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:57 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:57.469+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:47:57.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:47:57.970 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-19T01:47:57.970 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-19T01:47:57.970 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.10 2024-08-19T01:47:58.276 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:58 smithi146 systemd[1]: Stopping Ceph osd.10 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:47:58.700 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:58.275+0000 7f4366d15640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:47:58.700 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:58.275+0000 7f4366d15640 -1 osd.10 89 *** Got signal Terminated *** 2024-08-19T01:47:58.701 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:58.275+0000 7f4366d15640 -1 osd.10 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:47:58.701 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:58.431+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:58.701 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:58 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:58.431+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:47:58.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:58.565+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:58.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:58 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:58.565+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:47:59.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:59 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:59.597+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:47:59.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:59 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:59.597+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:47:59.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:47:59 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:47:59.597+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:47:59.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:59.470+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:47:59.951 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:47:59 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:47:59.470+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:48:00.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:00 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:00.591+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:00.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:00 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:00.591+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:00.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:00 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:00.591+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:00.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:48:00.470+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:48:00.950 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:00 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:48:00.470+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:48:01.700 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:01 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:48:01.422+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:48:01.700 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:01 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:48:01.422+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:48:01.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:01 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:01.592+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:01.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:01 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:01.592+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:01.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:01 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:01.592+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:02.700 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:02 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:48:02.418+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.133498+0000 front 2024-08-19T01:46:41.133650+0000 (oldest deadline 2024-08-19T01:47:45.833397+0000) 2024-08-19T01:48:02.701 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:02 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10[46667]: 2024-08-19T01:48:02.418+0000 7f436332b640 -1 osd.10 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:50.434954+0000 front 2024-08-19T01:46:50.435020+0000 (oldest deadline 2024-08-19T01:47:52.735112+0000) 2024-08-19T01:48:02.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:02 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:02.601+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:02.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:02 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:02.601+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:02.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:02 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:02.601+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:03.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:03 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:03.575+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:03.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:03 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:03.575+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:03.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:03 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:03.575+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:03.951 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:03 smithi146 podman[57510]: 2024-08-19 01:48:03.695081933 +0000 UTC m=+5.516044680 container died 88daebee43886adc47adb7a0205d2333807d9cad8661047023a9150344be1dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-19T01:48:04.368 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:04 smithi146 podman[57510]: 2024-08-19 01:48:04.049752434 +0000 UTC m=+5.870715180 container cleanup 88daebee43886adc47adb7a0205d2333807d9cad8661047023a9150344be1dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-08-19T01:48:04.368 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:04 smithi146 bash[57510]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10 2024-08-19T01:48:04.700 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:04 smithi146 podman[57525]: 2024-08-19 01:48:04.367572121 +0000 UTC m=+0.668272682 container remove 88daebee43886adc47adb7a0205d2333807d9cad8661047023a9150344be1dc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:48:04.834 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:04 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:04.534+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:04.834 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:04 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:04.534+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:04.834 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:04 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:04.534+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:05.030 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:04 smithi146 podman[57612]: 2024-08-19 01:48:04.639869317 +0000 UTC m=+0.016707038 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:48:05.030 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:04 smithi146 podman[57612]: 2024-08-19 01:48:04.745656616 +0000 UTC m=+0.122494366 container create fe6beb741e9de8a1c05071a47eb9c5b599157f31c7a4149d56845b788c96a8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-19T01:48:05.030 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:04 smithi146 podman[57612]: 2024-08-19 01:48:04.954170092 +0000 UTC m=+0.331007814 container init fe6beb741e9de8a1c05071a47eb9c5b599157f31c7a4149d56845b788c96a8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0) 2024-08-19T01:48:05.030 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:04 smithi146 podman[57612]: 2024-08-19 01:48:04.957498573 +0000 UTC m=+0.334336284 container start fe6beb741e9de8a1c05071a47eb9c5b599157f31c7a4149d56845b788c96a8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-19T01:48:05.030 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:05 smithi146 podman[57612]: 2024-08-19 01:48:05.02903025 +0000 UTC m=+0.405867976 container attach fe6beb741e9de8a1c05071a47eb9c5b599157f31c7a4149d56845b788c96a8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-19T01:48:05.422 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:05 smithi146 podman[57612]: 2024-08-19 01:48:05.150974131 +0000 UTC m=+0.527811836 container died fe6beb741e9de8a1c05071a47eb9c5b599157f31c7a4149d56845b788c96a8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-08-19T01:48:05.672 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.10.service' 2024-08-19T01:48:05.694 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:05 smithi146 podman[57635]: 2024-08-19 01:48:05.648548534 +0000 UTC m=+0.492114221 container remove fe6beb741e9de8a1c05071a47eb9c5b599157f31c7a4149d56845b788c96a8e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-08-19T01:48:05.695 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:05 smithi146 systemd[1]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.10.service: Deactivated successfully. 2024-08-19T01:48:05.695 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:05 smithi146 systemd[1]: Stopped Ceph osd.10 for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:48:05.695 INFO:journalctl@ceph.osd.10.smithi146.stdout:Aug 19 01:48:05 smithi146 systemd[1]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.10.service: Consumed 1min 16.512s CPU time. 2024-08-19T01:48:05.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:05.710 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-19T01:48:05.710 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-19T01:48:05.710 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.11 2024-08-19T01:48:05.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:05 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:05.513+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:05.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:05 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:05.513+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:05.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:05 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:05.513+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:05.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:05 smithi190 systemd[1]: Stopping Ceph osd.11 for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:06.380 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:06 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:06.025+0000 7fad4daad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:06.380 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:06 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:06.025+0000 7fad4daad640 -1 osd.11 89 *** Got signal Terminated *** 2024-08-19T01:48:06.380 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:06 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:06.025+0000 7fad4daad640 -1 osd.11 89 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-19T01:48:06.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:06 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:06.512+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:07.285 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:06 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:06.512+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:07.285 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:06 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:06.512+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:07.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:07 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:07.524+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:07.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:07 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:07.524+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:07.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:07 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:07.524+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:08.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:08 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:08.507+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:08.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:08 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:08.507+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:08.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:08 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:08.507+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:09.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:09 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:09.509+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:09.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:09 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:09.509+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:09.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:09 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:09.509+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:09.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:09 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:09.509+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-08-19T01:47:03.542388+0000 front 2024-08-19T01:47:03.542520+0000 (oldest deadline 2024-08-19T01:48:08.842389+0000) 2024-08-19T01:48:10.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:10 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:10.539+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-08-19T01:46:41.538542+0000 front 2024-08-19T01:46:41.538619+0000 (oldest deadline 2024-08-19T01:47:43.238619+0000) 2024-08-19T01:48:10.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:10 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:10.539+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-08-19T01:46:48.940580+0000 front 2024-08-19T01:46:48.940507+0000 (oldest deadline 2024-08-19T01:47:53.040386+0000) 2024-08-19T01:48:10.880 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:10 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:10.539+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-08-19T01:46:54.141191+0000 front 2024-08-19T01:46:54.141138+0000 (oldest deadline 2024-08-19T01:47:59.441310+0000) 2024-08-19T01:48:10.881 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:10 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11[46988]: 2024-08-19T01:48:10.539+0000 7fad460bb640 -1 osd.11 89 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-08-19T01:47:03.542388+0000 front 2024-08-19T01:47:03.542520+0000 (oldest deadline 2024-08-19T01:48:08.842389+0000) 2024-08-19T01:48:11.776 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:11 smithi190 podman[58109]: 2024-08-19 01:48:11.501149454 +0000 UTC m=+5.591012067 container died 27496b95acd8c3f1fe7d3363dc590b5e9609b8d29bb0279a51032abae025df8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-19T01:48:12.126 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:11 smithi190 podman[58109]: 2024-08-19 01:48:11.874746822 +0000 UTC m=+5.964609441 container cleanup 27496b95acd8c3f1fe7d3363dc590b5e9609b8d29bb0279a51032abae025df8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0) 2024-08-19T01:48:12.126 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:11 smithi190 bash[58109]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11 2024-08-19T01:48:12.380 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:12 smithi190 podman[58125]: 2024-08-19 01:48:12.125620884 +0000 UTC m=+0.626526060 container remove 27496b95acd8c3f1fe7d3363dc590b5e9609b8d29bb0279a51032abae025df8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-19T01:48:12.710 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:12 smithi190 podman[58214]: 2024-08-19 01:48:12.358790264 +0000 UTC m=+0.017115076 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-19T01:48:12.710 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:12 smithi190 podman[58214]: 2024-08-19 01:48:12.484779685 +0000 UTC m=+0.143104529 container create 863b5e78f6738b7b9af09eb8178ad0663d8dbce76f9cefec11f7f88f9007a433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-19T01:48:12.710 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:12 smithi190 podman[58214]: 2024-08-19 01:48:12.705960428 +0000 UTC m=+0.364285239 container init 863b5e78f6738b7b9af09eb8178ad0663d8dbce76f9cefec11f7f88f9007a433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-19T01:48:12.710 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:12 smithi190 podman[58214]: 2024-08-19 01:48:12.709794417 +0000 UTC m=+0.368119217 container start 863b5e78f6738b7b9af09eb8178ad0663d8dbce76f9cefec11f7f88f9007a433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:48:13.130 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:12 smithi190 podman[58214]: 2024-08-19 01:48:12.805954196 +0000 UTC m=+0.464279004 container attach 863b5e78f6738b7b9af09eb8178ad0663d8dbce76f9cefec11f7f88f9007a433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-19T01:48:13.130 INFO:journalctl@ceph.osd.11.smithi190.stdout:Aug 19 01:48:12 smithi190 podman[58214]: 2024-08-19 01:48:12.896870189 +0000 UTC m=+0.555194991 container died 863b5e78f6738b7b9af09eb8178ad0663d8dbce76f9cefec11f7f88f9007a433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-19T01:48:13.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@osd.11.service' 2024-08-19T01:48:13.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:13.571 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-19T01:48:13.571 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-19T01:48:13.571 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.a 2024-08-19T01:48:13.901 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:48:13 smithi103 systemd[1]: Stopping Ceph mds.a for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:14.303 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:48:13 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a[58670]: 2024-08-19T01:48:13.899+0000 7f94c4c3c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:14.304 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:48:13 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a[58670]: 2024-08-19T01:48:13.899+0000 7f94c4c3c640 -1 mds.a *** got signal Terminated *** 2024-08-19T01:48:15.553 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:48:15 smithi103 podman[108727]: 2024-08-19 01:48:15.281206755 +0000 UTC m=+1.508334462 container died a0704be86237a300581fe5d4f77bdcef0dfde24ff66694a29bc26afad192004f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-08-19T01:48:16.051 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:48:15 smithi103 podman[108727]: 2024-08-19 01:48:15.758332792 +0000 UTC m=+1.985460499 container cleanup a0704be86237a300581fe5d4f77bdcef0dfde24ff66694a29bc26afad192004f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-19T01:48:16.052 INFO:journalctl@ceph.mds.a.smithi103.stdout:Aug 19 01:48:15 smithi103 bash[108727]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-a 2024-08-19T01:48:16.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.a.service' 2024-08-19T01:48:16.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:16.215 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-19T01:48:16.215 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-19T01:48:16.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.d 2024-08-19T01:48:16.303 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:48:16 smithi103 systemd[1]: Stopping Ceph mds.d for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:17.053 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:48:16 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d[57487]: 2024-08-19T01:48:16.561+0000 7fe55cf62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:17.054 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:48:16 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d[57487]: 2024-08-19T01:48:16.561+0000 7fe55cf62640 -1 mds.d *** got signal Terminated *** 2024-08-19T01:48:20.554 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:48:20 smithi103 podman[108848]: 2024-08-19 01:48:20.259952534 +0000 UTC m=+3.821140019 container died d5bbd1a5cc5e01108766096ccc75780bdeedc4bcfb11306b733b6cc80135ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-19T01:48:21.053 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:48:20 smithi103 podman[108848]: 2024-08-19 01:48:20.838425972 +0000 UTC m=+4.399613488 container cleanup d5bbd1a5cc5e01108766096ccc75780bdeedc4bcfb11306b733b6cc80135ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0) 2024-08-19T01:48:21.053 INFO:journalctl@ceph.mds.d.smithi103.stdout:Aug 19 01:48:20 smithi103 bash[108848]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-d 2024-08-19T01:48:21.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.d.service' 2024-08-19T01:48:21.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:21.313 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-19T01:48:21.314 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-19T01:48:21.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.g 2024-08-19T01:48:21.660 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:48:21 smithi103 systemd[1]: Stopping Ceph mds.g for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:22.053 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:48:21 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g[58252]: 2024-08-19T01:48:21.659+0000 7f698a8ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:22.053 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:48:21 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g[58252]: 2024-08-19T01:48:21.659+0000 7f698a8ef640 -1 mds.g *** got signal Terminated *** 2024-08-19T01:48:23.554 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:48:23 smithi103 podman[108984]: 2024-08-19 01:48:23.275759302 +0000 UTC m=+1.723988965 container died 6e180b34a10d55bb651d3908704337e6898bc1a869a9f2399ef5e665715adc9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-19T01:48:24.020 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:48:23 smithi103 podman[108984]: 2024-08-19 01:48:23.759991843 +0000 UTC m=+2.208221551 container cleanup 6e180b34a10d55bb651d3908704337e6898bc1a869a9f2399ef5e665715adc9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:48:24.020 INFO:journalctl@ceph.mds.g.smithi103.stdout:Aug 19 01:48:23 smithi103 bash[108984]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-g 2024-08-19T01:48:24.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.g.service' 2024-08-19T01:48:24.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:24.188 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-19T01:48:24.188 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-19T01:48:24.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.j 2024-08-19T01:48:24.303 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:48:24 smithi103 systemd[1]: Stopping Ceph mds.j for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:24.803 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:48:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j[57071]: 2024-08-19T01:48:24.522+0000 7f4e474d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:24.803 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:48:24 smithi103 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j[57071]: 2024-08-19T01:48:24.522+0000 7f4e474d7640 -1 mds.j *** got signal Terminated *** 2024-08-19T01:48:28.803 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:48:28 smithi103 podman[109106]: 2024-08-19 01:48:28.536515377 +0000 UTC m=+4.122537580 container died 72e1d99557ae287c8e81c8af9897cb26085e245303ad1f41d9091d4a07b21fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-19T01:48:29.191 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:48:28 smithi103 podman[109106]: 2024-08-19 01:48:28.956368555 +0000 UTC m=+4.542390756 container cleanup 72e1d99557ae287c8e81c8af9897cb26085e245303ad1f41d9091d4a07b21fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-19T01:48:29.191 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:48:28 smithi103 bash[109106]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j 2024-08-19T01:48:29.451 INFO:journalctl@ceph.mds.j.smithi103.stdout:Aug 19 01:48:29 smithi103 podman[109119]: 2024-08-19 01:48:29.190887896 +0000 UTC m=+0.656998736 container remove 72e1d99557ae287c8e81c8af9897cb26085e245303ad1f41d9091d4a07b21fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-19T01:48:29.455 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.j.service' 2024-08-19T01:48:29.488 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:29.489 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-19T01:48:29.489 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-19T01:48:29.489 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.b 2024-08-19T01:48:29.800 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:48:29 smithi146 systemd[1]: Stopping Ceph mds.b for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:30.200 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:48:29 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b[52184]: 2024-08-19T01:48:29.798+0000 7f1c9e0eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:30.200 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:48:29 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b[52184]: 2024-08-19T01:48:29.798+0000 7f1c9e0eb640 -1 mds.b *** got signal Terminated *** 2024-08-19T01:48:31.441 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:48:31 smithi146 podman[57727]: 2024-08-19 01:48:31.150571451 +0000 UTC m=+1.459434894 container died 38f4334f45c1dc1d93f6d3b9c4b2c58fe0e1025c3447ea0b12a2ab70ad30de26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:48:31.700 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:48:31 smithi146 podman[57727]: 2024-08-19 01:48:31.540410795 +0000 UTC m=+1.849274232 container cleanup 38f4334f45c1dc1d93f6d3b9c4b2c58fe0e1025c3447ea0b12a2ab70ad30de26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0) 2024-08-19T01:48:31.701 INFO:journalctl@ceph.mds.b.smithi146.stdout:Aug 19 01:48:31 smithi146 bash[57727]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-b 2024-08-19T01:48:31.944 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.b.service' 2024-08-19T01:48:31.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:31.978 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-19T01:48:31.978 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-19T01:48:31.978 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.e 2024-08-19T01:48:32.450 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:48:32 smithi146 systemd[1]: Stopping Ceph mds.e for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:32.451 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:48:32 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e[51575]: 2024-08-19T01:48:32.309+0000 7fc0db857640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:32.451 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:48:32 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e[51575]: 2024-08-19T01:48:32.309+0000 7fc0db857640 -1 mds.e *** got signal Terminated *** 2024-08-19T01:48:35.450 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:48:35 smithi146 podman[57847]: 2024-08-19 01:48:35.157675091 +0000 UTC m=+2.948256587 container died 20f8936e6c78076affcb7bfc986973d7611782e749ca20cac19960ebeac7cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-19T01:48:35.875 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:48:35 smithi146 podman[57847]: 2024-08-19 01:48:35.485145684 +0000 UTC m=+3.275727169 container cleanup 20f8936e6c78076affcb7bfc986973d7611782e749ca20cac19960ebeac7cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-19T01:48:35.876 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:48:35 smithi146 bash[57847]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e 2024-08-19T01:48:35.876 INFO:journalctl@ceph.mds.e.smithi146.stdout:Aug 19 01:48:35 smithi146 podman[57861]: 2024-08-19 01:48:35.652795537 +0000 UTC m=+0.497546872 container remove 20f8936e6c78076affcb7bfc986973d7611782e749ca20cac19960ebeac7cffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-19T01:48:35.890 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.e.service' 2024-08-19T01:48:35.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:35.923 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-19T01:48:35.924 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-19T01:48:35.924 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.h 2024-08-19T01:48:36.200 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:48:36 smithi146 systemd[1]: Stopping Ceph mds.h for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:36.700 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:48:36 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h[51880]: 2024-08-19T01:48:36.287+0000 7f83718c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:36.700 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:48:36 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h[51880]: 2024-08-19T01:48:36.287+0000 7f83718c0640 -1 mds.h *** got signal Terminated *** 2024-08-19T01:48:39.450 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:48:39 smithi146 podman[57967]: 2024-08-19 01:48:39.151791593 +0000 UTC m=+2.991723188 container died 07cc3b08b067427acde2bdfeca23704746cb12ca58dffc21b0d7102f59f68a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, io.buildah.version=1.36.0) 2024-08-19T01:48:39.700 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:48:39 smithi146 podman[57967]: 2024-08-19 01:48:39.546881488 +0000 UTC m=+3.386813082 container cleanup 07cc3b08b067427acde2bdfeca23704746cb12ca58dffc21b0d7102f59f68a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-19T01:48:39.701 INFO:journalctl@ceph.mds.h.smithi146.stdout:Aug 19 01:48:39 smithi146 bash[57967]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-h 2024-08-19T01:48:39.925 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.h.service' 2024-08-19T01:48:39.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:39.960 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-19T01:48:39.960 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-19T01:48:39.960 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.k 2024-08-19T01:48:40.450 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:48:40 smithi146 systemd[1]: Stopping Ceph mds.k for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:40.451 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:48:40 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k[51264]: 2024-08-19T01:48:40.291+0000 7faebbd9d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:40.451 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:48:40 smithi146 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k[51264]: 2024-08-19T01:48:40.291+0000 7faebbd9d640 -1 mds.k *** got signal Terminated *** 2024-08-19T01:48:43.435 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:48:43 smithi146 podman[58098]: 2024-08-19 01:48:43.145686072 +0000 UTC m=+2.958782929 container died 28351662d1da3c6cb4b6609203e6b3f67ada978448eb81b81407be42ac70ae7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0) 2024-08-19T01:48:43.700 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:48:43 smithi146 podman[58098]: 2024-08-19 01:48:43.508274552 +0000 UTC m=+3.321371407 container cleanup 28351662d1da3c6cb4b6609203e6b3f67ada978448eb81b81407be42ac70ae7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-19T01:48:43.701 INFO:journalctl@ceph.mds.k.smithi146.stdout:Aug 19 01:48:43 smithi146 bash[58098]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-k 2024-08-19T01:48:43.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.k.service' 2024-08-19T01:48:43.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:43.983 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-19T01:48:43.983 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-19T01:48:43.983 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.c 2024-08-19T01:48:44.301 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:48:44 smithi190 systemd[1]: Stopping Ceph mds.c for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:44.559 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:48:44 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c[52219]: 2024-08-19T01:48:44.300+0000 7f4feb319640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:44.560 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:48:44 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c[52219]: 2024-08-19T01:48:44.300+0000 7f4feb319640 -1 mds.c *** got signal Terminated *** 2024-08-19T01:48:47.418 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:48:47 smithi190 podman[58316]: 2024-08-19 01:48:47.155274223 +0000 UTC m=+2.970671123 container died d641ac39669c432f3e100214683c74e5915403b452948d798ce0f98f192af1dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-08-19T01:48:47.793 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:48:47 smithi190 podman[58316]: 2024-08-19 01:48:47.517449129 +0000 UTC m=+3.332846075 container cleanup d641ac39669c432f3e100214683c74e5915403b452948d798ce0f98f192af1dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0) 2024-08-19T01:48:47.794 INFO:journalctl@ceph.mds.c.smithi190.stdout:Aug 19 01:48:47 smithi190 bash[58316]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-c 2024-08-19T01:48:47.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.c.service' 2024-08-19T01:48:47.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:47.973 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-19T01:48:47.973 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-19T01:48:47.973 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.f 2024-08-19T01:48:48.380 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:48 smithi190 systemd[1]: Stopping Ceph mds.f for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:48.380 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:48 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f[51916]: 2024-08-19T01:48:48.295+0000 7fa575178640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:48.380 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:48 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f[51916]: 2024-08-19T01:48:48.295+0000 7fa575178640 -1 mds.f *** got signal Terminated *** 2024-08-19T01:48:51.480 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:51 smithi190 podman[58448]: 2024-08-19 01:48:51.155046067 +0000 UTC m=+2.961829431 container died 151cd8713b53ba4323a2d8775ecf455a54a3935b092ec0e658fdb8a85d02a5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-19T01:48:51.772 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:51 smithi190 podman[58448]: 2024-08-19 01:48:51.579031399 +0000 UTC m=+3.385814773 container cleanup 151cd8713b53ba4323a2d8775ecf455a54a3935b092ec0e658fdb8a85d02a5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0) 2024-08-19T01:48:51.772 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:51 smithi190 bash[58448]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f 2024-08-19T01:48:52.021 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.f.service' 2024-08-19T01:48:52.043 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:51 smithi190 podman[58460]: 2024-08-19 01:48:51.771825576 +0000 UTC m=+0.615133511 container remove 151cd8713b53ba4323a2d8775ecf455a54a3935b092ec0e658fdb8a85d02a5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-19T01:48:52.043 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:52 smithi190 systemd[1]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.f.service: Deactivated successfully. 2024-08-19T01:48:52.043 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:52 smithi190 systemd[1]: Stopped Ceph mds.f for e721016a-5dc9-11ef-bccf-c7b262605968. 2024-08-19T01:48:52.043 INFO:journalctl@ceph.mds.f.smithi190.stdout:Aug 19 01:48:52 smithi190 systemd[1]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.f.service: Consumed 2.477s CPU time. 2024-08-19T01:48:52.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:52.055 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-19T01:48:52.055 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-19T01:48:52.055 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.i 2024-08-19T01:48:52.380 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:48:52 smithi190 systemd[1]: Stopping Ceph mds.i for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:52.880 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:48:52 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i[52519]: 2024-08-19T01:48:52.382+0000 7f697f94e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:52.880 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:48:52 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i[52519]: 2024-08-19T01:48:52.382+0000 7f697f94e640 -1 mds.i *** got signal Terminated *** 2024-08-19T01:48:55.380 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:48:55 smithi190 podman[58567]: 2024-08-19 01:48:55.148267926 +0000 UTC m=+2.863122337 container died 1848559706c10499b7b6ce48c6602088d572baf1d5d87b9ed6d81ab3caed8fe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-19T01:48:55.793 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:48:55 smithi190 podman[58567]: 2024-08-19 01:48:55.541317511 +0000 UTC m=+3.256171931 container cleanup 1848559706c10499b7b6ce48c6602088d572baf1d5d87b9ed6d81ab3caed8fe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-19T01:48:55.793 INFO:journalctl@ceph.mds.i.smithi190.stdout:Aug 19 01:48:55 smithi190 bash[58567]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-i 2024-08-19T01:48:55.929 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.i.service' 2024-08-19T01:48:55.960 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:55.961 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-19T01:48:55.961 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-19T01:48:55.961 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.l 2024-08-19T01:48:56.130 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:48:56 smithi190 systemd[1]: Stopping Ceph mds.l for e721016a-5dc9-11ef-bccf-c7b262605968... 2024-08-19T01:48:56.630 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:48:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l[51604]: 2024-08-19T01:48:56.286+0000 7ff02cdd8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-19T01:48:56.631 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:48:56 smithi190 ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l[51604]: 2024-08-19T01:48:56.286+0000 7ff02cdd8640 -1 mds.l *** got signal Terminated *** 2024-08-19T01:48:59.429 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:48:59 smithi190 podman[58689]: 2024-08-19 01:48:59.15344381 +0000 UTC m=+2.970087866 container died 9f525248d217e1ac8cebc14c8caf8447e66149e987555c978fbf95bf50baab3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-08-19T01:48:59.779 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:48:59 smithi190 podman[58689]: 2024-08-19 01:48:59.503078089 +0000 UTC m=+3.319722145 container cleanup 9f525248d217e1ac8cebc14c8caf8447e66149e987555c978fbf95bf50baab3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-19T01:48:59.779 INFO:journalctl@ceph.mds.l.smithi190.stdout:Aug 19 01:48:59 smithi190 bash[58689]: ceph-e721016a-5dc9-11ef-bccf-c7b262605968-mds-l 2024-08-19T01:48:59.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e721016a-5dc9-11ef-bccf-c7b262605968@mds.l.service' 2024-08-19T01:48:59.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-19T01:48:59.973 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-19T01:48:59.973 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e721016a-5dc9-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-19T01:49:00.179 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:49:02.799 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e721016a-5dc9-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-19T01:49:03.003 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:49:05.386 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e721016a-5dc9-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-19T01:49:05.590 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:49:07.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:49:08.010 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:49:08.039 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-19T01:49:08.066 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-19T01:49:08.068 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi103/crash 2024-08-19T01:49:08.069 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash -- . 2024-08-19T01:49:08.109 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-19T01:49:08.110 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-08-19T01:49:08.112 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi146/crash 2024-08-19T01:49:08.113 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash -- . 2024-08-19T01:49:08.147 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-19T01:49:08.147 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2024-08-19T01:49:08.150 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi190/crash 2024-08-19T01:49:08.151 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash -- . 2024-08-19T01:49:08.210 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-19T01:49:08.211 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-08-19T01:49:08.212 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-19T01:49:08.213 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-19T01:49:08.307 INFO:tasks.cephadm:Compressing logs... 2024-08-19T01:49:08.308 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T01:49:08.310 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T01:49:08.312 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-19T01:49:08.336 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-19T01:49:08.336 INFO:teuthology.orchestra.run.smithi146.stderr:: No such file or directory 2024-08-19T01:49:08.336 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-19T01:49:08.336 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.e.log 2024-08-19T01:49:08.336 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.h.log 2024-08-19T01:49:08.344 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-19T01:49:08.344 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-19T01:49:08.344 INFO:teuthology.orchestra.run.smithi146.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-19T01:49:08.344 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.b.log 2024-08-19T01:49:08.344 INFO:teuthology.orchestra.run.smithi146.stderr: 82.7% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-19T01:49:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.10.log 2024-08-19T01:49:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.4.log 2024-08-19T01:49:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.k.log 2024-08-19T01:49:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.4.log: 90.5% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-19T01:49:08.345 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.b.log 2024-08-19T01:49:08.347 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:: No such file or directory 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.z.log 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.2.log 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.11.log 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.f.log 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.2.log: /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-19T01:49:08.348 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.8.log 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr: 82.7% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.i.log 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.c.log 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.5.log 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.c.log: 93.0% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.z.log.gz 2024-08-19T01:49:08.349 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log 2024-08-19T01:49:08.350 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.5.log: 90.9% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-19T01:49:08.351 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.k.log: 90.9% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-19T01:49:08.351 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.y.log 2024-08-19T01:49:08.352 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-19T01:49:08.352 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-19T01:49:08.353 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-08-19T01:49:08.353 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-19T01:49:08.353 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.3.log 2024-08-19T01:49:08.354 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.d.log 2024-08-19T01:49:08.354 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log: /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.x.log 2024-08-19T01:49:08.355 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-19T01:49:08.355 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-19T01:49:08.355 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.a.log 2024-08-19T01:49:08.357 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.a.log 2024-08-19T01:49:08.357 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.9.log 2024-08-19T01:49:08.357 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.6.log 2024-08-19T01:49:08.358 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.j.log 2024-08-19T01:49:08.358 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.0.log 2024-08-19T01:49:08.358 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log 2024-08-19T01:49:08.358 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.g.log 2024-08-19T01:49:08.358 INFO:teuthology.orchestra.run.smithi103.stderr: 94.8% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.d.log.gz 2024-08-19T01:49:08.359 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.c.log 2024-08-19T01:49:08.360 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log: /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.g.log: 86.9% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-19T01:49:08.360 INFO:teuthology.orchestra.run.smithi103.stderr: 94.8% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.j.log.gz 2024-08-19T01:49:08.361 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log: 86.9% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-19T01:49:08.362 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.7.log 2024-08-19T01:49:08.370 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.l.log 2024-08-19T01:49:08.377 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.y.log: 92.9%gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.1.log 2024-08-19T01:49:08.378 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.y.log.gz 2024-08-19T01:49:08.384 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log 2024-08-19T01:49:08.419 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.1.log: /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log: 86.9% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-19T01:49:08.424 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.k.log.gz 2024-08-19T01:49:08.560 INFO:teuthology.orchestra.run.smithi103.stderr: 82.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-19T01:49:08.604 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.c.log: /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.l.log: 93.0% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.f.log.gz 2024-08-19T01:49:08.605 INFO:teuthology.orchestra.run.smithi103.stderr: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-19T01:49:08.621 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.l.log.gz 2024-08-19T01:49:08.634 INFO:teuthology.orchestra.run.smithi103.stderr: 91.0% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-19T01:49:08.678 INFO:teuthology.orchestra.run.smithi190.stderr: 93.5% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.c.log.gz 2024-08-19T01:49:08.777 INFO:teuthology.orchestra.run.smithi146.stderr: 94.6% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.e.log.gz 2024-08-19T01:49:08.794 INFO:teuthology.orchestra.run.smithi103.stderr: 90.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-19T01:49:08.824 INFO:teuthology.orchestra.run.smithi190.stderr: 91.7% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.c.log.gz 2024-08-19T01:49:08.832 INFO:teuthology.orchestra.run.smithi146.stderr: 93.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.h.log.gz 2024-08-19T01:49:08.881 INFO:teuthology.orchestra.run.smithi146.stderr: 91.7% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.b.log.gz 2024-08-19T01:49:09.019 INFO:teuthology.orchestra.run.smithi103.stderr: 93.9% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.a.log.gz 2024-08-19T01:49:09.124 INFO:teuthology.orchestra.run.smithi103.stderr: 90.1% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mgr.x.log.gz 2024-08-19T01:49:09.142 INFO:teuthology.orchestra.run.smithi103.stderr: 93.6% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.g.log.gz 2024-08-19T01:49:09.369 INFO:teuthology.orchestra.run.smithi190.stderr: 93.1% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.i.log.gz 2024-08-19T01:49:11.428 INFO:teuthology.orchestra.run.smithi146.stderr: 93.7% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mds.b.log.gz 2024-08-19T01:49:17.014 INFO:teuthology.orchestra.run.smithi103.stderr: 90.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-mon.a.log.gz 2024-08-19T01:49:23.678 INFO:teuthology.orchestra.run.smithi190.stderr: 91.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.8.log.gz 2024-08-19T01:49:24.565 INFO:teuthology.orchestra.run.smithi146.stderr: 91.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.4.log.gz 2024-08-19T01:49:25.326 INFO:teuthology.orchestra.run.smithi190.stderr: 91.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.2.log.gz 2024-08-19T01:49:26.090 INFO:teuthology.orchestra.run.smithi190.stderr: 91.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.11.log.gz 2024-08-19T01:49:26.097 INFO:teuthology.orchestra.run.smithi146.stderr: 91.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.10.log.gz 2024-08-19T01:49:26.239 INFO:teuthology.orchestra.run.smithi146.stderr: 91.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.7.log.gz 2024-08-19T01:49:27.449 INFO:teuthology.orchestra.run.smithi146.stderr: 91.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.1.log.gz 2024-08-19T01:49:27.452 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-08-19T01:49:27.452 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m19.127s 2024-08-19T01:49:27.452 INFO:teuthology.orchestra.run.smithi146.stderr:user 1m12.781s 2024-08-19T01:49:27.452 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m2.262s 2024-08-19T01:49:27.531 INFO:teuthology.orchestra.run.smithi190.stderr: 91.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.5.log.gz 2024-08-19T01:49:27.533 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-08-19T01:49:27.533 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m19.207s 2024-08-19T01:49:27.533 INFO:teuthology.orchestra.run.smithi190.stderr:user 1m9.121s 2024-08-19T01:49:27.534 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m2.169s 2024-08-19T01:49:35.813 INFO:teuthology.orchestra.run.smithi103.stderr: 91.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.3.log.gz 2024-08-19T01:49:39.502 INFO:teuthology.orchestra.run.smithi103.stderr: 91.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.9.log.gz 2024-08-19T01:49:55.315 INFO:teuthology.orchestra.run.smithi103.stderr: 91.3% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.6.log.gz 2024-08-19T01:50:02.807 INFO:teuthology.orchestra.run.smithi103.stderr: 91.4% -- replaced with /var/log/ceph/e721016a-5dc9-11ef-bccf-c7b262605968/ceph-osd.0.log.gz 2024-08-19T01:51:50.359 INFO:teuthology.orchestra.run.smithi103.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-08-19T01:51:50.361 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-08-19T01:51:50.361 INFO:teuthology.orchestra.run.smithi103.stderr:real 2m42.035s 2024-08-19T01:51:50.361 INFO:teuthology.orchestra.run.smithi103.stderr:user 2m58.708s 2024-08-19T01:51:50.361 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m7.667s 2024-08-19T01:51:50.362 INFO:tasks.cephadm:Archiving logs... 2024-08-19T01:51:50.362 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi103/log 2024-08-19T01:51:50.363 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T01:52:15.874 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi146/log 2024-08-19T01:52:15.876 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T01:52:28.123 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi190/log 2024-08-19T01:52:28.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T01:52:40.322 INFO:tasks.cephadm:Removing cluster... 2024-08-19T01:52:40.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e721016a-5dc9-11ef-bccf-c7b262605968 --force 2024-08-19T01:52:40.527 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:52:40.884 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e721016a-5dc9-11ef-bccf-c7b262605968 --force 2024-08-19T01:52:41.090 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:52:41.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e721016a-5dc9-11ef-bccf-c7b262605968 --force 2024-08-19T01:52:41.731 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: e721016a-5dc9-11ef-bccf-c7b262605968 2024-08-19T01:52:42.101 INFO:tasks.cephadm:Removing cephadm ... 2024-08-19T01:52:42.101 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T01:52:42.121 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T01:52:42.138 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T01:52:42.155 INFO:tasks.cephadm:Teardown complete 2024-08-19T01:52:42.155 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-19T01:52:42.289 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-19T01:52:42.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T01:52:42.293 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T01:52:42.296 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T01:52:42.409 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-19T01:52:42.449 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-19T01:52:42.450 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:52:42.453 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:52:42.456 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T01:52:42.468 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:52:42.469 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:52:42.471 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-08-19T01:52:42.472 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:52:42.473 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-08-19T01:52:42.473 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +271us[ +288us] +/- 47ms 2024-08-19T01:52:42.473 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -273us[ -256us] +/- 83ms 2024-08-19T01:52:42.473 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -81us[ -64us] +/- 85ms 2024-08-19T01:52:42.473 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:52:42.475 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:52:42.475 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-08-19T01:52:42.475 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +259us[ +258us] +/- 47ms 2024-08-19T01:52:42.475 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -315us[ -315us] +/- 83ms 2024-08-19T01:52:42.475 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -130us[ -130us] +/- 85ms 2024-08-19T01:52:42.475 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:52:42.476 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T01:52:42.477 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-08-19T01:52:42.477 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +256us[ +262us] +/- 47ms 2024-08-19T01:52:42.477 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -301us[ -301us] +/- 83ms 2024-08-19T01:52:42.477 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -138us[ -132us] +/- 85ms 2024-08-19T01:52:42.477 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T01:52:42.477 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-19T01:52:42.491 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-19T01:52:42.492 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-19T01:52:42.512 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T01:52:42.601 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T01:52:42.659 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T01:52:42.712 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:52:42.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:52:42.743 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:52:42.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:52:42.772 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T01:52:42.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:52:42.799 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-19T01:52:42.810 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-19T01:52:42.822 INFO:teuthology.task.internal:Duration was 1844.144015 seconds 2024-08-19T01:52:42.822 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-19T01:52:42.832 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-19T01:52:42.832 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T01:52:42.835 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T01:52:42.838 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T01:52:42.876 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:52:42.878 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:52:42.893 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T01:52:43.352 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-19T01:52:43.352 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-08-19T01:52:43.353 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T01:52:43.374 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-08-19T01:52:43.374 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T01:52:43.401 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-08-19T01:52:43.401 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T01:52:43.424 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-19T01:52:43.424 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T01:52:43.427 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T01:52:43.444 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T01:52:43.491 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-19T01:52:43.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T01:52:43.495 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T01:52:43.512 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T01:52:44.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-19T01:52:44.478 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-19T01:52:44.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T01:52:44.504 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T01:52:44.535 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T01:52:44.563 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-19T01:52:44.574 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:52:44.577 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:52:44.580 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:52:44.605 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-08-19T01:52:44.610 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-08-19T01:52:44.631 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-08-19T01:52:44.654 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:52:44.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:52:44.696 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:52:44.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:52:44.713 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T01:52:44.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T01:52:44.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-19T01:52:44.740 INFO:teuthology.task.internal:Transferring archived files... 2024-08-19T01:52:44.741 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi103 2024-08-19T01:52:44.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T01:52:44.794 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi146 2024-08-19T01:52:44.794 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T01:52:44.830 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860121/remote/smithi190 2024-08-19T01:52:44.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T01:52:44.866 INFO:teuthology.task.internal:Removing archive directory... 2024-08-19T01:52:44.867 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T01:52:44.869 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T01:52:44.872 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T01:52:44.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-19T01:52:44.930 INFO:teuthology.task.internal:Not uploading archives. 2024-08-19T01:52:44.930 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-19T01:52:44.941 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-19T01:52:44.941 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T01:52:44.943 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T01:52:44.946 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T01:52:44.956 INFO:teuthology.orchestra.run.smithi103.stdout: 262218 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 01:52 /home/ubuntu/cephtest 2024-08-19T01:52:44.960 INFO:teuthology.orchestra.run.smithi146.stdout: 262218 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 01:52 /home/ubuntu/cephtest 2024-08-19T01:52:44.974 INFO:teuthology.orchestra.run.smithi190.stdout: 262218 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 01:52 /home/ubuntu/cephtest 2024-08-19T01:52:44.975 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-19T01:52:44.986 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-19T01:52:45.013 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} duration: 1844.1440150737762 failure_reason: 'Command failed (workunit test suites/ffsb.sh) on smithi103 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=78215f5a1cf5eaf321d506a66539942e12851f34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=7ac1a3b0f2a044d184cbc21944e8e37b status: fail success: false 2024-08-19T01:52:45.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T01:52:45.089 INFO:teuthology.run:FAIL